Native transactions

It was interesting reading The Power of Native Transactions presentation, which could probably be subtitled: When You Don't Own a Transaction Manager, What Do You Do? Actually I don't mean to pick on Spring as this is a problem we used to come across many times when an independent TM vendor. Typically we'd hear "Aren't the transactions in my database good enough?", "Why do I need recovery?" and that old one but good one "I'm not interested in distributed transactions so I don't need two-phase commit."

I've said enough times that traditional transactions (aka ACID) aren't suitable for every occasion. The fact they have been shoehorned into situations where they aren't needed or aren't applicable has hurt the industry as well as disillusioned some people. Probably as a result, as we saw throughout the BTP effort people equate two-phase commit with ACID semantics, which is clearly wrong. Or that the two phases have to somehow be tied in with prepare/commit/rollback, which as BTP (and others) showed is again wrong.

At some point I need to finish my paper on consensus in distributed systems, because it's fairly obvious that some people who see transactions (and specifically 2PC) as "evil" and something that can be "programmed around" don't fully understand what's happening under the covers of any good transaction manager. Yes it's possible that some uses of transactions can be redesigned without them, but that doesn't mean they all can be.

Anyway, back to the topic at hand. I read the presentation with interest because it's always good to hear experiences from using transaction systems. What follows is almost a review of the presentation as if I had to review it for inclusion (or not) in a conference. I blame the fact that I sit on so many program committees and review a lot of papers each year: it alters your mindset a bit.

First I agree with one of the core messages in the Spring paper: that sometimes you don't need transactions at all and that one-size doesn't fit all. But sometimes you do need a good transaction manager and you need to understand when and why.

It would have been good to see a discussion of local and global transactions, but then I suppose the point of the talk is to emphasize the use of local transactions (through a native transaction manager). What I find frustrating is that yet again someone equates JDBC save points with nested transactions. Yes you can map a save point to a transaction boundary, but you that doesn't give you nested commit semantics or concurrency control, for instance. If you're going to talk about nested transactions, please stick with the standard definition. Even OTS managed to do that, despite breaking the model in other ways.

I know (hope) that the JBossTS team are going to say a thing or two about the common misunderstandings and misconceptions around XA, so I'll try not to steal their thunder. But I do need to point out that the read-only flag is neither a Spring invention nor something only found in specific databases: it's a core part of two-phase commit (through the read-only vote that can be returned during prepare). So JTA does support it. Of course how the resource manager figures out how to return read-only is something that the transaction manager doesn't mandate (and it shouldn't). So in that regard an annotation makes sense. But other approaches are possible (e.g., making locking rules explicit within the business object.)

The presentation also gives us the notion that 2PC is for distributed transactions only. No! As I said above, 2PC is a consensus protocol so it's needed when you have more than one participant, even if they are on the same machine. Unless of course you're not interested in them reaching agreement. So please, don't equate 2PC with distributed transactions: I also know of many distributed systems using transactions where 2PC isn't the norm!

I hate subjective statements like "XA is Non-Trivial to Set Up." Yes, so is the brake system in my car, but I need it nonetheless! So although XA can be a PITA at times, if used correctly (in the right situations) it's well worth paying. Plus it's often a one-off cost that over the lifetime of a deployment fades into insignificance compared to the benefits, or the setup costs of other components. So try to look on this objectively, weighing up the pros and cons (which is maybe what the presentation is trying to say, although perhaps for obvious reasons it seems to push the non-XA approaches more.)

Something else that surprised me about the presentation was the inference that an application that uses an XA transaction manager, or perhaps one that implements 2PC in general, must incur the overhead of a log even if there's a single resource involved in the transaction (something which using "native" transactions would miraculously avoid). So the one-phase commit optimization hasn't been invented? Once again this is something that's part of XA too. Any good transaction manager implementation should take advantage of this and in this situation you wouldn't get a log created! And I'm not even going to bother about other optimizations such as presumed abort.

I suppose this misunderstanding leads to another subjective statement: "Native Transactions - As Efficient As It Gets". Followed by the recommendation that you should try to use multiple (independent) native transactions, with a single resource in each of course. Yes, that's also one of the extended transaction models we documented when developing The Additional Structuring Mechanisms for the OTS. However, as was pointed out back then, you don't get the same semantics (remember what I was saying about consensus?) If you need atomicity then there are very few alternatives outside of magic or quantum mechanics!

In conclusion, I think the presentation wasn't too bad if a little vendor specific (which makes sense given the background and conference). But if you're really interested in transactions, where to use them, where not to use them, and precisely what the trade-offs are in a more objective manner then there's a lot of other good information out there.

Vacation, decorating, Spring and reservations

I'm on vacation this week, taking the time to decorate my 7 year old son's bedroom. If you want to get away from it all and let your mind come at things from different angles then I can recommend it (though probably not if you're a painter/decorator by trade!) I find it relaxing enough that my mind wanders in a good way. I used to find the same was true when I was a kid at school making and then painting models while thinking through exam questions.

Anyway, over the next few days I hope to write up my thoughts on a couple of things I noticed over the past month or so. One is a presentation on Native Transactions in Spring and the other is on the Reservation Pattern and SOA Transactions. Yes, there's a common theme there!

Elite is 25 years old?!

I didn't realise that one of my favourite games of all time, Elite, is 25 years old! I remember buying this for my BBC Model B computer and playing it at all hours of the day during my early university life. Happy memories!

The best laid plans ...

You go and do something with the best intentions and it suffers a few minor teething problems! With the benefit of hindsight I think we would definitely have gone about it differently, but at least it hasn't all been negative. Hopefully something good will come out of the various discussions, because that was always our aim!

Mind maps

I first came across mind maps back in the last 1980's when a friend/colleague at the University started to use them. I didn't really think any more of them until recently, when another friend pointed me at XMind. Very nice piece of software and it's opened up a whole new way of doing things for me. Gone are my scraps of paper, hastily scrawled notes and other ad hoc approaches!

That time of year again.

Is it really 8 years and counting?!

Google books

Now here's a book I haven't seen in a long time! Three of the authors were/are professors at the University when I was doing my PhD and several of the others visited regularly. A great book and still relevant today!

JBossWorld 2009

It's almost upon us! This year is unique for a couple of reasons: first we're holding the event at the same time and location as Red Hat Summit, which means we could have some interesting audience cross-overs, and second because this will be my first time as CTO since taking over from Sacha!

This year I'm giving a session on our work on testable architectures as well as my keynote. Normally I don't have any difficulties preparing presentations for conferences and workshops. I've done several keynotes in the past for other events, but never one in my current role. It's quite daunting, given the various people I'm following in this role. Maybe it's the masocist in me, but I'm looking forward to it. Now if only I could finish the blasted presentation!

Back and into the fray

Back from vacation and just spent two days catching up on email (lots of things ignored on the assumption that if they're important I'll see them again!) Now getting ready for JBoss World!

Canadian downtime

I'm going on vacation for a couple of weeks in a few days. This is our semi-annual visit to the parents-in-law in Canada. Should be good weather and is so far in the "outback" that doing nothing and relaxing is the norm. Looking forward to it, though I'll be taking some papers to read and write. Blogging will take a back seat though.

Facebook

I don't use Facebook or any social networking site, but I know people who do, including my wife. So I was surprised and annoyed when I heard via Savas that "Facebook has agreed to let third party advertisers use your posted pictures without your permission. Click on SETTINGS up at the top where you see the log out link. Select PRIVACY. Then select NEWS FEEDS AND WALL. Next select the tab that reads FACE BOOK ADS. There is a drop down box, select NO ONE. Then SAVE your changes. Please repost!"

I'm doing more and more work with "the cloud" (whatever that means!) and it's things like this (security, identity, trust) that concern me. Maybe when I finalize my opinions around this and other aspects I'll write something up. For now beware. And if you're on Facebook maybe you should change your setting?

Feynman

I grew up on Richard Feynman though wasn't really able to appreciate him until I started my physics undergraduate degree. He's definitely a hero of mine. One of the best programs I've watched about him didn't actually concentrate on his deep physics background, but was about his trip to Tuvalu. So it was with pleasure that I came across Project Tuva from our friends at MSFT.

Memories ...

While looking for something else, I came across this old ANSA/Esprit project proposal from about 1989. One of the earliest external project memories I have and a good one too! Then I thought "Oh, that's 2 decades ago! Oh frack!"

SoyLatte

So far my adventures in JDK 1.6 land have been constrained to a 10 year old Windows box, so I decided to upgrade my Mac. Unfortunately the machine I've got isn't 64 bit so the official Apple version doesn't work. Fortunately the SoyLatte distribution seems to work very well. Yet another example of good open source at work.

Scala round 2

I mentioned earlier this year how Mic had pointed me at Scala. I haven't had nearly enough time to play with it (which usually means me implementing a transaction service in the language!) but from what I've seen I'm impressed. It's interesting to read what others think too.

Walking with Dinosaurs

I remember watching Walking with Dinosaurs when it first aired on TV and being impressed with the graphics and story. The rest of my family were equally impressed. So when we heard about the Arena Spectacular version and that it was coming near us, it was something we had to see. I took the day off work to make sure no one could plan anything for me to get in the way and we went to see it: well worth it and I'm extremely impressed! So if you like dinosaurs or your kids do then go and see it if you get the chance. If you don't have any kids then borrow some so you won't look out of place!

Principles of Transaction Processing

I've known Eric as a colleague and friend for a long time, but my first run in with him was with his and Phil's first edition of Principles of Transaction Processing. Along with Jim's book, it has a key place on my bookshelf. Over the years I've probably bought a dozen or so copies for various groups I've worked with. Well it's good to be able to say that they've released a second edition. I was one of the reviewers of the book over the past couple of years so I know this is a solid replacement for the original. Well done!

Supporting performance?

Given my background in fault tolerant distributed systems, performance metrics are something that come up time and time again. I've spent many a happy hour, day, week, month pouring over timings, stack dumps etc. to figure out how to make RPCs go faster, replication to scale and transaction logs to perform. (Funnily enough I was doing the latter again over Christmas!) But this is something we've all had to do when working on Arjuna.

If you've ever had to do performance tuning and testing then you'll know that it can be fun, but often an almost never ending task. Then there's the comparisons between different implementations. This isn't just a vendor-specific area: even in academia there's always a bit of "mine is faster than yours" mentality. I suppose it's human nature and in some ways it's good and pushes you to improve. However, performance measurements need to be taken in a scientific manner otherwise they are useless. It's no good to simply state that you get "X transactions a second" if you don't specify the conditions under which that figure was achieved. This is true for a number of performance metrics, not just in the computing industry: there's a reason world records in athletics have defined conditions, for instance.

Now of course you may not have the same hardware to test and the tests themselves can be notoriously hard to access from one vendor/competitor to another. What tuning parameters they use as well as the general configuration can also be difficult to obtain. This is why Jim Gray and others wrote A Measure of Transaction Processing Power (though the principles in there can be applied much more widely.) This eventually produced the TPPC. But the general principle here is a scientific approach to performance metrics is the right way (though it's still possible to skew them in your favour.)

Whenever I see or hear of performance figures I always want to know the conditions (hardware, configuration, wind conditions etc.) It's not possible for me to take them seriously unless I have this additional data. Giving out the tests themselves is also a good thing. However, performance is not something you can support like you do with, say, transactions, where you either are transactional or you're not. With performance it can be a never ending struggle to keep improving, especially as hardware and software improve and your competitors get better too; you've always got performance even if it's bad! Performance is something that for some users can be as critical as new capabilities, or even more so. But it can't be performance at the cost of reliability or robustness: something that screams along faster than a speeding bullet yet crashes after 24 hours is unlikely to be as useful as something that is half as fast yet able to stay available indefinitely.

Internet dial tone?

The other day I had problems with my ISP that meant my connection to the rest of the world was up and down like a yo-yo. I self diagnosed the problem (the ISP help desk was pretty useless) and in the course of that I realized something: where's the equivalent of the telephone dial tone for the internet? Well if you're like me it's either Google or BBC news. Over the years whenever I need to determine if there's a problem with my connection or some site I'm trying to use, I'll use one of these sites to double check my own connectivity. OK it's not exactly a science, but I assume if I can talk to one or other of them then the fault lies elsewhere.

CFP for MW4SOC 2009

4th Middleware for Service-Oriented Computing (MW4SOC)
Workshop at the ACM/IFIP/USENIX Middleware Conference

Nov 30 – Dec 4, 2009
Urbana Champaign, Illinois, USA
http://www.dedisys.org/mw4soc09/

This workshop has its own ISBN and will be published as part of the ACM International Conference Proceedings Series and will be included in the ACM digital library.

Important Dates
===============
Paper submission: August 1, 2009
Author notification: September 15, 2009
Camera-ready copies: October 1, 2009
Workshop date: November 30, 2009

Call details
============
Service Oriented Computing (SOC) is a computing paradigm broadly pushed by vendors, utilizing and providing services to support the rapid and scalable development of distributed applications in heterogeneous environments. However, the influence of SOC today goes far beyond the concepts of the original disciplines that spawned it. Many would argue that areas like business process modelling and management, Web2.0-style applications, data as a service, and even cloud computing emerge mainly due to the shift in paradigm towards SOC. Nevertheless, there is still a strong need to merge technology with an understanding of business processes and organizational structures, a combination of recognizing an enterprise's pain points and the potential solutions that can be applied to correct them.

While the immediate need of middleware support for SOC is evident, current approaches and solutions still fall short by primarily providing support for only the EAI aspect of SOC and do not sufficiently address issues such as service discovery, re-use, re-purpose, composition and aggregation support, service management, monitoring, and deployment and maintenance of large-scale heterogeneous infrastructures and applications. Moreover, quality properties (in particular dependability and security) need to be addressed not only by interfacing and communication standards, but also in terms of integrated middleware support. Recently, massive-scale and mobility were added to the challenges for Middleware for SOC.

The workshop consequently welcomes contributions on how specifically service oriented middleware can address the above challenges, to what extent it has to be service oriented by itself, and in particular how quality properties are supported.

Topics of interest
==================
* Architectures and platforms for Middleware for SOC.
* Core Middleware support for deployment, composition, and interaction.
* Integration of SLA (service level agreement) and/or technical policy support through middleware.
* Middleware support for service management, maintenance, monitoring, and control.
* Middleware support for integration of business functions and organizational structures into Service oriented Systems (SOS).
* Evaluation and experience reports of middleware for SOC and service oriented middleware.

Workshop co-chairs
===============
Karl M. Göschka (chair)
Schahram Dustdar
Frank Leymann
Helen Paik

Organizational chair
====================
Lorenz Froihofer, mw4soc@dedisys.org

Program committee
=================
Sami Bhiri, DERI (Ireland)
Paul Brebner, NICTA (Australia)
Gianpaolo Cugola, Politecnico di Milano (Italy)
Francisco Curbera, IBM (USA)
Frank Eliassen, University of Oslo (Norway)
Walid Gaaloul, Institut Telecom (France)
Harald C. Gall, Universität Zürich (Switzerland)
Nikolaos Georgantas, INRIA (France)
Chirine Ghedira, Univ. of Lyon I (France)
Svein Hallsteinsen, SINTEF (Norway)
Yanbo Han, ICT Chinese Academy of Sciences (China)
Valérie Issarny, INRIA (France)
Arno Jacobsen, Univ. Toronto (Canada)
Mehdi Jazayeri, Università della Svizzera Italiana (Switzerland)
Bernd Krämer, University of Hagen (Germany)
Mark Little, JBoss (USA)
Heiko Ludwig, IBM Research (USA)
Hamid Reza Motahari Nezhad, HP Labs (USA)
Nanjangud C. Narendra, IBM Research (India)
Rui Oliveira, Universidade do Minho (Portugal)
Cesare Pautasso, Università della Svizzera Italiana (Switzerland)
Fernando Pedone, Università della Svizzera Italiana (Switzerland)
Jose Pereira, Universidade do Minho (Portugal)
Florian Rosenberg, Vienna University of Technology (Austria)
Regis Saint-Paul, CREATE-NET (Italy)
Dietmar Schreiner, Vienna University of Technology (Austria)
Bruno Schulze, National Lab for Scientific Computing (Brazil)
Stefan Tai, Institut für Angewandte Informatik und Formale Beschreibungsverfahren - AIFB, Karlsruhe (Germany)
Aad van Moorsel, University of Newcastle (UK)
Eric Wohlstadter, University of British Columbia (Canada)
Raymond Wong, UNSW (Australia)
Roman Vitenberg, University of Oslo (Norway)
Liming Zhu, NICTA (Australia)

The next wave?

I've been asked several times over the past few months what I think will be the next technology wave. If I knew that I'd be writing this entry from my own personal island in the sun! I've been involved in a few of these technology waves over the years, both as a user and a definer, but predicting them is another thing. For instance, who would have guessed at the turn of last century that SOAP would have taken on the major role it has today? Or that open source would have become such a defining wave?

I have my own theory for how technology waves begin and it wasn't until I watched a program from the BBC on rogue waves for the second time that I found a decent analogy. The relatively new theory (based on Schrodingers Equation) goes that these super waves, which are big enough to sink ships, are formed when energy is "stolen" from one wave to feed another. This builds and builds to create these towering monsters. Well I think technology waves are very similar: something relatively innocuous, such as SOAP, pulls in energy from other fields, such as EAI and the Web, to grow to the scale of a disruptive influence when it reaches a tipping point. This also makes it difficult to predict a priori whether or not something will be a wave: many different things contribute.

With that in mind, what do I think will influence the next technology waves? Here are a few ideas though not all. I should add a disclaimer that these are my own personal opinions and not necessarily those of my employer:

• A new programming language? Over the past 40 years or more we've seen languages come and go. The only constant is binary! Periodically we go back to high-level versus low-level language debates (e.g., can compilers really optimize as well as writing raw machine code?) Java has been influential for a long time, but if history has taught us anything it's that everything has a season, so it's only natural that at some point Java popularity will wane. But I'm not sure that a single language will replace it. Java didn't replace C, C++, COBOL or Lisp, for example. With the increase popularity of languages such as Erlang, Scala and even C++ making a comeback, variety is the right approach. Yet again a case of one-size doesn't fit all. When I was at University we learnt a lot of different languages, such as Pascal, Occam, Concurrent Euclid, Ada, C, C++, Lisp, Prolog, Forth, Fortran, 6502 (still one of my favourites), 68K, etc. When Java came along that seemed to change, with the focus more on the single language. Hopefully we'll go full circle, because as an industry we can't afford to keep reinventing software every 10 years to suit a new language.
• What about Cloud/Virtualization? Yes, there's a lot of hype in this area and I do think it offers some very interesting possibilities. But I'm not sure it's a wave in its own right. I suspect that we're still missing something to turn this into a Super Technology Wave. That could be SOA, fault tolerance, the economy, or Skynet.
• It will finally dawn on the masses that security (including identity, trust etc.) is something we take for granted and yet is not available (at all or sufficiently) in many of the things we need. (See Cloud, for example.) Security as an after thought should be replaced with security as an integral part of whatever we do. Yet again not a wave in and of itself, but something that will be pulled into one I hope.
• Of course REST has been around for a long time and given the many discussions and debates that have been raging for the past few years we're definitely seeing more and more take-up. I'm not going to debate the pros and cons here (have done that before), but I am sure this will become a wave, if it's not already.
• A unified modeling approach to building distributed systems, pulling together events, messages, etc. JJ has been talking about this for a while and it would be nice to see.

So there are a few thoughts. There are more, such as around designing for testability and HCI, but those can wait for another time. Maybe some or all of the above will be sucked into the new technology wave. Maybe none of them will. That's why I'm not sat writing this on some sun-soaked beach on my own private island!

Utter disbelief

I've been involved with standards since the early 1990s, working within the OMG, OASIS, GGF, W3C and others. They all have their rules and regulations, and most of the time they technical committees are populated by the vendors. That's not because these organizations are closed to end-users, but I think many of those end-users believe it's a lot of time and effort to participate. They'd be right.

Over those years I've been involved in a lot of political wrangling. You have to know when to push and when to give in. Priorities are important and today's foe could be tomorrow's ally. Sometimes people can get very emotional about this proposal or that, trying hard to justify why it should be voted for or against. That's good, because passion is important in what we do. But sometimes the arguments can be very fatuous. However, it wasn't until the other day though, that I realized I hadn't heard all of them!

To protect individuals I won't say what meeting it was, but suffice it to say that there was a face-to-face meeting recently that I attended by phone. Because of the time differences it meant I was doing my normal work from early in the morning my time and then in the evening jumping on a call for another 6 hours. For 3 days running! Anyway, during the 2nd day one of the vendors was trying to get their point across to everyone else and failing: pretty much everyone had decided to vote against, but this vendor kept trying hard. I had assumed that because this vendor (name withheld!) has been involved with standards for a while they knew the rules of the game. But apparently not, or at least the people representing them didn't.

So when it became pretty obvious that they were going to lose (this standards group uses the rule that one vendor has one vote, no matter how many employees it has on the working group) the cry went up "But what about the thousands of customers we have who want things our way?" Huh?! WTF?! First of all that's a very subjective statement. How do we know those customers exist and what they really want? If we were to have to take them all into account then we'd have to solicit their responses on all votes. Secondly if we go that route it'll become a "customer war", with vendor A saying "I've got 1000 customers" and vendor B saying "well I've got 1001!" This kind of weighted voting doesn't work! If customers really want a say then they can sign up to any of the standards groups. In fact I'd really encourage them to do that, because it makes a lot of difference: the best standards all have customer input!

After I heard this and stopped choking, I realized that it was really a case of desperation on the part of this vendor. Hopefully they'll read the rules before they turn up next time.

Web 2.0 architectures

One of the high points of any JavaOne that I go to is getting to meet up with friends such as Duane and James. As always we have fun catching up over a few drinks and good food (hopefully the pictures of this year's "event" won't turn up!) But this year was more special than usual because their book is out! It's been a long time in the works but it is well worth the wait. If you're at all interested in Web 2.0 then take a look at this book!

Sun-set

I've no idea if this is the last JavaOne as some rumours suggest (others that it may simply become an add-on to Oracle World), but it is with some sadness that I think about that and the acquisition of Sun by Oracle. Every time I think of this I keep hearing the words of Don't Let The Sun Go Down On Me (used to great effect in The Lost Boys.)

Putting aside my role within Red Hat, I've always had a soft-spot for Sun going back to the start of my career. In one way or another I've worked with Sun hardware and software for over 20 years. Back then, in the mid-1980's, the workstation was really starting to come into its own as the old batch-driven mainframe systems tailed off. My first work computer was a Whitechapel, the UK attempt to enter that market. A great machine, particularly if you were new to administering a multi-user Unix machine.

But in the Arjuna project the real king of machines was the Sun 3/60 with SunOS. For years we all strove to own one, either as a hand-me-down or when we got new funding. We went through to most of the Sun-4 series and into the Solaris years, with each year bringing a new top-of-the-line machine for one of us to manage (these were multi-user machines of course). Our project was so prominent within the University that others followed our lead where they could and where they couldn't they looked on slightly enviously.

During that time we often spoke with various Sun engineers in the hardware and software groups, e.g., Jim Waldo. This was on things as varied as transactions (not surprising really) through to operating systems (we got one of the first drops of the Spring operating system) and distributed computing. Then when Sun released Oak and eventually Java it was a natural extension for us to get involved with that heavily.

It was in the early 1990's that things started to change though. We were using Linux from the first public release Linus made (I remember Stuart and I taking it home one evening to replace Minix that we'd been running on the Atari's). Then when the project bought us Pentium machines, Stuart made the switch from a Sparc 2 running Solaris. He and I ran various benchmarks against my Sparc and for most of the things we needed at the time (compiling, execution etc.) the Pentium/Linux combination was faster as well as being a lot cheaper! The rest, as they say, is history. Year on year the PC and Linux combo got faster and faster, laptops came into this too pretty early, and the project steadily moved away from everything Sun. By the end of the 1990's the last few Sun workstations that were still in use were several years old and in the minority.

From conversations with others I've had over the years I think this is a very similar pattern to elsewhere. Was there anything Sun could do about this? Maybe and I'm sure that question will be debated for years to come. But for now it is with sadness that I contemplate a world without Sun. They helped shape what I am today from hardware, operating system, programming language(s) and so much more. In that regard I will always be in their dept.

The paperless office?

If you ever come to my house you'll see that we're a very heavily literate family, with books adorning book shelves in pretty much every room in the place. But recently my wife took to reading in ebook format, for convenience (she can now read with the lights out) and for storage (one SD card can accommodate 1000s of books and is a heck of a lot easier to store than 1000s of books!). I managed to convert an old Jornada 720 that I had from my HP days into an ebook reader so it was also good for the environment in that regard. But it got me to thinking again about paper vs bits as far as reading material goes.

I've had this discussion with friends and colleagues for decades and my opinion has always been to prefer paper than bits. Excluding the tens/hundreds of papers I read each year for work, or simply because I want to, I usually have at least another hundred or so that I go through for the various conferences/workshops that I'm involved with. Then there are the two or three PhD or MSc theses that I have to read as well.

Most of these papers come in electronic format although there are the odd exceptions. It's true that I rarely print out papers that come in, say, pdf. But I do print some of them off. Why? For convenience for a start: I can't remember the last time a stack of A4 sheets ran out of battery, and marking them up is a lot easier. But also because I like the tactile input that I get from paper. The same goes for books: I much prefer physical books than ebooks, even though the content is the same. Then there's the connectedness aspect of reading an ebook/epaper: when I'm reading them on my laptop and my mind starts to stray I quickly head for the internet or Eclipse, or something else that can take my attention and soak up my time. With a physical copy of the same material I find that my mind wanders less and if it does it tends not to go too far. Weird.

Probably the only time I yearn for an electronic version of a book or paper is if I want to do a search for something. So I suppose what I really need is for someone to invent an ebook reader that gives you the look and feel of paper!

Feeling more human again

It's a long weekend here (national holiday on Monday). I have a number of things I need to do for work, but at the moment I'm back coding on JBossTS for fun. Not something I have to do, but definitely something I like doing. It (coding, architecture, etc.) makes me feel good! I'm off to JavaOne next week so I'll take a few more coding tasks with me, probably on some other projects.

The clock is ticking!

Only a few hours left to finish the SOA/ESB book! It'll be a relief to be honest. A lot of time and effort, but it's definitely time to finish it and move on to the next one.

What ever happened to the Jiffy?

Back in the 70's when I first started to use computers we were told that computer time was measured in Jiffys (a 60th of a second back then). It was still a commonly used term when I started at university in the 80's (I even ran into it in physics). But since then it seems to have vanished from everyday use, at least in my field.

Science fiction to reality

Well over 30 years ago I read Do Androids Dream Of Electric Sheep? I loved the story and it's one I've returned to many times over the years. So when they announced a movie version of it I knew I had to go. I was a mere 16 when I went to see Bladerunner for the first time, on the back of such films as Star Wars, Close Encounters and Star Trek. I liked the film and have liked/owned every version since (yes, even the "original" with Harrison Ford/Deckard talking over in order to explain the plot). Of course they missed out bits from the book, but it was still a masterpiece.

One of the things I really liked in the movie is when Deckard uses a computer image scanner to look through photographs for signs of the Replicants. But this is no ordinary scanner: it uses data in the image to extrapolate further, uses reflections to look round corners and has incredibly good resolution. I found that idea fascinating and it was a high point for me when I played the Blade Runner game on my PC 10 years later. Over the years photographic resolutions and contrast capabilities have improved and we've all probably heard the debates of whether or not we can read newspapers from orbiting satellites. Not quite what Deckard was able to do, but interesting nonetheless.

However, today en route to Boston for a meeting, I found time to read my monthly dose of Scientific American. In this month's issue (May 2009) they have an interesting article on computer security and specifically how people can steal information from a computer without having direct access to it or the machine being networked at all. Of course they cover the areas known for a while, such as catching RF from screens, but it's the work on viewing screens through reflections off teapots, spoons and even the human eye that made me sit up and think! OK at the moment it requires a lot of time and money to do, but I'm sure it would be worth it for certain secrets. Plus it'll only be a matter of time before the equipment becomes smaller and less expensive.

What can you do about this information leakage? Well as the article says, "privacy filters" can increase the chances of being read in this way and flat-panel displays still emit some information. So curtains or blinds are the best defence at the moment. But I'm not going to worry too much. It was a nice article that brought back memories of a great movie and book.

One thing that goes missing from backups

My mobile phone decided to die yesterday. Bad enough in and of itself, but it's gone in such a way that I can't get access to those little things I take for granted, such as address book! Yes, I have a version on the SIM, but it's way out of date. I only copy from the phone to the SIM when I move phones and that relies on them both being available! I back up my machines regularly. I do the same for my PDA. But it never occurs to me to look at the phone too. I think that'll change now.

DoD and SOA

I was at the DoD sponsored SOA Symposium a couple of weeks back. I was speaking on Open Source and SOA, which is a little different to the abstract shown (more on that in a minute). Overall I was very pleased and impressed with the conference: it was by invitation only and limited to about 400 attendees (including the presenters). Every session was packed and there was a good mix of vendors and use-cases. I can hear what vendors think any time of the day, so it was the user-driven sessions and interactions that I was particularly interested in.

If this event was anything to go by then the DoD (and the US Government in general) really seems to have embraced SOA and open source. That's really good to see on a number of levels (I'm trying to remain objective here!) What blew me away though was the sheer scale of the projects that they run: lots of people involved and lots of money being spent. It should go without saying that reliability, fault tolerance and security (the Common Criteria is king here) are critical to everything that goes on, which can make it hard for the majority of vendors to get accepted.

Overall it was probably one of the most eye-opening events I've been to in recent years. The sessions were uniformly good, but it was the interactions between the presentations that really made this event shine. It's good to see (open source) SOA being used successfully in large-scale, mission critical applications.

Now back to my presentation. Because this was a DoD sponsored event you have to try to make your sessions vendor-neutral. Well I obviously missed that memo so had a rush on at the last minute to update the slides and remove all of the SOA Platform references. The presentation I ended up with was much better as a result: given my academic background I don't like white papers or sales-driven presentations anyway.

On the road again ...

I'm just about to get into a taxi to head off to the SOA Symposium in Washington on Government and Industry Best Practices in SOA. I'm presenting on Thursday morning on SOA and Open Source so if you're around come over. Since this is a DoD sponsored event I hear that the security will be pretty high, so it's going to be interesting for a number of reasons!

New job

I mentioned that Sacha was leaving. Well I can now mention who is taking over.

Analyze your blog

According to this blog analyzer I'm a doer.

"The active and playful type. They are especially attuned to people and things around them and often full of energy, talking, joking and engaging in physical out-door activities. The Doers are happiest with action-filled work which craves their full attention and focus. They might be very impulsive and more keen on starting something new than following it through. They might have a problem with sitting still or remaining inactive for any period of time."

Sacha is leaving

If you don't know by now, Sacha is leaving JBoss. There's not a lot more I can add to what Bob has said, except that I'll miss out almost daily interactions. Sacha started as a colleague (when he, Bob and Marc persuaded me to move from Arjuna) but grew to be a good friend and I wish him and his family all the best in the future. Today begins a new chapter in his life as well as that of JBoss. Good luck Sacha!

First dive of the season

It may only have been Ellerton again, but we managed to get the first dive of the season in today. It was a bit like swimming in tea (less than 2 metres visibility) and cold (8 degrees Centigrade), but it was worth it. Everyone else there was in dry suits but we managed to struggle through in our wet suits. I just wish someone would invent a spray-on suit, because I'm sure I get more of a workout putting the darn thing on that actually swimming afterwards.

RPC is dead?

From what Steve says it certainly sounds like this year's QCon London was fun. This year Steve was talking on the history of RPC, which has some overlaps with what I was saying at QCon London last year. But Steve goes in to a lot more detail and I recommend people checking it out.

One slight caveat though: as I've said many times in the past (so many that I leave it as an exercise to the reader to find the entries in my blog) RPC isn't dead and isn't something you should ignore. (I don't think Steve thinks that either.) Look around you and you'll see examples of old and new systems based on it. Are they all wrong to do so? No, of course not. Are some of them wrong to do so? Most probably yes. I know of several small and large-scale systems that are being developed in academia (no vendor pressure there) and industry (maybe some pressure) that are being based on RPC. In all cases those guys did their homework and understand the trade-offs that they're making in using RPC.

BTW I'm not a RPC fanboy by any means. Yes, I've used them and helped develop them, but I've also used and developed other approaches too.

What I'm trying to say is that RPC has its place. But a bit like ACID transactions, it can be (and has been) easily misused. There are better approaches these days for many of the things we'd once have considered the domain of RPC. But that doesn't mean RPC can't and shouldn't be used. As with everything, such as which language to use, which database, which communication protocol etc., you need to be aware of the pros and cons. This is an education problem more than a technical problem.

I also agree with Steve around Erlang (in fact it was Steve who put me on to it a while back). A very nice language. Now if only I could find the time to get back to it.

Congratulations Barbara!

Barbara Liskov won the Turing Award! This is really good news. I met Barbara a few times, starting during my PhD days. At that point she was leading the Argus project in transactions and replication, which was relevant to my own research. Thoroughly deserved. Well done Barbara.

DOA 2009

I'm co-chairing DOA again. Check out the CfP for 2009.

HPTS 2009

HPTS is probably my favourite workshop/conference. I've been lucky enough to be able to attend every one since the 1990's and hopefully this year will be no different. The CfP is now up, so take a look.

The ultimate Christmas dinner

OK it's a little late but then I didn't realise this video was online! I watched it for the first time over Christmas 2007 and then again last year. Heston Blumenthal has quickly become a favourite to watch and someday I hope to be able to get to his restaurant. Until then I'll sit and watch in awe!

Zakim paper

I've been using Zakim for a few years through various W3C standards committees. Nice paper.

Arjuna and the OTS

A couple of comments on my last reminiscence post got me thinking again. One of the things we use to go on about a lot in the early days of the company was the influence we'd had on standards and particularly the Object Transaction Service from the OMG. If you ever compare the two you'll see that by the time the OTS started Graeme, who co-created the Arjuna project (upon which he also got his PhD), was working for Transarc and involved in the standards definition. I remember visiting him in Pittsburgh a few times during that period and hearing a few horror stories about this brain-dead approach being pushed by vendor X and that dumb idea from vendor Y. But I'm sure it was all fun; it was definitely influential on the industry and me personally. When I came to start pushing the OTS in the OMG I did understand what Graeme had been on about with standards processes!

OMG where did the time go?

While writing the previous entry concerning building transactional applications I came across that HP paper by Nigel Edwards. That sent shivers down my spine because I can recall Nigel sharing the office with Stuart and I for months while he got to grips with Arjuna. We were both still doing our PhDs and also in the throws of making the first code release of Arjuna to the world. Here's the announcement for Release 2 (it would appear that our first release predates the web!):

Arjuna (Distr Prog System)

What: Release 2 of Arjuna Distributed Programming System
From: arjuna@newcastle.ac.uk (Arjuna Project)
Date: Mon, 17 May 1993 12:37:34 GMT

        We are pleased to announce the  availability  of a new  version 
of Arjuna:  a programming system for  reliable  distributed  computing, 
and the Arjuna mailing list.

        The software  and the manual  for  the  Arjuna  system  can  be 
obtained by anonymous ftp: arjuna.ncl.ac.uk (128.240.150.1)

Arjuna System

        This beta release of  ArjunaPR2.0  fixes all known bugs present 
in ArjunaPR1.2B that have  been  reported to us or  that we have found, 
and contains only minimal information about how to use the new features 
provided.   This  release  should  be  compilable  with  the  following 
compilers:

        AT&T Cfront Release 2.1, on SunOS 4.1.x,
            (using Sun supplied lex and yacc).
        AT&T Cfront Release 3.0.1, on SunOS 4.1.x and Solaris 2.1,
            (using Sun supplied lex and yacc).
        GCC versions 2.1, 2.2.2, on SunOS 4.1.x,
            (using flex(v2.3.x) and bison).
        Patched GCC version 2.3.3 on SunOS 4.1.x and Solaris 2.1,
            (using flex(v2.3.x) and bison).
        Sun C++ 2.1, on SunOs 4.1.x,
            (using Sun's lex++ and yacc++).
        HP  C++ (B2402 A.02.34), HP-UX 8.07,
            (using HP supplied lex and yacc or lex++ and yacc++).

The major new features are:

        - Faster object store.
        - Support for replicated objects.
        - Memory resident object store.
        - Support for ANSAware (not available via ftp)

        Arjuna supports nested atomic actions (atomic transactions) for 
controlling operations on objects (instances of C++ classes), which can 
potentially be persistent. Arjuna has been implemented in C++ to run on 
stock  platforms  (Unix  on  SUNs,  HPs  etc).  The  software available 
includes  a C++  stub generator  which hides  much  of the  details  of 
client-server  based  programming,  plus  a system  programmer's manual 
containing  details of  how  to  install  Arjuna and  use it  to  build 
fault-tolerant  distributed  applications.  The software and the manual 
can be obtained by anonymous ftp: arjuna.ncl.ac.uk (128.240.150.1)

        Several  enhancements   and   ports  on   various   distributed 
computing platforms are in progress.  We would be pleased  to hear from 
researchers and teachers  interested in using Arjuna.  The programmer's 
manual  contains the  e-mail  addresses for sending  your  comments and 
problem reports.

ANSAware version of Arjuna

The ANSAware version of Arjuna is available from:

Architecture Projects Management Limited
Poseidon House
Castle Park                                  Phone    +44 223 323010
Cambridge                                    Fax      +44 223 359779
CB3 0RD                                      Internet apm@ansa.co.uk
United Kingdom                               UUCP     ...uknet!ansa!apm

Arjuna Mailing List

To enable us to  help people using Arjuna,  an electronic mail list has 
been setup. You can join  the Arjuna mailing list  by sending an e-mail 
message to "mailbase@mailbase.ac.uk" containing:

join arjuna 

For example : join arjuna John Smith

Mail  messages  can  then   be  sent  to  "arjuna@mailbase.ac.uk",  for 
distribution.


Arjuna Project Team
The Department of Computing Science,
The University,
Newcastle upon Tyne.
NE1 7RU, UK.

The work we did with Nigel still seems so fresh in my mind and yet it is so long ago.

Then I also remembered the work on Stabilis that some of our Brazilian PhD students did a few years later. We'd always talked about how the programming model we had in Arjuna was good for building complex applications (and Nigel's work agreed with that), but it took us all by surprise when they presented Stabilis: here was the largest and most complex system (a full relational database) built entirely on Arjuna! This was an impressive demonstration of what was possible with what we'd spent the last 6 years working on.

Definitely a time to reminisce!

Transactions and AIT/TOJ

Jonathan pointed me to a recent discussion on TSS concerning writing transactional resources for non-transactional objects. It expanded into the more general topic of how to write XAResources, which it topical at the moment within Red Hat. Anyway it got me thinking that many of these requests aren't necessarily for "How do I write an XAResource?" but more for "How do I make my data transactional?" These are two different questions: the former infers the latter, but the latter doesn't infer the former. Yes believe it or not but transactions existed before XA came on the scene.

So it got me thinking and I realised that what many of them they really want is something akin to what we did in the original Arjuna system. It didn't have a name back then, since it was the whole reason for doing the Arjuna research, but eventually it became known as Arjuna Integrated Transactions (AIT), or Transactional Objects for Java (TOJ). The use of "it" has not been pushed much (at all) over the past few years, which is a shame because I still think it's a great model. So it may be time to dust it off and bring it back into the light.

Sock and Anti-Sock Pairs

Back when I was hard at work doing my Physics undergraduate degree (a quarter of a century ago!) I remember a few of us finding creative ways to spend the time during the 4 hour practical sessions we had twice a week. Well there are only so many times you can do Millikan's experiment or kill a cat in an enclosed box with a stray beta particle. (OK that last one is something we planned but never quite did for obvious reasons and it still scores high on the "what if?" discussions at reunions.) So what we often did was end up spending hours reading New Scientist.

The one article that has stuck in my mind for that length of time was a brief description of why you end up with odd socks in the wash and usually a piece of fluff. From what I can recall, it talked about spontaneous creation of anti-socks and sock/anti-sock annihilation. To us physics students it appealed on at least two levels: the obvious physics analogies concerning matter and anti-matter, but also the fact that for most of us this was the first time we'd been away from home and having to worry about doing our own washing, much of which was full of fluff and stray socks that we were sure weren't our own! Many times over the years I wish I'd kept that article. I had hoped that the wonder of the digital age would help, but this is the closest I've found.

DOA Program Committee and Topics

I'm chairing DOA 2009 and my co-chairs (Jean-Jacques Dubray and Fabio Panzieri) and I have been finalizing the program committee. Fortunately most of last year's PC have agreed to stay on so we've only had to add a couple of new people. Next step is to figure out the Call for Papers. If you've any thoughts on things you'd like to see at DOA 2009 that maybe weren't covered in the DOA 2008 CfP, let me know. After that we'll have to determine the invited speakers and keynotes. Fun fun fun!

Made my day

This article just made my day. Fantastic!

SOA Design Patterns released

I've been writing a book on SOA and ESB for a while with a couple of friends. It's due to be part of Thomas Erl's SOA series later this year. While writing it Thomas asked us to produce some patterns for his book on SOA. Well the finished book, SOA Design Patterns, is now out and it's well worth a read. A number of people throughout the industry have helped to contribute some of the patterns within it so this is definitely a cross-industry collaborative effort. Thomas also wants it to be a live-work in as much as new patterns can be contributed by anyone. So take a look at the book and if you see something missing consider contributing it.

Star Trek interlude

Anyone who's known me for long enough knows that I'm a Star Trek fan. (Not the variety that dresses up as Andorians or learns Klingon!) Being born in the late sixties sci-fi was fairly influential to me in one way or another, with my fondest TV memories coming from shows like Star Trek, Space 1999, Dr Who, The Six Million Dollar Man and Blakes 7.

I still say that the first couple of seasons of The Original Series were better than much of what was created in recent years. So it's with some apprehension that I look ahead to the new movie. I'm definitely not going to prejudge, but I had hoped they would retain some of the look-and-feel of the original.

On thing that has stayed with me over the past few years is a quote from Kirk to Picard from Star Trek Generations (watch the film if you want to understand the context):

"Don't let them transfer you... don't let anything take you off the bridge of that ship... Because while you're there, you can make a difference."

I've found that it applies to more than just starship captains!

Virtual conference

We're doing a virtual conference next month. I'm recording some sessions at the moment. Very strange, but I think it'll be fun to attend. Maybe this will encourage me to look into SecondLife.

SOA is dead (again?)

For the umpteenth time SOA is dead apparently. It's been shot so many times over the past year or so that you'd think it had a role in a Sam Peckinpah movie! However, maybe it's got 9 lives or you need to use a silver bullet, because each time it dies it keeps coming back.

This time it's my friend Anne who is reporting the demise of SOA. I've known Anne for many years and respect her, so unlike a few other obituaries I take note of this one. However, like Duane I have to disagree that SOA is dead. OK, the term may have been overloaded over the past few years almost to the point of being meaningless (analysts are as much at fault as anyone in the industry) but the concepts behind it are still very much alive and relevant (more so in today's economy.) Where we have lost our way is perhaps in having an agreed architectural definition for SOA: it is the 'A' after all, so surely we can agree on what that means (REST doesn't suffer this problem) as well as pushing the mantra that "SOA is not just a technology" (though given the number of people who have been saying this over the years I suspect that this is more a case of individuals simply not listening, certain vendors muddying the waters or it being lost in the noise.)

There is no global panacea for IT woes. There never has been and I doubt there ever will be one. For the most part software engineering practices evolve over time (Darwin would be proud). There are a few evolutionary dead ends on the way. I doubt we'll see any extinction-level events though (not unless they're associated with human extinction.) But these things take time (I'm sure the dinosaurs laughed at the evolutionary path that created the Coelacanth, but who's laughing now?)

We can't afford to keep jumping from one lifeboat to another just because some use cases don't match, or someone didn't quite understand that you need to think about how to use SOA before coding. The underlying requirements of loose coupling, security, federation, reliability etc. date back decades. The term SOA should have been good enough. It still is as far as I'm concerned. If it's not SOA then are we in for yet another round of acronym generation, like SOA 2.0, WOA etc? None of which really add much to help the people at the sharp end of IT problems!

I think in Anne's recent SOA obituary she is trying to indicate that the principles are fine but it's the term that needs to go. In the first case we can agree. In the second I don't think it's worth the industry wasting another 4 years to disagree on yet another acronym (if we do, I'd like to nominate XYZZY.) As I said above, for people actually using these principles and associated technologies it won't make a difference to them. (It may be good for analysts though.) So let's just stop this madness and start concentrating on where it really matters.

In this case I think it's definitely more a case of what Mark Twain once said, "The reports of my death have been greatly exaggerated".

Scala

My friend Mic and I have discussed a few languages I've been playing with over the past year or so, which include D and Erlang. I think I managed to persuade Mic that there is some benefit to Erlang, but maybe the jury's still out on D. Well a few weeks back Mic mentioned that he was looking at Scala, so I decided to take a look. I didn't find the time until I took a break from work for the festive season and have just spent a few days giving it a go. My first impressions are pretty positive (I've mentioned a few times in the past that I'm a fan of functional programming languages and particularly lisp which I've used for over 20 years). I'm trying to use it through Eclipse but am finding the plugin a bit clunky, so I may go back to the command-line and emacs (I'm a shell/emacs person at heart!)

So far I haven't got into any really complicated projects or applications so there may be problems with the language that I don't know about yet. But at the moment I like it and will persevere with some pet projects that I've been thinking about for a while. Even if it doesn't lead to anything ultimately, it's another language to learn and experience gained. But I have a suspicion it'll be more than a passing fad for me. Thanks Mic.

Eventual Consistency and XTP

I meant to post something about this article a while ago but only found the time due to the Christmas vacation.

It's a nice article and what Werner's talking about is part of what I consider real extreme transaction processing (not the hype term). As I've said elsewhere several times before, a few of us in the transaction community have been working on this type of problem dating back to at least the work we did in early SOA/Web Services standards. Even longer if you consider the work on extended transaction models. It's nice to see practical implementations of some of the theory behind these models actually being used outside of academia.

I'm hoping that in the next year we'll see some fruit coming from research and development we've been doing on real XTP, including the REST approach I've mentioned before as well as general approaches to large scale (extended) transactions. This should further illustrate what's possible with relaxation of the ACID properties we know and love, particularly when combined with high performance messaging and replication techniques. Sometimes it's hard to believe that this has been going on for so long.

Since this article started with a reference to Werner, it's worth ending with him too: congratulations my friend for your latest award. Well done! I think I could have predicated it ;-)

Sad day

I just read that Oliver Postgate has died. It's sad because I grew up in an age when Bagpuss and the Clangers were the high points of television. Those were some of my fondest memories.

SOA Symposium follow up

I seem to have forgotten to mention that I presented a couple of times at this year's first ever SOA Symposium a couple of months ago.

The Eternal Golden Braid

I first read An Eternal Golden Braid about 14 years ago and thought it was a great book. I still do. If you haven't read it then I would encourage you to do so: it is a wonderful journey through mathematics, computer science, philosophy, art and so much more. (I notice that there's a 20th anniversary edition, so that may be going on my present list to Santa since my original copy is a bit dog-eared.) So it was with some surprise that I noticed a translation of Godel's original work available! However, the reviews indicate that it may not be a good translation. But it may be worth a try anyway!

EDA followup

JJ commented on my earlier posting. Unfortunately he doesn't seem to have comments enabled, or I'd have responded there. (JJ, if you do have comments on that entry, let me know and I'll cross link).

Anyway, he talks about "the main failure of distributed computing is precisely the prevalent monotheism of traditional distributed computing approaches". Ignoring the fact that I don't believe distributed computing has failed, I think JJ read more into my posting than was intended. I didn't mean to suggest that any approach to distributed computing (e.g., RPC or message based) should be used to the exclusion of all others. I think I made that clear elsewhere too: you should use the best approach to hand. After all, isn't that what being a good engineer or architect is all about? I'm very fond of saying that one-size doesn't fit all, and by that I mean what I've just said. Of course systems exist that are entirely based on the concept of resources, services, events, RPC or message exchange, as examples. But there are examples of implementations that combine these different approaches very successfully too.

So in a way I disagree when JJ says "It is not being about fence sitting, or applying the right solution to a given problem." At this stage in computer science that is precisely what it is all about. However, I know what he's after and I've said elsewhere that in many ways computer science is a soft science and more of an art at the moment.

So looking forward, would I like to see a formal approach that combines all of the different approaches I've discussed to distributed computing, and which JJ mentions? Yes, I would. But would everyone use it in its entirety? No, of course they wouldn't: they would use the right aspect of it for the task at hand. Does that invalidate it? No. Does it offer an improvement over today? Yes. Is it going to happen soon? I doubt it, and I'm fairly sure JJ knows the reasons for that too (and the majority of them aren't technical). In fact I could probably write an entire article on that very subject because it relates to how close to a hard science computing science is and can come, with analogies between Galileo/religion and vendor values. But it's way too dark a subject to cover today.

DOA 2008 is over

I just finished chairing the last session for DOA 2008. It's been a good conference with some excellent presentations and invited speakers. Now for the long trip home.

Thoughts on EDA and CEP

Over the past year or so I've been seeing more and more discussions on Event Driven Architecture (EDA). I suppose the thing that really made me take notice was when Gartner tried to push SOA 2.0 and mentioned EDA in there. For me, event driven architectures (notice the case) have been around for as long as I can recall. I've used the concept of events and triggers in many systems I've been involved with without thinking it was somehow novel. The reason for this? Well as I say every time I'm asked about event driven architectures (still notice the case), I believe that they are all around us and have been since the very first computer system: they are a useful abstraction for building certain types of systems (not necessarily just distributed systems).

It's possible to model everything in terms of events when you think about it. For example, in a distributed object system that is based on RPC and uses client and server stubs to make the distribution opaque, the ultimate server object (the one that does the real work) is essentially registered with the server stub to be called (triggered) by it upon the receipt of the right message (i.e., the right opcode in the recipient message). Likewise, the client application (or thread of control) can be considered to register itself with the client stub to be triggered (woken, since we're working in a synchronous environment in this example) when the response message comes back. Pub/sub based messaging systems, such as JMS, support the concept of events as well. When you subscribe to a blog you're essentially registering to be triggered when an event (a new blog entry) becomes available. Anyone remember the CORBA Event Service or the DCE Event Management Service?

Back when I was starting my PhD one of the first things I did (for fun, as it really had nothing to do with any of the various subjects I was considering for my PhD at the time) was to write a hardware bus simulator in C++. It had the concept of components being connected to the various buses and registering to be triggered when specific events occurred. In the real world, hardware is event based, with IRQs as the events and the bus as the delivery mechanism. A very good example of an asynchronous system that is often ignored by the software community these days. Maybe we should be looking at calling the next wave Interrupt Driven Architectures instead?

I've written applications on a number of windowing systems over the past 20 years. Probably the best one was InterViews, but they all share a very common pattern: components (e.g., windows, scroll bars, minimisers, etc.) register for events (e.g., left mouse click, mouse move, etc.)

Over the past 5 years or so we've also seen a move away from the synchronous request/response approach to one-way messages with various Web Services standards and SOAP implementations. Services register their interest in specific events (receipt messages) and clients register their interest in specific events (response messages). But it's still very similar to the stub approach I mentioned earlier. But what we're seeing in this approach is making the concept of events and triggers more explicit than it was before (at least in the Web Services world). This is definitely a good thing for loose coupling when you look at asynchronous environments. But is is a good thing for developers and users? Maybe.

So why have we not seen more event based frameworks pushing events as first class entities up to the developer, which seems to be what the current discussions around EDA are about? I think the reason for this is abstractions: we typically try to hide things like events because most of the time we don't think in terms of them. When a sound wave hits my ear drum and is converted into electrical impulses that trigger a response in me, I don't normally equate the sound wave or the electrical impulses as events. When I receive a letter through my post box telling me I've won the lottery (I wish!) I don't think about the steps it took to be delivered as a series of events (though they clearly are). If you look back at what I said earlier concerning how event driven architectures have been with us for many years, you'll see that what's really happened is that we've tended to hide events with abstractions like stubs and message exchanges.

I've said before that I think an event driven approach to SOA (or pretty much anything) is good if it helps you to better reason about the system in question. If it gets in the way, though, there are alternatives that could be better. As with anything, you use the right abstraction for the job. Is this going to be like RPC, where we realise that hiding distribution is a bad thing (we knew all along really)? Is hiding events bad? Well as I just said, that will depend upon what you're doing. But just because you start thinking in terms of events doesn't solve all of your problems immediately. There may be benefit to exposing some business events to the higher level, but we shouldn't think that rewriting everything we currently use in an event-explicit manner will be a global panacea - it may even be a step backwards. As an industry we need to stop pushing point solutions as global solutions and remember that one-size doesn't fit all.

And what about Complex Event Processing (CEP)? In a nutshell CEP gives you the ability to correlate events from different streams over time (time should really be modeled as an event too) to determine patterns, manage and predict other events (such as failure conditions) etc. The data streams can contain messages at various levels of a system architecture, so it's possible to use CEP at the low level (e.g., allowing a system administrator to predict when machines a more likely to fail given daily usage patterns) as well as the higher level (e.g., allowing a business analyst to determine the buying habits of specific individuals given the market trends elsewhere in the world).

So if you've got CEP do you have an EDA? Well you obviously have something that is generating the event (data) streams that are input to the CEP implementation, but that doesn't mean the entire system is (or should be) architected using an event abstraction.

If you've got an EDA do you need CEP? This is more clear-cut: of course you don't need CEP if you've got events. There are more examples of event driven architectures that don't use (and don't need to use) CEP than those that do. But in some cases, it can make things easier to use CEP.

DOA 2008

I wasn't going to be able to come to DOA 2008, which was a shame because I enjoyed DOA 2007. However, neither of my co-chairs could attend either, so it fell to me to come. It's in Monterrey, Mexico this year and the trip here wasn't something I would like to repeat very often. But I'm here now and looking forward the conference. It'll also give me a chance to catchup on some work that I haven't been able to do (because of work) and blog entries that have been filling up my to-do queue. So one way or another the next few days should be interesting.

MAX sounds like it'll be fun

I was originally booked to presented at Adobe MAX as well as QCon SF (since both are back-to-back it was very convenient). However, I had to pull out. It's a shame because both events sound like they'll be a lot of fun and I wish I could attend.

New Macs

I've been a Mac user for a few years (keep promising myself that I'll return to Linux one day), so I was looking forward to the announcement of new Macs. But I obviously missed something because I'm not really impressed with what I've seen. No button on the track-pad? No 1900x1200 resolution? Slightly improved performance. But gone is the metal screen surround, to be replaced by black plastic?! I'm not sure what I expected from Apple, but it wasn't this. Plus, maybe the thing will look better when viewed up close rather than through a web page. If the new design does get around the erosion problem then it'll be worth an upgrade eventually.

Transactive Memory

I've been doing some research on Software Transactional Memory recently and as a result started to read about Transactive Memory. Now it quickly became apparent that the two things are unrelated, but I found transactive memory to be very interesting (not to indicate that STM isn't interesting!) Put simply, transactive memory is the name for the process whereby people selectively remember things and rely on their relationships with other people, or other things, to remember everything else. So for example, you don't remember every single phone number for everyone you know: you're selective (because that's how the brain works) and remember the "top" 6 or 7, but will probably rely on a telephone book or a PDA for the others. In the home, you may not remember how all of your friends are related to one another, maybe relying on your partner to do that. If you are a technophobe then you may rely on someone in your family to program the DVD player so you don't have to remember how to do it. If you've got kids then you may rely on your partner to do the bulk of their welfare. And the list goes on.

So what has this to do with STM? Well as I said at the start, absolutely nothing. But it does have relevancy to something else I've been interested in for the past few years: repositories. I've always believed that systems such as repositories are better implemented in a federated manner: they scale much better. This means that although you may have a logical notion of a repository, each physical instance is responsible for some different aspect(s) of the whole "mind". This is important because, for example, how you store and index service binaries is different to how you would store and index service contracts, or workflow definitions etc. In this kind of set up if you ask one repository instance "do you know about X" it should say either "no, but I know a man who does" or "no, but hold on and I'll get the answer for you". I'd never been able to put a name to the approach (beyond federation), but it seems to me that this is precisely transitive memory from a software system. DNS works in the same way.

CSP book available for free

I'll never know how I missed this announcement from 4 years ago, but the original CSP book is one of the most dog-eared in my library. If you don't have a copy, then I encourage you to download this and read it: it's as applicable today as it was 20 years ago (if not more so!)

To Twitter or not to Twitter?

A number of friends are "on" Twitter and apparently I should be too. I'm not quite sure why though. If blogging is anything to go by, it'll probably take me a couple of years to get round to it.

Nice article on REST

I haven't blogged much recently because I really haven't had the time. The same is true at the moment, since I'm sitting in yet another airport lounge, this time coming back from the first SOA Symposium (hopefully there'll be a blog entry or two from me on that subject later). But I wanted to say that I really enjoyed the latest article from Jim, Savas and Ian. It's been a few years since Jim and Savas worked for me and blogs/articles are often the only regular way we keep in touch. Nice work guys!

Farewell John Warne

I just heard that a friend of mine, John Warne, died this morning. The immediate cause was pneumonia, after he had been diagnosed with liver cancer five weeks ago.

I first met John Warne back in 1987 when I was doing my PhD and he was working at ANSA on behalf of Nortel. He was a great guy, always very passionate about his work but also very down to earth. Over the years our paths crossed time and time again, and no matter how long it had been between meetings it was always like it has only been a few days. He was a great guy to listen to, with some wonderful stories about things he'd seen and done over the years. I remember him telling the story about how he'd been sent by Nortel to work on the Hong Kong Race Course and that they (the Race Course) had so much money they would fly him first class, best hotels etc. He was a good mentor and always willing to spend time to understand problems or share his knowledge openly. In short, John was one of those rare people in our industry: a really nice and intelligent person, someone you looked forward to seeing.

It's a very sad day. I'm not sure what else I can say. Farewell John, it was an honour and a pleasure to know you.

It's that time of year again ...

It's September 11th. 'Nuff said.

A gift interlude

For Christmas 2007 my wife got me an Aqua Sphereing experience. The local centre is also only 5 miles from our house. Given that she was also the one who got me into SCUBA diving, I figured this would be fun. Unfortunately the weather in the UK this summer hasn't been that good, but eventually I managed to find the time and the weather held. Although the whole thing lasted only a minute or so, I had a fantastic time and now know what it feels like to be inside a giant washing machine. My friend Duane did something similar at JavaOne earlier this year, but I don't think there was water involved. If you get the chance, I'd definitely recommend the experience!

For our anniversary this year, my wife got me an EcoSphere (hmmm, there's definitely a spherical theme going on here!). It turned up the other day and it's fantastic. I love this sort of thing: mini-worlds, whether they're real or virtual, have always fascinated me. Maybe it's a god-complex or something, but I can spend hours just watching these worlds evolve.

Erlang and Mac OS 10

Just back from holiday and still playing catch-up, but this caught my eye. I've been using Erlang on my Mac for a while but this would have saved me some time back when I started exploring the language (which I like more and more each day).