RESTful transactions round two

I've been working on a second draft of the RESTful transactions work that I've mentioned before. This time I'm doing it for Bill and his REST-* effort. I revised the original for JavaOne but didn't get a chance to use it in our presentation. So I'm taking this opportunity to apply some standards boiler-plate and bring it up to date. Plus it's always good to revisit something you did almost a decade ago and use the benefit of those intervening years and the experience gained.

Enterprise OSGi: two is obviously better than one

I think OSGi is important for several reasons. I think Enterprise OSGi is an interesting approach, particularly as it leverages JEE. I've even contributed to some of the work, for example around the transactions component. JBoss is doing a lot of implementation work around OSGi too.

I have to admit that I haven't been paying close attention to OSGi for a few months. However, I had heard about the new Apache Aries project. Unfortunately I just heard from a friend about the Eclipse Gemini project. Now I've been involved with standards long enough to have experienced first hand the political games that rivals play with each other. It's unfortunate because it rarely benefits users, tending to obscure the reasons for choosing one approach over another, confuse people, and ultimately delaying the uptake of the standard or technology involved.

Maybe I'm missing the underlying reasons why Oracle and SpringSource decided that Aries wasn't the right project for them. However, I really wish that as an industry driven primarily by technologists we could leave the politics behind and try to work far more collaboratively, and particularly where open source is concerned! As a slight aside, that's one of the things I really like about HPTS: it doesn't matter which company you're from, people talk and interact freely to try to better our collective understanding of problems and lessons learnt.

Update: I should point out that in the paragraph above I wasn't siding with Aries over Gemini, simply that Aries started first.

The future of Java

For one reason or another I've been thinking about the future of Java for a while; more the language than the platform (JEE). I think that the JEE platform will continue to evolve over the coming years, more likely into a series of specific vertical solutions. But this entry isn't about the platform, it's about the language.

Although I've been using Java since it was known as Oak and have written a fair amount with it (for example, I wrote the first ever Java transaction implementation over Christimas 1996), it's never been my favourite programming language. There are some things that I liked about the language from the start, such as threading, but others, such as garbage collection (yes, I like the control of a good delete) and lack of multiple inheritance, that I didn't. The Java language has certainly evolved over the past decade, mostly for the better (garbage collection is a lot better now and we have templates), but in general the language still takes a lowest common denominator approach. And it has got rather bloated.

Ignoring several assembly languages, over the years I've learnt many high level languages including Pascal, Mesa, Smalltalk-80, Lisp, Prolog, C/C++, Simula, D, Forth and of course Java. My favourite is still C++ though. Yes I know it has its flaws and yes I know it's not the most forgiving of languages. I can't quite put my finger on precisely why I still prefer C++. I remember when we were testing the first pre-releases of Cfront for AT&T back in the mid 1980's and wondering along with Graeme and Stuart as to whether we could port it to our Atari's using Metacomco C. I seem to recall us making some progress, but Linux came on the scene bringing with it gcc. But none of this explains why I prefer C++. Maybe it's the level of direct control it gives you (as with C). Or maybe it's something else. Whatever, I'm sure it's all subjective.

Anyway, I digress. Where does this leave Java? Well I think if you look back over the past 40+ years of high level programming languages one thing is very obvious: change happens. Our ability to reason about complex algorithms and the best way of putting them into code evolves, whether it's declarative, procedural, object-oriented or something else. I think it's fairly clear that Java's dominance in the industry will wane. Yes it'll be legacy for many years to come; the 21st Century COBOL, so it will continue to be important. But something new is coming. Maybe not today and maybe not tomorrow. But it's coming nonetheless.

ArchiteCloud 2010

I have the pleasure of bring on the ArchiteCloud 2010 program committee. If you've any papers hiding in your "to do" lists then get them in, as this promises to be a great event! Maybe I can use this as an excuse to visit Australia next year too!

Santa is an architect

It's drawing near to that time of the year again when thoughts turn to snow, presents, turkeys and all things festive. So it was that I was watching a program on TV yesterday where Santa was the main character and my 7 year old and I began to discuss the ways in which Santa manages to get presents to all of the good little girls and boys around the globe in a single night. Of course we covered all of the usual ideas, such as time dilation, wormholes and even time travel. My son thought that magic was the solution, but I pointed out that these days what with global warming and the fact that it's been shown that continual use of magic harms the environment, it's doubtful. Let's also not forget that magic reindeers produce a lot of CO2 as well as other effluent.

So where does that leave us (apart from with a rapidly disillusioned child)? The answer was obvious: although in the past he's probably used a combination of all of the above techniques (have to placate child), today he's taken a software architecture course and figured out that federation works well and scales. He has millions (billions?) of proxies in each country who do his work for them. He sends them information about what needs getting (in advance of course) and relies on them to buy the presents and distribute them locally. Those proxies may themselves have proxies in a recursive manner. Yes we all know it's the elves who build and distributed the toys to the shops, but it's the masses of proxies that get the delivery work done. And of course these helpers are parents, grand-parents etc.

So next time the question arises you'll know the answer: Santa is a coordinator and we're all interposed coordinators in the grand scheme of things ;-)

In-memory durability and HPTS

Back in the 1980's when I was writing the proposal for my PhD work I was looking at various uses for replication (at that point strong consistency protocols). There are a number of reasons for replicating data or an object, including high-availability, fault tolerance through design diversity and improving application performance. For the latter this could include reading data from a physically closer replica, or one that resides on a faster machine available through a faster network path.

But in terms of how replication and transactions could play well together it was using replicas as "fast backing store" aka a highly available in-memory log that seemed the logical thing to concentrate on. We certainly had success in this approach, but the general idea of replication for in-memory durability didn't really seem to take off within the industry until relatively recently. I think one of the important reasons for this is that improvements in network speeds and faster processors have continued to outstrip disk performance, making these kinds of optimization less academic and more mainstream. So it was with a lot of interest that I listened to presentation after presentation at this year's HPTS about this approach. Of course there were presentations on improving disk speeds and using flash drives as a second-level cache too, so it was a good workshop all round.

A great tribute to Jim

Congratulations to Tony, Savas and everyone else involved. This is a great tribute to both the person and scientist that is Jim Gray.

Interesting discussion on SOA Manifesto

I mentioned earlier that I'm on a group collaborating to create a SOA Manifesto along with Steve. Well as part of the effort to inform the public and solicit feedback I produced something for InfoQ. The result of that is a very useful discussion on the comments to that article, which I encourage anyone interested in the SOA Manifesto to check out. Unfortunately I was in Boston last week in meetings so didn't have much chance to participate in the discussion, but as Steve points out, we'll certainly take it to the working group. Thanks to everyone who participated!

SOA Manifesto Working Group

As a member of this working group I can only really second what Steve has to say on the subject.

SOA transactions and reservations

Arnon and I have spoken about SOA transactions in the past, so it was interesting to read his latest article (book chapter?)

It's a nice paper to read, though I disagree with several things he has to say. For instance, I don't like the term "semi-state" or that somehow the use of compensations increases the service's contract footprint. Furthermore compensations (not just Sagas) don't have to use locks. As we kept saying during BTP development, that's a back-end service implementation choice that doesn't have to be exposed to users.

The reservation pattern is important and it's something that's been described several times before, particularly in the area of specific implementation approaches such as BTP, WS-TX and WS-CAF. So although I haven't seen Arnon's book, I can appreciate why he's including this chapter and I liked the discussions on possible risks when using the pattern. However, it would be good to see the aforementioned approaches referenced, particularly since there's a discussion about how you could do this with EJB3.

But the one thing (the elephant in the room) that is glossed over in the discussion (which isn't if you read about the reservation approach elsewhere) is that there's something in the environment directing the flow of interactions across many services. If it looks like a coordinator, behaves like a coordinator and smells like a coordinator then chances are it's a coordinator.

Native transactions

It was interesting reading The Power of Native Transactions presentation, which could probably be subtitled: When You Don't Own a Transaction Manager, What Do You Do? Actually I don't mean to pick on Spring as this is a problem we used to come across many times when an independent TM vendor. Typically we'd hear "Aren't the transactions in my database good enough?", "Why do I need recovery?" and that old one but good one "I'm not interested in distributed transactions so I don't need two-phase commit."

I've said enough times that traditional transactions (aka ACID) aren't suitable for every occasion. The fact they have been shoehorned into situations where they aren't needed or aren't applicable has hurt the industry as well as disillusioned some people. Probably as a result, as we saw throughout the BTP effort people equate two-phase commit with ACID semantics, which is clearly wrong. Or that the two phases have to somehow be tied in with prepare/commit/rollback, which as BTP (and others) showed is again wrong.

At some point I need to finish my paper on consensus in distributed systems, because it's fairly obvious that some people who see transactions (and specifically 2PC) as "evil" and something that can be "programmed around" don't fully understand what's happening under the covers of any good transaction manager. Yes it's possible that some uses of transactions can be redesigned without them, but that doesn't mean they all can be.

Anyway, back to the topic at hand. I read the presentation with interest because it's always good to hear experiences from using transaction systems. What follows is almost a review of the presentation as if I had to review it for inclusion (or not) in a conference. I blame the fact that I sit on so many program committees and review a lot of papers each year: it alters your mindset a bit.

First I agree with one of the core messages in the Spring paper: that sometimes you don't need transactions at all and that one-size doesn't fit all. But sometimes you do need a good transaction manager and you need to understand when and why.

It would have been good to see a discussion of local and global transactions, but then I suppose the point of the talk is to emphasize the use of local transactions (through a native transaction manager). What I find frustrating is that yet again someone equates JDBC save points with nested transactions. Yes you can map a save point to a transaction boundary, but you that doesn't give you nested commit semantics or concurrency control, for instance. If you're going to talk about nested transactions, please stick with the standard definition. Even OTS managed to do that, despite breaking the model in other ways.

I know (hope) that the JBossTS team are going to say a thing or two about the common misunderstandings and misconceptions around XA, so I'll try not to steal their thunder. But I do need to point out that the read-only flag is neither a Spring invention nor something only found in specific databases: it's a core part of two-phase commit (through the read-only vote that can be returned during prepare). So JTA does support it. Of course how the resource manager figures out how to return read-only is something that the transaction manager doesn't mandate (and it shouldn't). So in that regard an annotation makes sense. But other approaches are possible (e.g., making locking rules explicit within the business object.)

The presentation also gives us the notion that 2PC is for distributed transactions only. No! As I said above, 2PC is a consensus protocol so it's needed when you have more than one participant, even if they are on the same machine. Unless of course you're not interested in them reaching agreement. So please, don't equate 2PC with distributed transactions: I also know of many distributed systems using transactions where 2PC isn't the norm!

I hate subjective statements like "XA is Non-Trivial to Set Up." Yes, so is the brake system in my car, but I need it nonetheless! So although XA can be a PITA at times, if used correctly (in the right situations) it's well worth paying. Plus it's often a one-off cost that over the lifetime of a deployment fades into insignificance compared to the benefits, or the setup costs of other components. So try to look on this objectively, weighing up the pros and cons (which is maybe what the presentation is trying to say, although perhaps for obvious reasons it seems to push the non-XA approaches more.)

Something else that surprised me about the presentation was the inference that an application that uses an XA transaction manager, or perhaps one that implements 2PC in general, must incur the overhead of a log even if there's a single resource involved in the transaction (something which using "native" transactions would miraculously avoid). So the one-phase commit optimization hasn't been invented? Once again this is something that's part of XA too. Any good transaction manager implementation should take advantage of this and in this situation you wouldn't get a log created! And I'm not even going to bother about other optimizations such as presumed abort.

I suppose this misunderstanding leads to another subjective statement: "Native Transactions - As Efficient As It Gets". Followed by the recommendation that you should try to use multiple (independent) native transactions, with a single resource in each of course. Yes, that's also one of the extended transaction models we documented when developing The Additional Structuring Mechanisms for the OTS. However, as was pointed out back then, you don't get the same semantics (remember what I was saying about consensus?) If you need atomicity then there are very few alternatives outside of magic or quantum mechanics!

In conclusion, I think the presentation wasn't too bad if a little vendor specific (which makes sense given the background and conference). But if you're really interested in transactions, where to use them, where not to use them, and precisely what the trade-offs are in a more objective manner then there's a lot of other good information out there.

Vacation, decorating, Spring and reservations

I'm on vacation this week, taking the time to decorate my 7 year old son's bedroom. If you want to get away from it all and let your mind come at things from different angles then I can recommend it (though probably not if you're a painter/decorator by trade!) I find it relaxing enough that my mind wanders in a good way. I used to find the same was true when I was a kid at school making and then painting models while thinking through exam questions.

Anyway, over the next few days I hope to write up my thoughts on a couple of things I noticed over the past month or so. One is a presentation on Native Transactions in Spring and the other is on the Reservation Pattern and SOA Transactions. Yes, there's a common theme there!

Elite is 25 years old?!

I didn't realise that one of my favourite games of all time, Elite, is 25 years old! I remember buying this for my BBC Model B computer and playing it at all hours of the day during my early university life. Happy memories!

The best laid plans ...

You go and do something with the best intentions and it suffers a few minor teething problems! With the benefit of hindsight I think we would definitely have gone about it differently, but at least it hasn't all been negative. Hopefully something good will come out of the various discussions, because that was always our aim!

Mind maps

I first came across mind maps back in the last 1980's when a friend/colleague at the University started to use them. I didn't really think any more of them until recently, when another friend pointed me at XMind. Very nice piece of software and it's opened up a whole new way of doing things for me. Gone are my scraps of paper, hastily scrawled notes and other ad hoc approaches!

That time of year again.

Is it really 8 years and counting?!

Google books

Now here's a book I haven't seen in a long time! Three of the authors were/are professors at the University when I was doing my PhD and several of the others visited regularly. A great book and still relevant today!

JBossWorld 2009

It's almost upon us! This year is unique for a couple of reasons: first we're holding the event at the same time and location as Red Hat Summit, which means we could have some interesting audience cross-overs, and second because this will be my first time as CTO since taking over from Sacha!

This year I'm giving a session on our work on testable architectures as well as my keynote. Normally I don't have any difficulties preparing presentations for conferences and workshops. I've done several keynotes in the past for other events, but never one in my current role. It's quite daunting, given the various people I'm following in this role. Maybe it's the masocist in me, but I'm looking forward to it. Now if only I could finish the blasted presentation!

Back and into the fray

Back from vacation and just spent two days catching up on email (lots of things ignored on the assumption that if they're important I'll see them again!) Now getting ready for JBoss World!

Canadian downtime

I'm going on vacation for a couple of weeks in a few days. This is our semi-annual visit to the parents-in-law in Canada. Should be good weather and is so far in the "outback" that doing nothing and relaxing is the norm. Looking forward to it, though I'll be taking some papers to read and write. Blogging will take a back seat though.