SOA transactions and reservations

Arnon and I have spoken about SOA transactions in the past, so it was interesting to read his latest article (book chapter?)

It's a nice paper to read, though I disagree with several things he has to say. For instance, I don't like the term "semi-state" or that somehow the use of compensations increases the service's contract footprint. Furthermore compensations (not just Sagas) don't have to use locks. As we kept saying during BTP development, that's a back-end service implementation choice that doesn't have to be exposed to users.

The reservation pattern is important and it's something that's been described several times before, particularly in the area of specific implementation approaches such as BTP, WS-TX and WS-CAF. So although I haven't seen Arnon's book, I can appreciate why he's including this chapter and I liked the discussions on possible risks when using the pattern. However, it would be good to see the aforementioned approaches referenced, particularly since there's a discussion about how you could do this with EJB3.

But the one thing (the elephant in the room) that is glossed over in the discussion (which isn't if you read about the reservation approach elsewhere) is that there's something in the environment directing the flow of interactions across many services. If it looks like a coordinator, behaves like a coordinator and smells like a coordinator then chances are it's a coordinator.

Native transactions

It was interesting reading The Power of Native Transactions presentation, which could probably be subtitled: When You Don't Own a Transaction Manager, What Do You Do? Actually I don't mean to pick on Spring as this is a problem we used to come across many times when an independent TM vendor. Typically we'd hear "Aren't the transactions in my database good enough?", "Why do I need recovery?" and that old one but good one "I'm not interested in distributed transactions so I don't need two-phase commit."

I've said enough times that traditional transactions (aka ACID) aren't suitable for every occasion. The fact they have been shoehorned into situations where they aren't needed or aren't applicable has hurt the industry as well as disillusioned some people. Probably as a result, as we saw throughout the BTP effort people equate two-phase commit with ACID semantics, which is clearly wrong. Or that the two phases have to somehow be tied in with prepare/commit/rollback, which as BTP (and others) showed is again wrong.

At some point I need to finish my paper on consensus in distributed systems, because it's fairly obvious that some people who see transactions (and specifically 2PC) as "evil" and something that can be "programmed around" don't fully understand what's happening under the covers of any good transaction manager. Yes it's possible that some uses of transactions can be redesigned without them, but that doesn't mean they all can be.

Anyway, back to the topic at hand. I read the presentation with interest because it's always good to hear experiences from using transaction systems. What follows is almost a review of the presentation as if I had to review it for inclusion (or not) in a conference. I blame the fact that I sit on so many program committees and review a lot of papers each year: it alters your mindset a bit.

First I agree with one of the core messages in the Spring paper: that sometimes you don't need transactions at all and that one-size doesn't fit all. But sometimes you do need a good transaction manager and you need to understand when and why.

It would have been good to see a discussion of local and global transactions, but then I suppose the point of the talk is to emphasize the use of local transactions (through a native transaction manager). What I find frustrating is that yet again someone equates JDBC save points with nested transactions. Yes you can map a save point to a transaction boundary, but you that doesn't give you nested commit semantics or concurrency control, for instance. If you're going to talk about nested transactions, please stick with the standard definition. Even OTS managed to do that, despite breaking the model in other ways.

I know (hope) that the JBossTS team are going to say a thing or two about the common misunderstandings and misconceptions around XA, so I'll try not to steal their thunder. But I do need to point out that the read-only flag is neither a Spring invention nor something only found in specific databases: it's a core part of two-phase commit (through the read-only vote that can be returned during prepare). So JTA does support it. Of course how the resource manager figures out how to return read-only is something that the transaction manager doesn't mandate (and it shouldn't). So in that regard an annotation makes sense. But other approaches are possible (e.g., making locking rules explicit within the business object.)

The presentation also gives us the notion that 2PC is for distributed transactions only. No! As I said above, 2PC is a consensus protocol so it's needed when you have more than one participant, even if they are on the same machine. Unless of course you're not interested in them reaching agreement. So please, don't equate 2PC with distributed transactions: I also know of many distributed systems using transactions where 2PC isn't the norm!

I hate subjective statements like "XA is Non-Trivial to Set Up." Yes, so is the brake system in my car, but I need it nonetheless! So although XA can be a PITA at times, if used correctly (in the right situations) it's well worth paying. Plus it's often a one-off cost that over the lifetime of a deployment fades into insignificance compared to the benefits, or the setup costs of other components. So try to look on this objectively, weighing up the pros and cons (which is maybe what the presentation is trying to say, although perhaps for obvious reasons it seems to push the non-XA approaches more.)

Something else that surprised me about the presentation was the inference that an application that uses an XA transaction manager, or perhaps one that implements 2PC in general, must incur the overhead of a log even if there's a single resource involved in the transaction (something which using "native" transactions would miraculously avoid). So the one-phase commit optimization hasn't been invented? Once again this is something that's part of XA too. Any good transaction manager implementation should take advantage of this and in this situation you wouldn't get a log created! And I'm not even going to bother about other optimizations such as presumed abort.

I suppose this misunderstanding leads to another subjective statement: "Native Transactions - As Efficient As It Gets". Followed by the recommendation that you should try to use multiple (independent) native transactions, with a single resource in each of course. Yes, that's also one of the extended transaction models we documented when developing The Additional Structuring Mechanisms for the OTS. However, as was pointed out back then, you don't get the same semantics (remember what I was saying about consensus?) If you need atomicity then there are very few alternatives outside of magic or quantum mechanics!

In conclusion, I think the presentation wasn't too bad if a little vendor specific (which makes sense given the background and conference). But if you're really interested in transactions, where to use them, where not to use them, and precisely what the trade-offs are in a more objective manner then there's a lot of other good information out there.

Vacation, decorating, Spring and reservations

I'm on vacation this week, taking the time to decorate my 7 year old son's bedroom. If you want to get away from it all and let your mind come at things from different angles then I can recommend it (though probably not if you're a painter/decorator by trade!) I find it relaxing enough that my mind wanders in a good way. I used to find the same was true when I was a kid at school making and then painting models while thinking through exam questions.

Anyway, over the next few days I hope to write up my thoughts on a couple of things I noticed over the past month or so. One is a presentation on Native Transactions in Spring and the other is on the Reservation Pattern and SOA Transactions. Yes, there's a common theme there!

Elite is 25 years old?!

I didn't realise that one of my favourite games of all time, Elite, is 25 years old! I remember buying this for my BBC Model B computer and playing it at all hours of the day during my early university life. Happy memories!

The best laid plans ...

You go and do something with the best intentions and it suffers a few minor teething problems! With the benefit of hindsight I think we would definitely have gone about it differently, but at least it hasn't all been negative. Hopefully something good will come out of the various discussions, because that was always our aim!

Mind maps

I first came across mind maps back in the last 1980's when a friend/colleague at the University started to use them. I didn't really think any more of them until recently, when another friend pointed me at XMind. Very nice piece of software and it's opened up a whole new way of doing things for me. Gone are my scraps of paper, hastily scrawled notes and other ad hoc approaches!

That time of year again.

Is it really 8 years and counting?!

Google books

Now here's a book I haven't seen in a long time! Three of the authors were/are professors at the University when I was doing my PhD and several of the others visited regularly. A great book and still relevant today!

JBossWorld 2009

It's almost upon us! This year is unique for a couple of reasons: first we're holding the event at the same time and location as Red Hat Summit, which means we could have some interesting audience cross-overs, and second because this will be my first time as CTO since taking over from Sacha!

This year I'm giving a session on our work on testable architectures as well as my keynote. Normally I don't have any difficulties preparing presentations for conferences and workshops. I've done several keynotes in the past for other events, but never one in my current role. It's quite daunting, given the various people I'm following in this role. Maybe it's the masocist in me, but I'm looking forward to it. Now if only I could finish the blasted presentation!

Back and into the fray

Back from vacation and just spent two days catching up on email (lots of things ignored on the assumption that if they're important I'll see them again!) Now getting ready for JBoss World!

Canadian downtime

I'm going on vacation for a couple of weeks in a few days. This is our semi-annual visit to the parents-in-law in Canada. Should be good weather and is so far in the "outback" that doing nothing and relaxing is the norm. Looking forward to it, though I'll be taking some papers to read and write. Blogging will take a back seat though.

Facebook

I don't use Facebook or any social networking site, but I know people who do, including my wife. So I was surprised and annoyed when I heard via Savas that "Facebook has agreed to let third party advertisers use your posted pictures without your permission. Click on SETTINGS up at the top where you see the log out link. Select PRIVACY. Then select NEWS FEEDS AND WALL. Next select the tab that reads FACE BOOK ADS. There is a drop down box, select NO ONE. Then SAVE your changes. Please repost!"

I'm doing more and more work with "the cloud" (whatever that means!) and it's things like this (security, identity, trust) that concern me. Maybe when I finalize my opinions around this and other aspects I'll write something up. For now beware. And if you're on Facebook maybe you should change your setting?

Feynman

I grew up on Richard Feynman though wasn't really able to appreciate him until I started my physics undergraduate degree. He's definitely a hero of mine. One of the best programs I've watched about him didn't actually concentrate on his deep physics background, but was about his trip to Tuvalu. So it was with pleasure that I came across Project Tuva from our friends at MSFT.

Memories ...

While looking for something else, I came across this old ANSA/Esprit project proposal from about 1989. One of the earliest external project memories I have and a good one too! Then I thought "Oh, that's 2 decades ago! Oh frack!"

SoyLatte

So far my adventures in JDK 1.6 land have been constrained to a 10 year old Windows box, so I decided to upgrade my Mac. Unfortunately the machine I've got isn't 64 bit so the official Apple version doesn't work. Fortunately the SoyLatte distribution seems to work very well. Yet another example of good open source at work.

Scala round 2

I mentioned earlier this year how Mic had pointed me at Scala. I haven't had nearly enough time to play with it (which usually means me implementing a transaction service in the language!) but from what I've seen I'm impressed. It's interesting to read what others think too.

Walking with Dinosaurs

I remember watching Walking with Dinosaurs when it first aired on TV and being impressed with the graphics and story. The rest of my family were equally impressed. So when we heard about the Arena Spectacular version and that it was coming near us, it was something we had to see. I took the day off work to make sure no one could plan anything for me to get in the way and we went to see it: well worth it and I'm extremely impressed! So if you like dinosaurs or your kids do then go and see it if you get the chance. If you don't have any kids then borrow some so you won't look out of place!

Principles of Transaction Processing

I've known Eric as a colleague and friend for a long time, but my first run in with him was with his and Phil's first edition of Principles of Transaction Processing. Along with Jim's book, it has a key place on my bookshelf. Over the years I've probably bought a dozen or so copies for various groups I've worked with. Well it's good to be able to say that they've released a second edition. I was one of the reviewers of the book over the past couple of years so I know this is a solid replacement for the original. Well done!

Supporting performance?

Given my background in fault tolerant distributed systems, performance metrics are something that come up time and time again. I've spent many a happy hour, day, week, month pouring over timings, stack dumps etc. to figure out how to make RPCs go faster, replication to scale and transaction logs to perform. (Funnily enough I was doing the latter again over Christmas!) But this is something we've all had to do when working on Arjuna.

If you've ever had to do performance tuning and testing then you'll know that it can be fun, but often an almost never ending task. Then there's the comparisons between different implementations. This isn't just a vendor-specific area: even in academia there's always a bit of "mine is faster than yours" mentality. I suppose it's human nature and in some ways it's good and pushes you to improve. However, performance measurements need to be taken in a scientific manner otherwise they are useless. It's no good to simply state that you get "X transactions a second" if you don't specify the conditions under which that figure was achieved. This is true for a number of performance metrics, not just in the computing industry: there's a reason world records in athletics have defined conditions, for instance.

Now of course you may not have the same hardware to test and the tests themselves can be notoriously hard to access from one vendor/competitor to another. What tuning parameters they use as well as the general configuration can also be difficult to obtain. This is why Jim Gray and others wrote A Measure of Transaction Processing Power (though the principles in there can be applied much more widely.) This eventually produced the TPPC. But the general principle here is a scientific approach to performance metrics is the right way (though it's still possible to skew them in your favour.)

Whenever I see or hear of performance figures I always want to know the conditions (hardware, configuration, wind conditions etc.) It's not possible for me to take them seriously unless I have this additional data. Giving out the tests themselves is also a good thing. However, performance is not something you can support like you do with, say, transactions, where you either are transactional or you're not. With performance it can be a never ending struggle to keep improving, especially as hardware and software improve and your competitors get better too; you've always got performance even if it's bad! Performance is something that for some users can be as critical as new capabilities, or even more so. But it can't be performance at the cost of reliability or robustness: something that screams along faster than a speeding bullet yet crashes after 24 hours is unlikely to be as useful as something that is half as fast yet able to stay available indefinitely.

Internet dial tone?

The other day I had problems with my ISP that meant my connection to the rest of the world was up and down like a yo-yo. I self diagnosed the problem (the ISP help desk was pretty useless) and in the course of that I realized something: where's the equivalent of the telephone dial tone for the internet? Well if you're like me it's either Google or BBC news. Over the years whenever I need to determine if there's a problem with my connection or some site I'm trying to use, I'll use one of these sites to double check my own connectivity. OK it's not exactly a science, but I assume if I can talk to one or other of them then the fault lies elsewhere.