The trouble with transactions ...

The folks over at Microsoft working on Indigo have been doing some interesting stuff with their transaction engine. I've been keeping abreast on this as much as possible for some time, so it's nice to see it coming to fruition. What's even more interesting from a personal perspective, is that the architecture they've come up with and the requirements it fulfills, is very similar to what we've been using now for many years: basically a lightweight transaction engine that can be used in situations where not all of the ACID properties are required, but which can dynamically "step up to the plate" (as one Microsoft friend described it) if the need arises.

In some previous posts I've touched on the fact that we have in house something that fits that bill: ArjunaCore. This is a transaction coordinator (plus some associated tools and development frameworks) that has a very small footprint (hey, it runs on an old HP Jornada 720 in significantly less than 32 Meg of memory!) It took us several years of design and development to come to the current architecture, that allows it to be used within every transaction-related product we currently have. Without going into too many details (read the paper), it allows us to relax some or all of the various ACID properties in a controlled manner within products and is extremely fast and agile.

Note, this is not a "one-size fits all" protocol I'm talking about here: it's purely implementation. There's an important difference and the evolution of ArjunaCore has simply reinforced that distinction in my mind. I'm in the process of writing a paper on this, so will hold off saying too much more here.

Transaction processing is a difficult technology to implement well and also to convince people that they need. Terms such as "two-phase commit" and "persistence" are ingrained into people's minds as bad things, synonymous with "poor performance" and "high overhead". When talking to people about how they can use transactions, I've often heard reasons for them not using them begin with "Well, the trouble with transactions in our view is ..." But that needn't be the case and in many situations, the lack of transactions can lead to a much higher overhead if failures occur. Again, I'm writing a paper on this subject at the moment, but may blog on some aspects of it later.

I've had the opportunity to work for the Microsoft/Indigo crowd on several occasions but the timing has never been right. So it's nice to see two independent groups coming to the same ultimate conclusion about how to bring transactions to the masses.

There are many applications out there that don't use transactions and should; likewise, there are applications using transactions that really don't need to. I've believed for a long time that there are several things that we as an industry can (and should) do to help people understand where transactions fit into their architecture: education (why are transactions important? what happens if you don't have them? etc.), ease of use, flexibility (this crosses many different aspects, but includes what can be controlled, e.g., transactions existed before XA, so you don't have to limit yourself to XA-aware participants), performance etc.

We've managed to get a lot of the implementation-side of the equation into ArjunaCore and are constantly working on the education aspect. I know other companies and organisations are starting to look at a similar approach, and in the meantime it's always nice to get some implicit validation from a company like Microsoft.