Wednesday, May 03, 2006

Complexity theory

I just gave an invited presentation on Web Services at School of Computer Science at Manchester University. I covered basic and advanced Web Services, Web 2.0, REST and Semantic Web (cramming to get all of this into an hour). The audience appeared interested and I got some good follow-up questions. But as you might imagine, it was tricky to get everything in to the allocated time slot.

However, one thing that became apparent to me over the past few days as I wrote the presentation was how difficult it is to actually cover *just* Web Services in an hour! I remember giving talks on Web Services back in 2002/2003 when there was really just SOAP, WSDL and a smattering of UDDI: that's an hour, depending on how much detail you want to go into around SOA. In the intervening years we've standardised on more aspects of the WS architecture and complexity has risen. Now don't get me wrong: I'm not saying that we could avoid the complexity or that it's a bad thing. I've worked on a range of distributed systems architectures over the years and there really is no such thing as a free lunch: if you need advanced facilities such as session management, transactions, security and fault-tolerance then you've got to pay for them somewhere, and pushing things "behind the service endpoint" only works to a point.

Web Services started out as simple (often compared to the complexity of CORBA), but have inevitably gotten more complex as we try to do more and more with them. In some ways, the closest analogy I can come up with is Java: it started out as being "better" than C++ because it was so much simpler - none of those waste-of-space features such as multiple inheritance, pass-by-reference or templates and none of the huge libraries that you never needed (I'm joking here, because I always felt the need for such features). But look at it today: we've now got templates and the JDK is 60Meg! A simple Hello World application takes many Meg to execute!

However, one question I posed during my presentation was: Can it be simpler? What about the WWW? Technically it hasn't changed much and yet we're now working in a Web 2.0 world (I don't like the term, but I agree with the categorisation). The REST approach is often mentioned as more natural for services on the Web than Web Services (evolution rather than revolution), so what about that as the foundation on which to build a simpler and better Web service (small 's') architecture? As you may have gathered, I think the answer is no. That's not to say that I don't think some aspects of a pure REST-ful approach wouldn't be simpler (e.g., URI's have been pretty successful for the WWW, so we may not need something like WS-A), but other aspects may be more complicated (e.g., transactions - my first foray into transactions on the Web started out effectively REST-based I think and it was difficult to do in those days so it ended up quite different). However, the approach does intrigue me enough that, should I ever find any spare time, I'd like to flesh out how a REST service architecture might match up to its Web Services cousin. Anyone wanting to help, just drop me a line.

But back to Web Services! The death of Simple Web Services was signaled as soon as we wanted to do more complicated things with it. We shouldn't ignore that fact and continue as though we were still working in 2002. The S in SOA doesn't stand for, or imply, Simplicity. Whether you're talking services, objects or procedures, complex things such as transactions require complex solutions. This shouldn't come as a surprise to anyone who's been working in distributed systems over the past 20+ years.


Jason Lenhart said...

Hi Mark,

Cannot imagine fitting all your subject matter into one hour. I agree 110% with you that it is not Simple and in essence is difficult to make easier - However I think there may be ways to abstract the complexities giving the illusion of ease. I wonder if the difference with Java and the WS-* specs is that there is a tighter reference model/architecture for vendors to work within. Perhaps frameworks like SCA can be applied within the context of (dare I say it) 'tools' that could use more a Model Driven approach - thus making it easier for the typical company to implement. Perhaps Grid Computing can offer a simpler means for scaling and monitoring. I cannot help but read the WS-* docs (yes I have too much time) and believe that it is 3-5 years outside the mainstream, I would be interested on your thoughts.

Take Care,


Mark Little said...

I think there's definitely room for abstractions, but just not within the WS-* arena. I'd put that in the space of particular language bindings. The problem is, however, that if you go down that route without thinking too much about the underlying model (SOA), you end up just doing distributed objects over SOAP/HTTP, which isn't what we want. MSFT started down that route (primarily because of the existing tooling). I haven't checked recently, but it wasn't pretty and did encourage an OO style. That's fine if all you want is interoperability (which is very important, so don't get me wrong). But if you want loose coupling, that's not the right approach IMO. (Take a look at WS-RF for an example of how quickly Web Services can degenerate into OO.)

I think there's definitely stuff that can be learnt from the Grid community, but it's a two-way street. The WS community has been educating them on not reinventing the wheel (ironic, I know!) so they're now looking to leverage Web Services (but please not WS-RF!)

Back in 1995 the OMG had put the finishing touches to its basic architecture. There have been many revisions since, but enterprise deployments happened pretty quickly (for better or for worse). I think there's a good chance Web Services could have as quick a take up simply because there is so much momentum behind it. The downside is (as happened with CORBA) companies may use technologies (implementations) that aren't up to the task and live to regret it!

Chui said...

I agree with "if you go down that route without thinking too much about the underlying model (SOA), you end up just doing distributed objects over SOAP/HTTP, which isn't what we want."

The distributed world is inherently unreliable.

Should we make it appear to be reliable, and abstract away it's unreliability? Of course not.

Similarly, some of the complex things we may wish to do over WS like 2PC may actually be the wrong world view.

Remember when people started programming distributed objects as they would ordinary objects? Again, case of bringing the wrong paradigm into the distributed world.

As engineers, we all like to believe we can create the world we want. But first, we have to see the web for what it is.

Disconnected, Notification-based, Coarse-grained interfaces.

Discoverability is overrated IMHO. I've never seen someone migrate an Oracle app to SQL Server without porting. Similarly, I don't think anyone will be switching services on the fly.