We've decided to put up a public endpoint for the WS-Context interoperability demonstrator at http://services.arjuna.com:8080/wscafdemo. In order to use it you'll need to check out the TC use case document, which for some reason isn't publicly available off the WS-CAF TC homepage. I'll post an update later if it turns out we (the TC) can make this available. Thanks go to my colleagues Malik Saheb and Kevin Conner for making this possible.
I work for Red Hat, where I lead JBoss technical direction and research/development. Prior to this I was SOA Technical Development Manager and Director of Standards. I was Chief Architect and co-founder at Arjuna Technologies, an HP spin-off (where I was a Distinguished Engineer). I've been working in the area of reliable distributed systems since the mid-80's. My PhD was on fault-tolerant distributed systems, replication and transactions. I'm also a Professor at Newcastle University and Lyon.
Wednesday, November 24, 2004
The Activity Service
Recently I wrote an article with Bruce Martin about the J2EE Activity Service. In essence, what we show is that ACID transactions aren't sufficient for everything and there are a number of extended transaction models that allow for the controlled relaxation of the ACID properties. What this means is that one size doesn't fit all and as usual it is necessary to use the right tool for the right job. Anyone who's worked in the area of (distributed) transactions for a while will typically have a story or two to tell about where ACID transactions were shoe-horned into areas they simply weren't suitable for.
Therefore, rather than provide support for a single model, we did work with IBM, IONA Technologies, Bank of America, Alcatel and others in the OMG on the Additional Structuring Mechanisms for the OTS. This defines an infrastructure to support a wide range of extended transaction models. The architecture is based on the insight that the various extended transaction models can be supported by providing a general purpose event signaling mechanism that can be programmed to enable activities (application specific units of computations) to coordinate each other in a manner prescribed by the extended transaction model under consideration.
Once this was adopted within the OMG, IBM, ourselves and others moved to push this into J2EE via JSR 95: the J2EE Activity Service. This has recently been finalised and presumably will become part of some future version of J2EE.
Now whether or not this work will actually change the way people develop transactional applications in J2EE is open to debate. I know that when we were doing the work it seemed "obvious" that this was the future of transactions and alongside IBM we've had some interesting development experiences. But the world has moved on since 1997 and I often have interesting discussions with Greg and others as to whether the Activity Service is of much use. Only time will tell.
However, what cannot be in doubt is the influence this work has had elsewhere. Since the original OMG work was released, it has influenced the WS-Coordination specification from IBM, Microsoft and BEA, as well as the WS-Coordination Framework from ourselves, IONA, Oracle, Sun and Fujitsu. It also formed the basis of our initial submission to the OASIS BTP technical committee back in 2001; though its influence after that point was mimimal.
In some ways the new OASIS WS-CAF architecture, which factored context our of the original Activity Service, is a purer form of the model we were trying to achieve back then.
Therefore, rather than provide support for a single model, we did work with IBM, IONA Technologies, Bank of America, Alcatel and others in the OMG on the Additional Structuring Mechanisms for the OTS. This defines an infrastructure to support a wide range of extended transaction models. The architecture is based on the insight that the various extended transaction models can be supported by providing a general purpose event signaling mechanism that can be programmed to enable activities (application specific units of computations) to coordinate each other in a manner prescribed by the extended transaction model under consideration.
Once this was adopted within the OMG, IBM, ourselves and others moved to push this into J2EE via JSR 95: the J2EE Activity Service. This has recently been finalised and presumably will become part of some future version of J2EE.
Now whether or not this work will actually change the way people develop transactional applications in J2EE is open to debate. I know that when we were doing the work it seemed "obvious" that this was the future of transactions and alongside IBM we've had some interesting development experiences. But the world has moved on since 1997 and I often have interesting discussions with Greg and others as to whether the Activity Service is of much use. Only time will tell.
However, what cannot be in doubt is the influence this work has had elsewhere. Since the original OMG work was released, it has influenced the WS-Coordination specification from IBM, Microsoft and BEA, as well as the WS-Coordination Framework from ourselves, IONA, Oracle, Sun and Fujitsu. It also formed the basis of our initial submission to the OASIS BTP technical committee back in 2001; though its influence after that point was mimimal.
In some ways the new OASIS WS-CAF architecture, which factored context our of the original Activity Service, is a purer form of the model we were trying to achieve back then.
Saturday, November 20, 2004
Better late than never
WS-Context has gone to committee draft in OASIS WS-CAF, which is somewhat of a relief, as it means we can move on to the WS-Coordination Framework specification.
As I've said in an earlier post, WS-Context took me by surprise with its applicability in a wide area, despite the fact that it is probably conceptually the simplest of the 3 specifications we're looking at in the technical committee. IMO one of the most important areas where WS-Context can assist is in maintaining a scalable Web Services architecture for statefull Web Services. I agree with Jim Gray that truly stateless Web Services aren't of any real use, and hope to have a paper I've written on the subject published soon. State is a necessity, it's just what constitutes state that may be open for debate.
In this post I'll try to give a flavour of how I think WS-Context fits into this debate. The area I'll consider is that of stateful sessions, where a user wants to return to a specific state time and again over the course of hours, days, weeks etc. Now in Web Services there are essentially two proposed ways of doing this:
(i) WS-Addressing, which uses ReferenceProperties to explicitly bind a Web Services address to a specific session (and hence state).
(ii) WS-Context, which provides a more lightweight, generalized session model. One analogy that you could use, is that of a Web Services cookie, that isn't tied to a single Web server: I'm glad we didn't call the specificatoin WS-Cookie though.
Both models support stateful interactions over a period of time. The problem is that (i) encourages tightly coupled system design: the address is only good for a specific session, and it is tied to a single endpoint. If you've only got to manage a small number of these addresses, then you obviously could, but if the session spans hours or days then you're going to want to make them durable somehow. That's not necessarily a hardship, unless you have many of them.
However, if a client application interacts with multiple services within the same logical session, then it is often the case that the state of a service has relevance to the client only when used in conjunction with the associated states of the other services. This necessarily means that the client must remember each service reference and somehow associate them with a specific interaction; multiple interactions will obviously result in different reference sets that may be combined to represent each sessions.
For example, if there are N services used within the same application session, each maintaining m different states, the client application will have to maintain N*m reference endpoints. This obviously does not scale. However, an alternative approach is to use (ii). Each interaction with a set of services can be modeled as a session and this in turn can be modeled as a WS-Context activity with an associated context (at a minimum essentially just a URI). Whenever a client application interacts with a set of services within the same session, the same context (same URI) is propagated to the services and they map this context to the necessary states that the client interaction requires. How this mapping occurs is an implementation specific choice that doesn't need to be exposed to the client. Furthermore, since each service within a specific session gets the same context, upon later revisiting these services and providing the same context again, the client application can be sure to return to a consistent set of states. You only need to remember the context, no matter how many services your application uses in that activity. Thus, this model scales much better.
The Web Services architecture is not prescriptive about what happens behind service endpoints. This gives flexibility of implementation, allowing systems to adapt to changes in requirements, technology etc. without directly affecting users. It also means that issues such as whether or not a service maintains state on behalf of users or their (temporally bounded) interactions, has been an implementation choice not typically exposed to users. The WS-Context session model encourages loose coupling of services and their users and keeps any implementation specific choices about state where they belong: behind the service endpoint.
As I've said in an earlier post, WS-Context took me by surprise with its applicability in a wide area, despite the fact that it is probably conceptually the simplest of the 3 specifications we're looking at in the technical committee. IMO one of the most important areas where WS-Context can assist is in maintaining a scalable Web Services architecture for statefull Web Services. I agree with Jim Gray that truly stateless Web Services aren't of any real use, and hope to have a paper I've written on the subject published soon. State is a necessity, it's just what constitutes state that may be open for debate.
In this post I'll try to give a flavour of how I think WS-Context fits into this debate. The area I'll consider is that of stateful sessions, where a user wants to return to a specific state time and again over the course of hours, days, weeks etc. Now in Web Services there are essentially two proposed ways of doing this:
(i) WS-Addressing, which uses ReferenceProperties to explicitly bind a Web Services address to a specific session (and hence state).
(ii) WS-Context, which provides a more lightweight, generalized session model. One analogy that you could use, is that of a Web Services cookie, that isn't tied to a single Web server: I'm glad we didn't call the specificatoin WS-Cookie though.
Both models support stateful interactions over a period of time. The problem is that (i) encourages tightly coupled system design: the address is only good for a specific session, and it is tied to a single endpoint. If you've only got to manage a small number of these addresses, then you obviously could, but if the session spans hours or days then you're going to want to make them durable somehow. That's not necessarily a hardship, unless you have many of them.
However, if a client application interacts with multiple services within the same logical session, then it is often the case that the state of a service has relevance to the client only when used in conjunction with the associated states of the other services. This necessarily means that the client must remember each service reference and somehow associate them with a specific interaction; multiple interactions will obviously result in different reference sets that may be combined to represent each sessions.
For example, if there are N services used within the same application session, each maintaining m different states, the client application will have to maintain N*m reference endpoints. This obviously does not scale. However, an alternative approach is to use (ii). Each interaction with a set of services can be modeled as a session and this in turn can be modeled as a WS-Context activity with an associated context (at a minimum essentially just a URI). Whenever a client application interacts with a set of services within the same session, the same context (same URI) is propagated to the services and they map this context to the necessary states that the client interaction requires. How this mapping occurs is an implementation specific choice that doesn't need to be exposed to the client. Furthermore, since each service within a specific session gets the same context, upon later revisiting these services and providing the same context again, the client application can be sure to return to a consistent set of states. You only need to remember the context, no matter how many services your application uses in that activity. Thus, this model scales much better.
The Web Services architecture is not prescriptive about what happens behind service endpoints. This gives flexibility of implementation, allowing systems to adapt to changes in requirements, technology etc. without directly affecting users. It also means that issues such as whether or not a service maintains state on behalf of users or their (temporally bounded) interactions, has been an implementation choice not typically exposed to users. The WS-Context session model encourages loose coupling of services and their users and keeps any implementation specific choices about state where they belong: behind the service endpoint.
Thursday, November 18, 2004
Java on the slide?
I came across this a few years ago and it's interesting to monitor periodically. I see that Java has taken over an 8% dive recently and is now overtaken by C. Assuming their measurement metrics are right, then if you check out the graph there's been a downward trend for Java since 2001. I don't think this is surprising or necessarily the deathknell for the language. There was a lot of hype for Java 4+ years back and that will have encouraged people to try it when it simply isn't suitable for all use cases. Now I suspect it's just starting to find its right level.
I love Java and have used it since it was calledOak: my first program in it was a game where you could shoot off bits of Duke in a fairly gorey manner. However, my favourite programming language still has to be C++.
I love Java and have used it since it was calledOak: my first program in it was a game where you could shoot off bits of Duke in a fairly gorey manner. However, my favourite programming language still has to be C++.
Web Services context, coordination and transactions
Well Doug Bunting and I gave our presentation on the state of the Web Services architecture and how OASIS WS-CAF could be used to fill in some of the gaps. I thought it went OK and we had a few good questions from the audience and more offline. Unfortunately the proceedings from the conference aren't publicly available, so I'll have to wait until I get back to the office before I put a copy on our Web site.
It's amazing how these things pan out, but WS-Context was always the specification I assumed would interest people the least. However, and with the benefit of 20-20 hindsight, because it probably has the most relevance, it is the one that everyone latches onto most often. It's always nice to have people come up and give you new use cases for how context (and the notion of activity) fit nicely into their architecture. It kind of reinforces that the work we're doing isn't just useful in a narrow field.
Fairly obviously though, I think that the 3 specifications that form WS-CAF (WS-Coordination Framework and WS-Transaction Management being the other two) create a "pyramid of interest", tapering to transactions at the top. Put another way, the further up the stack we go, the less likely people are to find them generally useful. Don't get me wrong though, that's not necessarily a bad thing.
It's amazing how these things pan out, but WS-Context was always the specification I assumed would interest people the least. However, and with the benefit of 20-20 hindsight, because it probably has the most relevance, it is the one that everyone latches onto most often. It's always nice to have people come up and give you new use cases for how context (and the notion of activity) fit nicely into their architecture. It kind of reinforces that the work we're doing isn't just useful in a narrow field.
Fairly obviously though, I think that the 3 specifications that form WS-CAF (WS-Coordination Framework and WS-Transaction Management being the other two) create a "pyramid of interest", tapering to transactions at the top. Put another way, the further up the stack we go, the less likely people are to find them generally useful. Don't get me wrong though, that's not necessarily a bad thing.
Arjuna Technologies blog
At the moment Arjuna Technologies doesn't have support for blogs, but hopefully that'll change shortly. Watch this space.
I'd encourage anyone interesting in reliable middleware for J2EE or Web Services to check out Arjuna. We've got some pretty cool products and a collection of bright people behind them. Despite the fact that Hewlett-Packard decided to drop out of the middleware business, these guys stuck together and did a great job of pushing the envelope.
I'd encourage anyone interesting in reliable middleware for J2EE or Web Services to check out Arjuna. We've got some pretty cool products and a collection of bright people behind them. Despite the fact that Hewlett-Packard decided to drop out of the middleware business, these guys stuck together and did a great job of pushing the envelope.
WWW 2005 update
Myself and Dan Suciu (Washington State University) just made our first pass over the papers for our Web Services track. Looks like everyone will have about 10 papers :-(
The rough breakdown on subject area seems to be:
Practice and experience: 10%
Formal methods: 5%
Query and searching: 20%
XPath: 5%
XML: 25%
Fault tolerance: 5%
Workflow: 10%
Principles of Web Services: 15%
Miscellaneous: 5%
I need to think a bit about this to see if there's any interesting conclusions to come out of the distribution (hey, it's 8am here in DC and I've been up since 3am).
The rough breakdown on subject area seems to be:
Practice and experience: 10%
Formal methods: 5%
Query and searching: 20%
XPath: 5%
XML: 25%
Fault tolerance: 5%
Workflow: 10%
Principles of Web Services: 15%
Miscellaneous: 5%
I need to think a bit about this to see if there's any interesting conclusions to come out of the distribution (hey, it's 8am here in DC and I've been up since 3am).
Gartner mentions WS-CAF
I was speaking to Eric last night. He's been attending the Gartner conference/summit on Web Services in Florida. (Probably better weather than we're having here in DC.)
He told me that Gartner said that there were only 3 technical committees in OASIS at the moment that are doing anything worth watching, and one of those is WS-CAF.
I wonder what the other two are ;-) ?
He told me that Gartner said that there were only 3 technical committees in OASIS at the moment that are doing anything worth watching, and one of those is WS-CAF.
I wonder what the other two are ;-) ?
Wednesday, November 17, 2004
Searching for White Dwarfs
Savas, Jim and others have been doing some nice work on combining Grid and Web Services, called WS-GAF. Check it out if you want to see how it is possible to implement Grids using today's Web Services and still maintain the SOA principles.
One of the applications that they are using to exemplify their techniques is based on searching for White Dwarf stars. I spent over a third of my university undergraduate degree doing astrophysics, so I find this really interesting. Hey, maybe this application could be developed into the next generation Seti@home.
One of the applications that they are using to exemplify their techniques is based on searching for White Dwarf stars. I spent over a third of my university undergraduate degree doing astrophysics, so I find this really interesting. Hey, maybe this application could be developed into the next generation Seti@home.
More interesting reads
If you're looking for interesting things to read, then I'd recommend checking out the blogs of Jim and Savas. They're friends and ex-work colleagues, which whom I may not always agree but have always had interesting discussions.
I don't think I've ever had a conversation with them about work that I came away from thinking it was a waste of time. Other topics, such as Savas' tendency to believe that the Greek nation invented everything first, is another matter ;-)
We've written quite a few articles and papers together over the years and hopefully that'll continue.
I don't think I've ever had a conversation with them about work that I came away from thinking it was a waste of time. Other topics, such as Savas' tendency to believe that the Greek nation invented everything first, is another matter ;-)
We've written quite a few articles and papers together over the years and hopefully that'll continue.
A Washington Monument
When I first came to DC many years ago for a conference in Annapolis, I came with my friend Stuart and we spent a few days looking around. It was great and we saw all of the usual monuments and sights. At the time one of Stuart's friends who lived here gave us a tour (tip: get a local to show you round, you see more than in the guide books) and took us to see a monument/sculpture/work-of-art that I'd like to find again, but just don't know where to look. It was a giant hand coming up from the ground, near the ocean. Very cool looking. Must take a look through some of the local guide books.
WWW 2005
I'm a co-chair of the Web Services track for WWW2005, which is going to be in Japan. I doubt I'll be able to attend the actual conference, which is a shame since I've heard interesting things about Japan from friends like Jim and Lindsay.
Regardless, it's been nice to be co-chair, until the submitted papers came flooding in. Looks like all of us on the Web Service track will be looking at about a dozen different papers to review over Christmas. Oh well, I suppose it beats watching the Queen's speach!
Regardless, it's been nice to be co-chair, until the submitted papers came flooding in. Looks like all of us on the Web Service track will be looking at about a dozen different papers to review over Christmas. Oh well, I suppose it beats watching the Queen's speach!
That was easy!
As anyone who knows me can attest, I've been saying for years that I won't blog, because I didn't see the point. So I feel a bit strange doing this. To be honest, I felt forced into it ;-) Over the past few years I've watched friends like Greg Pavlik, Eric Newcomer, Steve Vinoski and Mark Potts all start blogging (and that is by no means a complete list), so there must be something in it! And when I found myself reading their blogs as well as others and finding them interesting, something clicked.
So here we are. I'm not saying that what I'll be writing will be interesting or that you'll want to read it, but at least it'll be fun - I hope.
So here we are. I'm not saying that what I'll be writing will be interesting or that you'll want to read it, but at least it'll be fun - I hope.
WS-Context and XML 2004
I'm here at XML 2004 in Washington DC for a couple of reasons. The first is that we're doing an interoperability demonstration with Oracle and IONA Technologies of our respective WS-Context implementations, just to prove that it is an interoperable spec. and (hopefully) soon-to-be standard.
This is cool, because I've been working on this notion of context for the best part of a decade, starting with some nice work I did with my friend Stuart Wheater and friends/colleagues at IBM Hursely (Ian Robinson, Tom Freund, Tony Storey and Iain Houston to name a few) and IONA Technologies (Eric Newcomer) on the OMG Activity Service (aka Additional Structuring Mechanisms for the OTS specification - bit of a mouthful!) and moving through J2EE to Web Services. So it's nice to see the fruits of your work, so to speak. However, it's morphed quite a bit over the years, particularly over the past couple. And it's nice to see a wider adoption (and constructive input) by companies such as Oracle.
It's also nice to see more and more specifications coming out that use context. WS-Enumeration, WS-Security, WS-Reliability to name a few. Now if only we could encourage these guys to use a standard context format rather than developing ad hoc solutions.
Although Web Services are as much about interoperability as they are the Web, it's interesting to note that interoperability isn't something that's a given for Web Services specifications. It's like anything: unless you design with it in mind from the outset, you can quickly end up with a closed system. For example, without naming names, there are several WS specifications out there that define extensibility elements (##other to you and me). Now that isn't necessarily a bad thing (hey, we do the same in WS-Context). It becomes a bad thing when either the specification requires you to use them (and doesn't provide concrete rules for the contents) or implementations require them to be filled with vendor specific information (and again, often don't externally publish this information for other vendors to use). IMO this is an easy route back to vendor lock-in.
We had the same issue back in the OMG OTS days, where the transaction context defines an extensibility element (a CORBA any). The intention was that an implementation could fill this with whatever information it needed to provide optimizations for itself (e.g., build up a subordinate transaction hierarchy in a downstream node when a request is made on a service on that node and then pass it back up to the parent when the response is delivered, rather than call back to the parent as you build up the hierarchy before executing the request), but if that information wasn't present then heterogeneous implementations could still interact, albeit in a possibly less efficient manner. However, it quickly got hijacked and implementations required the CORBA::any to be present and filled with specific information in order for them to work. Quite nice it you want to prevent interoperability and encourage lock-in.
Anyway, I digress. The WS-Context interoperability demo hosted by OASIS went well. We used two local implementations (Arjuna's running on my laptop and IONA's running on their own laptop) and a remote on (Oracle). It'd be nice to say that that's the configuration we always intended, because it shows us talking form DC to Philadelphia, where the Oracle endpoint is located, but I'd be lying. The reason we did this was that poor Simeon (Green) from Oracle had his laptop die on him an hour before the demonstration! Sods law!!
Tomorrow Doug Bunting (Sun) and I are presenting on WS-CAF and how it fills some of the gaps in the current Web Services architecture. Hopefully that will go as well as the interop. demo.
This is cool, because I've been working on this notion of context for the best part of a decade, starting with some nice work I did with my friend Stuart Wheater and friends/colleagues at IBM Hursely (Ian Robinson, Tom Freund, Tony Storey and Iain Houston to name a few) and IONA Technologies (Eric Newcomer) on the OMG Activity Service (aka Additional Structuring Mechanisms for the OTS specification - bit of a mouthful!) and moving through J2EE to Web Services. So it's nice to see the fruits of your work, so to speak. However, it's morphed quite a bit over the years, particularly over the past couple. And it's nice to see a wider adoption (and constructive input) by companies such as Oracle.
It's also nice to see more and more specifications coming out that use context. WS-Enumeration, WS-Security, WS-Reliability to name a few. Now if only we could encourage these guys to use a standard context format rather than developing ad hoc solutions.
Although Web Services are as much about interoperability as they are the Web, it's interesting to note that interoperability isn't something that's a given for Web Services specifications. It's like anything: unless you design with it in mind from the outset, you can quickly end up with a closed system. For example, without naming names, there are several WS specifications out there that define extensibility elements (##other to you and me). Now that isn't necessarily a bad thing (hey, we do the same in WS-Context). It becomes a bad thing when either the specification requires you to use them (and doesn't provide concrete rules for the contents) or implementations require them to be filled with vendor specific information (and again, often don't externally publish this information for other vendors to use). IMO this is an easy route back to vendor lock-in.
We had the same issue back in the OMG OTS days, where the transaction context defines an extensibility element (a CORBA any). The intention was that an implementation could fill this with whatever information it needed to provide optimizations for itself (e.g., build up a subordinate transaction hierarchy in a downstream node when a request is made on a service on that node and then pass it back up to the parent when the response is delivered, rather than call back to the parent as you build up the hierarchy before executing the request), but if that information wasn't present then heterogeneous implementations could still interact, albeit in a possibly less efficient manner. However, it quickly got hijacked and implementations required the CORBA::any to be present and filled with specific information in order for them to work. Quite nice it you want to prevent interoperability and encourage lock-in.
Anyway, I digress. The WS-Context interoperability demo hosted by OASIS went well. We used two local implementations (Arjuna's running on my laptop and IONA's running on their own laptop) and a remote on (Oracle). It'd be nice to say that that's the configuration we always intended, because it shows us talking form DC to Philadelphia, where the Oracle endpoint is located, but I'd be lying. The reason we did this was that poor Simeon (Green) from Oracle had his laptop die on him an hour before the demonstration! Sods law!!
Tomorrow Doug Bunting (Sun) and I are presenting on WS-CAF and how it fills some of the gaps in the current Web Services architecture. Hopefully that will go as well as the interop. demo.