Tuesday, November 30, 2010

More sad news!

I met Maurice Wilkes a few times when he came to the University to talk. So it's sad to hear that he has passed away. Definitely a pioneer and definitely a very nice man. A sad day indeed.

Busy Bees

Once again, congratulations to Sacha and team!

Monday, November 29, 2010

Android woes coming to an end!

My time as an Android user is drawing to a close. I may come back when they sort things out so that I am no longer at the mercy of the vague policies from then handset manufacturers or operators for upgrades, when in reality they really want me to buy a new phone and contact every 6 months in order to get even security upgrades.

It was fun whilst it lasted and even with the benefit of hindsight I think it was the right choice at the time. However, it has left a bad taste and Google needs to sort out the mess. Until then I'll probably move to an iPhone. And I can't really think of a better way of putting it than Douglas Adams: So Long, And Thanks for All the Fish!

Sunday, November 21, 2010

Management and PaaS

Many years ago I spent a few years working for HP as a Distinguished Engineer via their acquisition of Bluestone. Now I learnt a lot through that experience and one of those things was the importance of system management software such as OpenView. In truth I already knew, having done research with HP, IBM and others over the years prior to this; but spending quality time with the team was still informative and illuminating.

Managing middleware systems is a complex task, particularly if you want to do it efficiently and without adversely impacting the system that is being monitored/managed. (Almost like Schrodinger's Cat experiment!) But for large scale, dynamic systems such as those I mentioned previously, good monitoring and management is critical. This remains true if you remove the dynamic/autonomous aspects and assume that a human system administrator will be using the information provided.

This shouldn't come as a surprise. Even your favourite operating system has enjoyed monitoring and management capabilities of various quality over the decades. The good implementations tend to remain in the background doing their job and you don't even know they are there. And of course as soon as you add in the distributed factor, the need for good monitoring and management cannot be over stated.

But of course you can do without; it's just that you may then have to put up with sub-par performance or efficiency as you need to manually cope with failures, changes in network or machine characteristics, etc. Worse still, issues may go unnoticed until far too late, e.g., the lack of orphan detection can lead to multiple competing services corrupting data or conflicting with one another.

So why am I mentioning this? Is it because it's not obvious, or that I'm running out of interesting things to discuss? Not quite! It should be clear that management/monitoring software is important for middleware solutions (and others). Depending upon what you consider to be monitoring and management, it may be a core component of middleware, or a component in middleware (e.g., the previous example on orphan detection and elimination.) But if you're looking at a separate monitoring/management component or infrastructure, then you wouldn't consider that to be your development platform for arbitrary applications, right? You wouldn't, for instance, consider such a software system to be a replacement for your middleware platform, e.g., for CORBA or Java EE.

Over the years I've never heard anyone suggest such a thing, so this isn't something I've even considered people would think. However, recently this has come up in several conversations I've had. And yes, it's in the area of Cloud and specifically PaaS. The statements have been made that PaaS is the monitoring/management, or is all about the monitoring/management. Why would that be any more sensible a statement or position to take in the Cloud world if it wasn't sensible prior to Cloud? I really don't know. It doesn't make sense any more than ...

"Why would a Wookiee, an eight-foot tall Wookiee, want to live on Endor, with a bunch of two-foot tall Ewoks? That does not make sense! But more important, you have to ask yourself: What does this have to do with this case? Nothing. Ladies and gentlemen, it has nothing to do with this case! It does not make sense! Look at me. I'm a lawyer defending a major record company, and I'm talkin' about Chewbacca! Does that make sense? Ladies and gentlemen, I am not making any sense! None of this makes sense! And so you have to remember, when you're in that jury room deliberatin' and conjugatin' the Emancipation Proclamation, does it make sense? No! Ladies and gentlemen of this supposed jury, it does not make sense! If Chewbacca lives on Endor, you must acquit! The defense rests."

With PaaS we're talking about a Platform (the first letter in the acronym kind of gives it away). Now unless you're only developing applications, services or components for your favourite monitoring/management infrastructure, you'll probably consider your platform to be something else entirely, e.g., Java Enterprise Edition with all of its capabilities such as transactions, EJB3, POJOs, messaging, persistence etc. This (as an example) would be your definition of your platform. Very few people would consider OpenView (or whatever they're calling it these days) as their platform. So although something like OpenView, or even JON, are important to any platform, I certainly wouldn't consider them to be the platform.

Or maybe I'm missing something?!

Friday, November 19, 2010

REST + transactions paper

We've been working on REST+transactions for a long time now, with a number of iterations and implementations spanning over a decade. After various discussions over the past few months I'm hoping that a few new organizations (academic and industrial) will be coming forward in the near future to help contribute to this work. However, I think it's time we wrote a paper on this and got it published. We've got enough material and experience to target a range of conferences and workshops. Plus I haven't written a paper in a while: I'm starting to feel rusty!

Iconic distributed systems research comes back around

Distributed systems research has been going on since the very first time someone decided to network multiple computers. Industry and academia have shared this burden and we're here today because of many different people and organisations. Some of this work is often referenced and built on, such as Lamport's paper on Time, Clocks and the Ordering of Events in a Distributed System, or RFC 707, concerning RPCs. But some of it, such as Stuart's work on Coloured Actions, or much work on weak consistency replication.

However, sometimes it's simply a matter of timing, with some research happening before it's really needed or truly appreciated. Case in point is a lot of the work that we saw produced during the mid 1990's on configurable distributed system and particularly that presented and documented by the IEEE Conference on Configurable Distributed Systems (there were other workshops and institutions doing similar work, but this conference was one that I had personal knowledge about since I had several papers published there over the years). Much of this work concerned autonomous systems that reacted to change, e.g., the failure of machines or networks, or increased work load on a given machine that prevented it from meeting performance metrics. Some of these systems could then dynamically adapt to the changes and reconfigure themselves, e.g., by spinning up new instances of services elsewhere to move the load, or route messages to alternative machines or via alternate routes to the original destination thus bypassing network partitions.

This is a gross simplification of the many and varied techniques that were discussed and developed almost two decades ago to provide systems that required very little manual intervention, i.e., they were almost entirely autonomous (in theory, if not always in practice). With the growing popularity of all things Cloud related, these techniques and ideas are extremely important. If Cloud (whether public or private) is to be differentiated from, say, virtualizing infrastructure in IT departments, then autonomous monitoring, management and reconfiguration is critical to ensure that developers can have on-demand access to the compute resources they need and that the system can ensure those resources are performing according to requirements. This needs to happen dynamically and be driven by the system itself in most cases because there should be little/no involvement by your friendly neighbourhood system administrator (in fact in some cases such an individual may not exist!)

I'm hoping that just because Cloud didn't exist as an identifiable concept back in the 1990's, people and organizations today don't overlook the fact that relevant R&D happened back then. Reworking and retasking some of this prior work could help save us a lot of time and effort, even if it's just to convince engineers today that certain paths or possible solutions aren't viable. For a start I know that I'll be getting my copies of those proceedings out again to refresh my memory!

Tuesday, November 09, 2010

Nice move by Sacha and Cloudbees

From Devoxx ...

"CloudBees, the new company from former JBoss CTO Sacha Labourey, today consolidated its position as the de facto source for Hudson-based continuous integration solutions, from on-premise to the cloud with the acquisition of InfraDNA, the company founded by Hudson creator Kohsuke Kawaguchi to provide software, support and services for Hudson. Kawaguchi joins CloudBees, and will continue to grow and lead the Hudson project. The first release today from the integrated company is Nectar 1.0, formerly InfraDNA’s Certified Hudson for Continuous Integration (ICHCI) offering, which has been enhanced with new features and rebranded for CloudBees. Enhancements include VMware Virtual Machine auto configuration and deployment; enhanced backup services; pre-bundled and configured plug-ins; and an auto-update service. Nectar annual subscriptions start at $3,000 and include free minutes on CloudBees DEV@cloud service, which is currently in beta."


Saturday, November 06, 2010

PaaS is language specific?

Recently I've been involved in some discussions with people on the subject of PaaS and specifically how the application language impacts it. I was surprised to hear (or read, since some of this was via email) several of the participants assume that you would need a PaaS implementation for each programming language. So that'd be a Java PaaS, a Ruby PaaS, presumably a C++ PaaS and maybe even a COBOL PaaS.

In my most diplomatic manner, I asked if certain individuals had missed the last 20+ years of middleware development; perhaps they had been stuck on some island somewhere, or maybe off on some rocket traveling close to the speed of light and while we experienced 20 years, for them only a few days have passed. See, I do like to give people the benefit of the doubt where possible!

The idea that we need to have a PaaS implementation for all possible programming languages out there (or at least those that may be used in the Cloud), is ridiculous. Think of it like this: if we can virtualize away the hardware, so it's really no longer a major issue in deployment, then we should be able to virtualize (abstract away) the infrastructure upon which the applications will rely. And yes, that infrastructure is middleware.

Oh wait ... we've done this before. Several times in fact! I won't cover them all, but let's just consider a couple of the obvious examples. First, CORBA, where services could be implemented in a range of different programming languages (e.g., C++, Java and even COBOL!) and made available to users that may be written in a completely different language. Second, Java, and specifically J(2)EE. Like it or not, despite the write once run anywhere message that came with Java, many vendors and users simply did not want to, or could not, discard their existing investments in other languages and so found ways to integrate services and components written outside of Java and often without the knowledge of the end users. For instance, there are non-Java transaction services that are integrated into application servers through the JTA. The same goes for JMS.

As an industry we have decades of experience in virtualizing the middleware layer. It doesn't matter if you're writing your applications and services in Java, Ruby or something else entirely: as long as someone provides a binding to the underlying middleware capabilities, you don't expect, and shouldn't be interested in, whether or not the middleware, or PaaS, is implemented in the same language. And in fact it shouldn't be. I'll keep saying this, but we simply cannot afford to stop the world and reinvent the middleware wheel for PaaS. We need to evolve, but we really don't need to start from scratch. I think I got my message across during the aforementioned conversations, but maybe next time I'll resort to drawing some diagrams, perhaps even with some timelines dating back to the 1960's!

Addendum: In case it wasn't clear in the above, of course there will need to be a language specific component to PaaS, but only that which is sufficient in order to allow the underlying implementations (written in whichever language is appropriate) to be interfaced with the application or service. Consider the JTA and JMS examples above.

Tuesday, November 02, 2010

JCP EC (re) election

The results are in...the 2010 JCP EC Elections have officially concluded, and final results are available on jcp.org. The 2010 JCP EC Election ballot closed at midnight pacific time on 1 November. Congratulations to the new and re-elected JCP EC Members!


Ratified seats: Apache Software Foundation, Red Hat
Open Seat Election: Eclipse, Google

Ratified seats: Research in Motion (RIM), Samsung, TOTVS
Open Seat Election: Apix, Stefano Andreani

New and re-elected members will take their seats on Tuesday, 16 November.

JCP members did not ratify Hologic for the SE/EE EC. As the JCP process document prescribes, the PMO will hold an additional ratification ballot for the remaining SE/EE seat soon.

Complete results will be available at the JCP Election page and the JCP blog later today.

Personal versus company resources

A long time ago in a galaxy far far away ... OK, so maybe not necessarily that long ago and certainly not that far away, when work started to intrude on my personal love for computers, I made sure I had a separate machine for personal use (a desktop in those days) and one (or more) for work. That was fine for a while, but eventually it became too much of a hassle to synchronise email, source code etc. Plus, in those days there was very little distinction between my personal (for pleasure) efforts and my work. So multiple machines gave way to one machine. I also tried this with mobile phones (with one for personal use and one for work), but with much the same conclusion.

This situation existed for the best part of a decade and a half. However, in the past year I've been finding that work is intruding more and more on my leisure efforts around computing. Whether those are writing research papers or just playing around with new languages, I find it is harder to stop work (via email, IRC etc.) from distracting me. And the phone just makes it worse!

So recently I've reverted to the way things I used to do things, with a dedicated work machine that knows only about work related "stuff" (including email) and a dedicated personal machine, that knows nothing about work (and its emails). The phone problem is about to be fixed with a similar strategy. I wonder how long I'll be able to keep up this separation of concerns. But I'm looking forward to trying!