Time to reflect

I don't use my blog as much as I used to due to lack of time and something to say that I don't say through other avenues. But something happened today that made me stop and think that perhaps I could use this blog for a more personal posting than usual.

I've always thought that life is precious and yet we often take it for granted, typically until the last moment. We make a big show of people being born because whatever your faith or beliefs, seeing a new life born into the world is a wonderful thing! Death is often more dour and a more personal thing. Typically unless someone we knew died, we all only hear about the deaths of celebrities, many of whom probably had little or no impact on our own lives.

Death is a sad enough occasion at the best of times. Again depending upon your faith or belief system it probably is the last time that unique individual will set foot on this planet and mingle with people here. Some aspects of what made them human, such as the raw materials, will eventually find their way back into the environment and, just as we're all made of "star stuff", back into other people in one way or another. But their uniqueness, their individuality, is gone forever - as best we can tell. That is sad. At it's rawest, this is a loss of information that can never be retrieved. A loss of memories, experiences etc. that helped to make the person who they were.

We often hear statements like "they're not dead as long as we remember them". Thinking about the sentiment behind these kinds of statements it makes sense. And we can all probably know someone who died, family or celebrity, that we remember fondly. But what of those people who have no one? That's the biggest loss of all: there's no one to remember them, to remember what made them unique within the 7 billion people on the planet. Maybe they weren't celebrities. Maybe they weren't world leaders or people who went down in the history books. But they were people nonetheless and to not be remembered is like them falling into a black hole, where no trace remains.

If you've gotten this far then you may be wondering why I'm writing this. I live with my family in an area of the country that means we have only 2 neighbouring houses. Both houses have people in them who have lived there for over 7 decades (we've been here for 14 years). Today one of those people, John Hudspith, died. He was 80 and a kind, quietly spoken gentleman; a man of his era. But he was alone. No family left alive. Few friends, other than ourselves and the other neighbours; none really close. Even then he was a private person. And it struck me that in his death he would be forgotten because he lacked celebrity status or family or history-worthiness. Well this is my small attempt to give him a little immortality, because if you've read here then you've paused for a moment to wonder about this John Hudspith, who he was and why I would want to remember him. Thank you.

Encrypting Data?

I read that the FBI doesn't want Google or Apple to encrypt data on phones by default. Their reasoning is that it makes it harder for them to track evil doers. I do understand their concerns but I don't believe in their solution: no encryption, or give them keys to decrypt. It's not that I distrust the police or security forces or believe criminals should be able to get away with their crimes, but if my data can be decrypted by one group then there's a good chance it can be decrypted by others (backdoors can and will be exploited). I don't encrypt my data to hide it from the law; I encrypt it to stop it getting into the hands of criminals and people who could use it against me or others!  And if we're not allowed to encrypt phones then what's next? Laptops? Cloud?

Unencrypted data may make their job easier, but surely they do detective work too? Just imagine if the FBIs approach had been enabled decades or even centuries ago. Letters couldn't go into envelopes or envelopes could be opened at any time (probably happened/happens today anyway); it would be illegal to write in anything other than plain English or natural languages (no codes); presumable all of your data would be easily accessible (no bank vaults, or their codes would have to be available to the police without a warrant!) The latest Sherlock Holmes stories would be very mundane as he'd just need to access the criminals' documents to discover their evil plans.

The reality is that encryption of data, hiding of that data, has always happened. Whether it's Sherlock Holmes story The Adventure of the Dancing Men, the Germans during WW2, the Romans, there are countless examples of coded information being used for one reason or another. And good detective work, aided by people in the field, has always been at the heart of the solutions. I don't want criminals to have access to my data and if that means the police need to do a bit more work then so be it.

Microservices and transactions - turning back the clock?!

A cross-post of an entry I wrote for the JBossTS/Narayana blog. Microservices and transactions, oh my!

Microservices Architecture

I wrote a piece on InfoQ a while back on Microservices and SOA. While researching for the article and afterwards, I was struck by something else I wrote almost 8 years ago around SOA 2.0. I've got to say that I see a lot of similarities: people trying to come up with new terms for something that already exists and which really doesn't need to be redefined just better understood! Please, no Microservices. Let's stick with SOA!

The future adaptive middleware platform

Back in the 1990's, way before the success of Java let alone the advent of Cloud, there was a lot of research in the area of configurable and adaptable distributed systems. There was even a conference, the IEEE Conference on Configurable Distributed Systems. Some of the earliest work on distributed agents, adaptable mobile systems and weak consistency happened here and I'm glad to have been a part of that effort. However, two decades ago the kinds of environments that we envisioned were really just the thing of blue-sky research. Times change though and I believe that a lot of what we did back then, and research that has happened in the intervening years, is now much more applicable and in fact necessary for the kind of middleware platform that we need today.

For a start one of the aims of some of the research we did was environments that could self manage, monitor themselves and adapt to change, whether due to failures in network topology and machines, or changes in application (user) requirements. These were systems that we believed would need to be deployed for long periods of time with little or no human intervention. Precisely the kinds of environments that we are considering today for Cloud and IoT: reducing the number of system administrators needed to manage your applications is a key aim for the Cloud, and imagine if you had to keep logging in to your smart sensors every time the wifi went down or a new device was added to the network.

There's a lot that can change within a distributed system and much of it inadvertent or unknowable a priori. This includes failures of the network (partitions, overloading) and the nodes (crash failures or overloading making the machine so slow that it appears to have crashed). Machines being added to the environment may mean that it's more efficient to migrate entire applications or components (services) to these new arrivals to maintain desired SLAs, such as performance or reliability. Likewise a machine may become so overloaded that it's simply impossible to maintain an SLA and so migration off it elsewhere may be necessary. Traditional monitoring and management approaches would work here but tend to be far too manual. This tends to mean that whilst problems (faults) can be tolerated, the negative impact on clients and applications, such as downtime, can be too much.

The middleware should be able to detect these kinds of problems or inabilities to match SLAs, or even predict that these SLA conflicts are going to occur (Bayesian Inference Networks are good for this). It may seem like a relatively simple (or subtle) addition to monitoring and management (or governance) but it's crucial. Adding this capability doesn't just make the middleware infrastructure from being a little more useful and capable, it elevates it to an entirely different level. The infrastructure needs to have SLAs and QoS built in from the start for components as well as higher level services. JON-like in monitoring and managing the surroundings as well as itself.

Each component needs to potentially be driven through a smart proxy so that things can be dynamically switched from local to remote implementations. Maybe environment specific component implementations if existing ones cannot fit or be tuned to fit, e.g., a component written in C for embedded environments where the JVM cannot run due to space limitations. It also needs to add in something like stub-scion pairs (Shapiro in the 80's or Caughey with Shadows in the 90's) to allow for object migration with dependency tracking and migration. Also add in disconnected operation work from the 80's and 90's: yes, the network has improved a lot over the years but disconnection is more likely now because we are used to being connected so much.

We need new frameworks and models for building applications, though current approaches should work. Being transparent is best, but opaque allows for using existing applications. Each component needs something that implements a reconfigure/adapt interface. Listens on a bus for these events. Adapt based on available memory, processor changes such as speed or number, network characteristics, disconnection, load on processor, dependency on other components, etc. Include the dispatcher architecture to help adaptation at various levels throughout the invocation stack.

OK so let's summarise the features/capabilities and add a few things that are implicit:

- Can adapt to changes in environment. Autonomous monitoring and management.
- All components can have contracts and SLA.
- Event oriented backbone/backplane/bus.
- Asynchronous interactions. Synchronous on top if necessary.
- Flexible threading model.
- Core low overhead and footprint. Assumed to be everywhere and all applications or services plug into it. So much of these other capabilities would be dynamically added when/if needed, such that the core needs to know how to do very little initially.
- Repository of components with social-like tagging for contract selection.
- Native language components ranging from thin client through to entire stack.
- DNA/RNA-like configuration to allow for recovery in the event of catastrophic failure or to support complete or partial migration to new (blank/empty) hardware.
- Self healing.
- Tracking of dependencies between objects or services so that if the migration of an object/service is necessary (e.g., due to a node failure or being powered down), all related services/objects can be migrated as well to ensure continued operation.

So this is it in a (relative) nutshell. The links I've included throughout should help to give more details. If I get the time I may expand on some of these topics in future entries. Even better, maybe I'll also get a chance to present on one or more of these items at a conference or workshop. One of the problems I have with my job is that there's so much to do and so little time. Whilst all of this is wrapped up inside my head, the time to write it down doesn't come in a block large enough to do it justice in one sitting. Rather than spend months writing it up, I wanted to take a good open source approach and "release early, release often". Consider this a milestone or beta release.

Rush and Senna

This is going to seem like a strange entry but I wanted to record it if only for myself. When I was growing up I was a serious Formula One fan and two events remain in my mind as defining moments of enjoying the sport. The first was James Hunt winning the F1 Championship (and Niki Lauda’s accident), and the second was the death of Ayrton Senna. I wasn’t watching the former live, but saw it on repeats (TV was much more limited in those days, so live feeds were few and far between). The latter I was watching with friends as it happened and can still remember where I was to this day - in the dorms at university, having just eaten lunch! There was excitement, risk and sadness in equal measure for each event and yet over the years they dwindled in my mind as other things, such as real life, took hold and dragged me through the present and into the future.

That was until I watched the movies Senna and Rush over the course of the last couple of years. Both of these movies are very different beasts and yet both of them excel in brining to the watcher that sense of what is was like being there at the time. As someone who was there at the time and someone who watches F1 today when I have time, they also illustrate what I think the sport misses these days: excitement and challenges (ok, the risk is a big element here). If you’re at all interested in Formula One then check out either or both of these films!

Speaking at Dynamo14

I'm going to be speaking at the Dynamo14 conference in a few weeks time. I'll be hot off a plane from Boston to be interviewed by the BBCs Rory Cellan-Jones! Looking forward to it, though I hope I'm not too jet lagged!

Another "I told you so" moment

I mentioned earlier about how most cloud providers had agreed that there is such a thing as Private Cloud. It's nice to hear that it's a "tectonic shift" now ;-)

Hybrid Cloud

I've read a few articles from analysts or media recently where they talk about 2014 as the year of Hybrid Cloud. I think this is good to hear, but I also wonder why it's taken so long for some in the industry to accept the inevitable? OK that's more of a rhetorical question, since I do know the answer and I'm sure many of you will too.

In reality I find it amusing, with more of an "I told you so" approach! Several people, as well as myself, have been talking about the need for private cloud. Whether it's issues with security, reliability or simply getting data into and from public clouds, public cloud has limitations and is definitely not right for everyone or every application/service. That's not to say that private clouds are problem free but it's not necessarily the same problems and neither should it be.

The combination of private and public clouds, coupled with cloud bursting when needed, is what we should all be aiming for, with pure private or pure public use simply a degenerate case. I find it interesting that Amazon now has a position on private and hybrid clouds, which is very different from what they were saying 4 years ago.

I also still believe, as I stated back then, that the whole definition of cloud needs to change. Whether it's ubiquitous computing or the Internet of Things, the explosion of devices around us and their capabilities (processor speed, memory, network access, etc.) has to become part of the cloud, and probably its biggest part. It makes no sense to ignore the powerful options that this gives us, let alone the fact we'll have no choice due to Shannon's Limit.

So although 2014 may be the year of hybrid cloud, it's the true cloud of devices that I'm still more excited about. Most of the options it creates still lie unexplored before us and I think that's where we should be adding a little more focus.