Wednesday, May 20, 2015

Kids and asynchronous messaging

We've heard a lot recently about asynchronous frameworks. Typically what these really do is prevent or discourage blocking (synchronous) calls; such calls may be made, e.g., making an RPC to a remote service, but the sender thread/process either doesn't block (so doesn't actually make the call directly), or gets back some kind of token/promise that it can later present (locally or back to the receiver) in order to get the response when it needs it. Of course there's a lot more to this which I'm deliberately glossing over, but the point I'm trying to make it that most of these frameworks don't use the term "asynchronous" in the way we might understand it elsewhere. The FLP problem is a reality yet not one with which they tend to be concerned.

It's strange how the mind wanders when you're doing other things because this was precisely the problem I started to think about whilst cooking dinner the other day. With a lot of people believing that "asynchronous" is the way to go (and in many cases they're not necessarily wrong) I'm more often than not unsure as to whether they really understand the full implications. And whilst cooking I was getting frustrated with my kids, both of whom I'd tried contacting by SMS and neither of whom had responded.

So my mind wandered and I made the not-so-huge leap: I'm trying to communicate with them asynchronously. I sent each of them an SMS and I had no idea of it had arrived. Of course I could have added the SMS-delivery ack request to the messages but that can be disabled by the receiver. Therefore, until and unless they responded, I had absolutely no way to know if they'd got the message and were ignoring me, or if it had gone missing en route (SMS isn't reliable, after all). I sent more SMS messaged but with no responses I was left in the same quandary from before.

Now of course I could ring them but if they don't answer then all that does is mean I've deposited a message (voice) into some semi-persistent queue that they may or may not look at later. Another option would be to find someone who knows them and talk with them, asking for messages to be forwarded. By increasing the number of people I ask I increase the chances that one of them will eventually get through and I may receive a response (gossip protocols). However, ultimately until and unless they or a proxy (friend) for them responded (or came home) I couldn't really know directly whether or not they'd received my messages and acted upon them. Asynchronous is a bitch!

To conclude the story - they came home, had received the messages and didn't believe a response had been needed. Dinner was very tasty though!

Saturday, May 16, 2015

A little more on transactions and microservices

I had some interesting discussions this week which made me want to write something more on transactions (really XA) and microservices.

Sunday, May 10, 2015

Monoliths and a bit of a history lesson

Just another cross-post for those interested.

Update: one thing I forgot to make explicit but hopefully comes through in the above post and the earlier one on web-scale is that to suggest is that to suggest application servers (Java or something else) aren't sufficient for web-scale is difficult to justify since many high profile sites do run today quite well on them.

Web-scale: I do not think it means what you think it means!

When I blogged about transactions recently one of the comments referenced "web-scale" and how application servers aren't well suited for web-scale distributed applications. I'm going to write something specific about this later but it got me to thinking: what does that actually mean? To paraphrase Inigo Montoya "You keep using that term, I do not think it means what you think it means." Many people and groups use the term but there really isn't a common (agreed) definition. It's a cop out to claim that we don't need such a definition because everyone who is sufficiently "in the know" will "know it when they see it".

The web has been around now for a long time. Over the years it has grown considerably and yet many sites back in the late 1990's would not be considered "web-scale" today. But back then they almost certainly were. Despite what many people may suggest today, sites then and now got by well using "traditional" (or dated?) technologies such as relational databases, CORBA, probably even COBOL! Coping will millions of requests, terabytes of data etc.

It's only fair that our definition of "web-scale" should change over time in much the same way as our definition of things like "personal transport" change from "horse" to "car", or even "jet pack". But we don't even have a definition of "web-scale" that we can all get behind and then evolve. It's subjective and therefore fairly useless as a way of signifying whether anything is or is not suitable for the kinds of applications or services people develop for the web. Ideally we'd fix this before anyone (person, group, company) used the term again, but I doubt that'll happen.

Thursday, May 07, 2015

Slightly frustrating article ...

I came across an article the other day from last year which tried to poke holes in transactions. I kept putting off writing about it but eventually decided I had to say something. So ... here it is.

Sunday, April 26, 2015

Listening versus talking

As the song goes ... "To every thing there is a season, and a time to every purpose under the heaven". In any conversation or meeting there's a time to listen and a time to talk. Unfortunately in many environments there's an implicit peer pressure to be heard and be seen to be heard (!) even if what is then said doesn't advance the conversation. Yet often the right thing to do is listen, think and remain quiet until and unless you've got something to say which does add to the overall thread. But guess what? That can be harder to do than you might think, especially as meetings become very vocal and people verbally challenge each other to be the dominant individual.

It takes a lot of control in these situations to listen and not react, especially when there may be so many other people jostling to be heard and yet not really saying anything additive to the meeting. In fact others in such meetings may take silence as an indication of your disconnection from the conversation, or lack of understanding, which could lead you to want to say something (anything) just to prevent that kind of interpretation. But trust me ... Sometimes silence really is golden!

SOA done right?

I said something the other day in a presentation I was giving that I immediately regretted: I was asked how to position microservices and SOA for marketing, to which I responded "Microservices are SOA done right". A nice sound bite, but the reason I regret it is that someone could infer from it that anything termed a microservice is a good SOA citizen. That's not the case. Just as not everything termed SOA or REST was following good SOA or REST principles, so too will be the case with microservices. And the lack of a formal definition of microservices, whether in terms of recognisable SOA principles or something else, doesn't help matters.

So what should I have said? In hindsight, and remembering that it had to be short and to the point, I'd probably revisit and say something along the lines of "If the service follows good SOA principles then it's probably on the way to being the basis of a microservice". Not quite as snappy a response, but probably as far as I'd feel comfortable going in a single sentence. Of course it then could lead to follow up questions such as "What's the difference between a good SOA citizen/service and a microservice?" but then we're into a technical domain and no longer marketing.

Saturday, April 25, 2015

Microservices and events

I wanted to write more on microservices (SOA) and specifically around reactive, event-orientation. We've heard a lot recently about frameworks such as Vert.x or Node.js which are reactive frameworks for Java, JavaScript and other languages (in the case of Vert.x). Developers are using them for a range of applications but because they present a reactive, event driven approach to building services and applications, it turns out they're also useful for microservices. Therefore, I wanted to give an indication of why they are useful for microservices and when I started to put proverbial pen to paper I found I was repeating a lot of what I'd written about before and specifically when I discussed some of the core reasons behind the future adaptive middleware platform.

Now I'm not suggesting that Vert.x, Node.js or any reactive, event-oriented framework or platform fulfills everything I mentioned in that earlier entry because I don't think they do, at least not today. However, they do represent a good basis from which to build. Of course that doesn't preclude existing architectures and associated software stacks from being used to build microservices. In fact I think a combination of these new approaches and existing mature stacks are the right way to go; with each new generation we learn the mistakes of the past and their successes, building upon the latter and hopefully not reproducing the former.

Unfortunately I'm not going to endorse the reactive manifesto, because I'm not entirely convinced that what's encompassed there really represents much more than good fault tolerance and message-driven architecture. For instance, responsive? I'd like to think that any useful platform should be responsive! I think we should add that to the "bleeding obvious" category. My definition of reactive would include fault tolerant, scalable, reliable, message-oriented (supporting synchronous and asynchronous), self-healing (resiliency is only part of that) and event-driven.

Therefore, when I say that reactive, event-oriented is a good fit for microservices (remember ... SOA) I'm talking about something slightly different than reactive as defined within the manifesto. It's all that and more. We've come a long way since the 1970's and 80's when things were typically synchronous and RPCs were the main paradigm. Even those are event driven if you think about it. But since then synchronous interactions fell out of favour, asynchronous rose in interest and so did event-orientation and associated efforts like CEP. Hopefully given what I wrote in that earlier article it should be fairly obvious what kind of platform characteristics I believe are necessary in the future - microservices is just a good approach that can take advantage of it.

As I said earlier though, I don't believe any implementation today really embodies where we need to be. There's a lot we can leverage from the past decades of enterprise software development but there is more work to be done. For instance, we hear a lot about asynchronous systems where in reality what most people are talking about is synchronous interactions performed by a separate thread to the main application (business logic) thread of control; there are known time bounds associated with work or interactions, even in a distributed environment. If we really want to talk about true asynchronous interactions, and I believe we do, then we are looking at unbounded interactions, where you simply do not know when a message is overdue. That causes a number of problems, including failure detection/suspicion (timeouts are a favourite way of trying to determine when a machine may have failed and now you can't use time as a factor), and distributed consensus, which has been proven to be unsolvable in an asynchronous environment.

Of course these may seem like purely theoretical or academic problems. But think about the kinds of environments we have today and which are growing in applicability: cloud, mobile, internet of things with sensors, gateways etc. These are inherently asynchronous, with unpredictable interaction patterns and message exchanges. Whilst synchronous or multi-threaded asynchronous may be a good stop-gap approach, it will inevitably present scalability and resiliency problems. True asynchronous, with all of the problems it represents, is the ultimate goal. But we're not there yet and it's going to take time as well as recognition by developers that further research and development is a necessity.

Tuesday, April 21, 2015

Theory of Everything

These days I watch more movies on a plane than anywhere else. One of those I saw yesterday on a trip to Boston was The Theory of Everything. Stephen Hawking has been a hero of mine from before I started at university doing my physics degree. I read a number of his papers and about him before I read his seminal book A Brief History Of Time. Several times.

Despite moving more into computing over the years I've continued to track physics, with books and papers by the likes of Penrose, Hawking and others, but obviously time gets in the way, which if you've read some of the same could be said to be quite ironic! Anyway, when I heard they were doing a film of Hawking's life I was a little nervous and decided it probably wasn't going to be something I'd watch. However, stuck on a 7 hour flight with not much else to do (laptop couldn't function in cattle-class due to person in front reclining) I decided to give it a go.

It's brilliant! Worth watching several times. The acting, the dialogue, the story, all come together. There's a sufficient mid of the human and physics to make to appeal to a wide audience. A singular film! (Sorry, a really bad pun!)

I also managed to watch The Imitation Game, which I think is probably just as good but for other reasons.