Christoph Jahn

For some time the topics of SOA Governance and BPM have been looked at as if they were two relatively unrelated things. And this perception is correct in the sense that you don’t have to have them together. However, more and more people realize what huge additional benefits are in for them if they combine the two things. In many cases the idea is that you need some logic to govern the actual work (design, development, testing etc.) for a process that has been modeled in a nice fancy tool.

But you can also do it the other way around: Think about what you would get if you could govern your whole IT lifecycle management from one tool. The idea goes like this: You store all relevant information about “objects” that are relevant for your organization in a central repository, and the different attributes that describes those objects (aka assets) are completely freely configurable. You probably need to attach additional information to them, like existing documentation etc. So in a way you can think of these information in the repository as a way to store the knowledge about all relevant aspects of the organization and then leverage this knowledge.

Now based on that groundwork, whenever a request for a new a feature in the IT landscape comes in, you can have it go through a “workflow”. The first steps would probably be about an approval chain. So people from various functions (e.g. product management, operations, security, marketing etc.) would need to either approve or reject this. How the final outcome is determined can be a bit tricky (and is much more a political topic than a technical one).

Then come steps like gathering requirements, signing them off, doing the development etc. You probably also want to integrate this whole thing with your development chain (automated testing, continuous integration etc.). At any given point in time you know where your development stands in terms of the project plan.

So if you step back a bit and look at what you get, we are not talking about development tools any more. Instead this is true, real-time end-to-end visibility. There are clear responsibilities for assigning tasks (a human being has to decide on something) and you no longer need to fear emails that are lost in the inbox of someone’s email program. Instead you get a view into the currently open tasks, their due dates etc. Other advantages existing but for now those are the critical ones. The reason for this is that these functionalities allow you to have automatically generated documentation that satisfies your compliance requirements. In most organizations these things eat up enormous amounts of resources and affect processes that should deliver value to the organization.

Let’s leave it here for now. I am quite interested in your comments on this, so please let me know.

When David A. Fisher wrote this paper in 2006, the hype around Web Services and SOA had just begun. The point that struck me most when reading the executive summary, was that Fisher does not limit his thoughts to technical systems. Instead he accepts the fact that the involved people are also an important part of the equation. This aspect seems to be ignored by most authors and is -in my view- a major reason why many theories seem to be so far from reality.

The paper is more than 60 pages long, so nothing for a quick lunch break reading. However, I recommend reading at least the executive summary. It made me curious enough to schedule special time for the rest of the document.

I have visited the beautiful village of Boscastle a couple of times. It is located in Cornwall in the South-West of England. You can do a lot of sightseeing in the area and the scenery is quite stunning.

The village is located near Tintagel castle that is attributed to King Arthur. Also, a few of the openings scenes of the film “Saving Grace” were shot near Boscastle.

Boscastle became known to the world when a MAJOR flooding occurred in August 2004 (BBC page with pictures). If you have seen the pictures it seems a miracle that nobody died in the event.

[Show as slideshow]

All too often I have heard things like “You need fast storage? Use SAN”. While this is certainly correct, most people have a relatively vague understanding about SANs in general and what they offer performance-wise compared to local disks in particular. I will not go into the details of things like the various protocols etc. Rather I want to highlight where this requirement for high performance comes from and why SANs are in many cases a good approach to it.

I am looking at all this from the point of view of commercial applications only. What they are doing -in a nutshell- is retrieving, processing and sending data. By the way, if you never touched upon the semantics of terms like data, information, knowledge and how they are related to one another, I suggest to do so. It is an interesting topic and knowing the differences can be helpful in many discussions.

But back to things …. For most commercial applications the overall performance is determined not so much by CPU power but the throughput with which information can be received or sent (or read and written from/to disk when speaking more of persistance). In technical terms this is commonly referred to as I/O throughput. And that is, by the way, a major reason why mainframes will most likely stay around for quite a few more years. They have been optimized for I/O performance for literally decades. This includes hardware (special I/O subsystems that free the CPUs from processing I/O requests), the operating system and also the applications (because people have been aware of this much more than in the open systems’ world).

But I did not want to focus on mainframes today. Rather this post is about a rather common misconception when it comes to I/O or more specifically storage performance. Many people, whenever they hear “We need high performance storage” instantly think about putting the respective data onto a SAN. While this may be the right choice in many cases, the reasoning is not so often correct. For many folks SAN is just a synonym for “some centralized ultra-fast storage pool”. It is in my view important to understand how a SAN is related to other storage devices.

There is no reason that local hard disks are “by definition” slower than their counterpart in the SAN. At the end of the day the main difference between a SAN and local hard disks is that the connection from the computer to the disks is different. For SAN this is either fiber channel (FC) or iSCSI (basically SCSI over IP). For local disks it can be SCSI (serial or good old parallel), Serial ATA or whatever. What determines the speed is the number and individual performance of the physical disks that make up a RAID. The more disks you have and the faster they each are, the faster your logical drive will be. This is generally indepedent from whether they are connected to a local RAID controller or “simply” sit in the SAN somewhere in your data center.

So as soon as you compare the SAN to a local RAID system (and I am not talking about the RAID-capabilities many motherboards offer these days but dedicated RAID controllers) the perceived performance gap goes pretty much away. You may even have an advantage with local disks because you have direct control over how many drives your file system spreads while with a SAN this may be a bit more difficult to achieve.

I will leave it here for now. Hopefully this somewhat demystifies things for you.

You might now think: Has he finally lost his marbles? What’s going on here? Well, I have decided to regularly mention famous citations here and give a bit of an explanation about their background. And this first posting should of course be a bit of an eye-catcher. So here you go:

This is a Latin citation that is attributed to the Roman emperor Vespasian (9-79). Part of his efforts to consolidate the public budget was the introduction of a special tax for “restrooms”, or more specifically the newly built Cloaca Maxima (sic!). When challenged by his son Titus about it he responded with the above statement. Here are some examples how this saying still lives in today’s world

• It is widely used to justify money coming from questionable sources.
• In Paris public restrooms are called “Vespasiennes”.
• And in Italy they are “Vespasiani”.

In the context of SOA (Service-Oriented Architecture) there has been a revival of the asynchronous communication pattern. And that is really what it is: a pattern. First and foremost we are not talking about a specific product, protocol or API but simply a way how to design systems and applications. After this has (hopefully) been clarified between us, let’s look at some of the in my opinion important aspects. I start off with a (certainly non-academic) definition:

Asynchronous communication simply means that when making a call into an “external system”, my program/component/service etc. does not expect an immediate response with the actual result (this would be synchronous communication). I am only interested in getting a positive acknowledgement that my request has been received (but not processed) correctly. I don’t care about any further logic that is going to be executed in some other program. Instead things continue on my side and at some other place the result of my request may be received and processed further. (There are quite a few use-cases when I don’t expect a result at all.)

At a first glance this may seem way more complicated than simply making a call, wait for the result and then continue. And to a degree this perception is correct, although I must say that in my view people exaggerate greatly here. In all the cases I have seen so far the really bad problems were not caused by the pattern itself. Rather it was a cumbersome integration on the tool-level. If you have to bother with e.g. complicated mappings before you can send data over to your message broker from the “regular” code, this will certainly inhibit the use of the pattern (and rightly so). But you should blame your tool set and not the pattern in this case!

What is indeed a bit challenging when you first start using this pattern, is that it requires a fundamental shift of your mindset for designing your system or application. What you need is a loose coupling of your components. Technically speaking this means that there must be another component that is ready to receive a response matching your original request and then continues working on it.

(If you followed the discussion around SOA lately, you may have come across that term “loose coupling” more than once already. So you can think of asynchronous communication as a means for reaching this design goal. Bear in mind though, that we only look at the communication layer here! Loose coupling should also be concerned about the semantics, so in technical terms the data model needs to support this as well. However, I wanted to discuss asynchronous communication here, so let’s leave it with loose coupling for now.)

With one component sending out a request and another one handling the response, we have a solid foundation for a highly distributed system . Yes, you can also design a distributed system with synchronous communication but this is probably more difficult. What comes to mind when talking about distributed systems is scalability. More specifically, we are talking about horizontal scalability (scale out), which is spreading the load over more machines instead of putting more resources into a single machine. Having many relatively small systems work on things typically has two main advantages compared to going for bigger machines: Firstly the approach scales further and secondly it is cheaper because you can use standard hardware.

Another big plus of asynchronous communication is robustness, provided you are using a message broker that offers guaranteed delivery. Once your message broker has confirmed the receipt of the request you can be sure that it will be delivered to the receiver. (You can use the publish-subscribe pattern to allow the sender not having to know about receivers. More on that in a separate post later.) And it is certainly easier to only have to make the message broker highly available than to do the same for each and every piece of code. So by having the message broker as a central piece of infrastructure you can reach a high level of high availability relatively easily.

That’s it for now, there will be more posts on related topics.

The guys from Manager Tools have set up a special page with free access to some of their premium content. It covers how to deal with the consequences of recent events in the (financial) market. If you are not already a premium subscriber you should probably check this out.

Many questions in life are not so easy to answer. However, when it comes to what my favourite game was on the C-64, it is different. There is only one answer possible and those who know me long enough know it:

The Great Giana Sisters

You can probably find the game these days on the various archives for emulators (there was also a version for the Commodore Amiga by the way). And some really crazy guys developed a Flash version. The controls are a bit sluggish, but the idea is really cool nevertheless!!!

When you look for a great selection of free material around management you can hardly avoid this website. It contains a lot of material from Marshall Goldsmith and you should also check out the blog. The latter contains a lot of posts one could call philosophical and it is one of the best blogs I have come across so far.

When you work in the software industry you are probably used to seeing things change rapidly. One of the disadvantages that come with this is a “heterogeneous” use of terms. It is often due to the fact that we do not want to bother ourselves with precise definitions. A very sloppy use of language is the natural consequence.

As some people have noticed in the past I tend to speak rather neatly. I can attribute this to my parents who were very much concerned with a correct usage of language (my father used to work as a German teacher after all). So I may be a bit more aware of the implications of an imprecise use of language. Therefore, today I want to look at some terms that are often used either without much differentiation or without a precise notion of the relationships between them. They are

• Monitoring,
• Business Activity Monitoring (BAM) and
• Reporting

Monitoring is mostly used to describe an operational activity that looks after currently ongoing things at an individual level. It typically deals with questions like “What activities in my organization are currently causing problems?”. In a Business Process Management (BPM) context this means checking that all process instances are running fine. An operator will query the BPM system and ask for process instances that failed entirely, exceeded timeout limits for single process steps etc. Also automated activities fall into this category. So if you have set up an alert that alarms a human being or another system if some threshold has been exceeded, I would also consider this as monitoring.

However, we are gradually moving into a grey area here. Depending on the type of the rule that is associated with this alert, the focus moves more towards the Business Activity Monitoring (BAM) area. So imagine a threshold that is not concerned about a single incident any more. Rather it can be of a “composite” nature and look at an average for a certain time frame (e.g. average order amount per day of the week). Such figures are usually called Key Performance Indicators (KPIs). Managers tend to be much more interested in KPIs than in the characteristics of a single, although very important, activity in the organization. So instead of asking “How are we doing with order 5297523?” they would like to know “How is our on-time order fulfillment rate today?” You can think of this as monitoring for operations management. If you want to know how you are doing NOW from an overall perspective, BAM is what you probably need.

Take away the “now” attribute from this and you pretty much end up with Reporting. This answers question like “How was our on-time order fulfillment rate this month?”. In many cases this will be closely linked to Business Intelligence (BI) and Data Warehousing (DWH) topics. Some of the BI/DWH providers have also realized that their customers do not only want to know how they did in the last month but how things are going at the moment. The simple reason is that looking into the past is good to identify structural issues that can then be tackled. However, it does not help to identify a current or upcoming problem and avoid its worst consequences by taking instant counter measures.

Being able to react in almost real-time to what’s going on, is the real value of BAM. The later you realize that something is going wrong the fewer options you have and the more likely it becomes that you can only try to repair damage already done. Although implicitly mentioned in the above paragraph, I want to highlight that BAM is not only concerned with existing but also with upcoming problems. You should expect a BAM solution to be able to identify trends and alert you based on that.

So there are distinct, yet related use-cases for all these functions and you definitely want to combine them to get most from them. I can well imagine that over time these things will, at least to a degree, merge into broader solutions. One last word here: The statements given here can not reflect all possible variations and in your particular case things may seem to be different. Don’t be disturbed by that feeling. Rather analyze the situation and you will probably discover that the general lines are still the same, although the details may vary.