Service discovery, part 3: Meet in the Middle

Finally I found some time to finish this series. My apologies to those who've been waiting for part 3, but ..... here it is.

I have outlined two different approaches so far: bottom-up and top-down. As is usually the case, both have merits and drawbacks. In fact, the real solution is to do both, and not just one. That's what I call 'Meet in the Middle'. The top-down approach is very important to achieve the business goals, where the bottom-up approach is just as important to utilize the investments made in the current systems for the last few years. By combining these two approaches you get the best of both worlds: integration with the current application infrastructure (in a service-oriented way) and preparedness for more process driven functionality which supports the business needs.

But (of course there's a but) there's a little more to it. Matching the two approaches is not as easy as it sounds. There will be quite a gap between the business services as described in the top-down approach and the application services discovered in the application infrastructure. The trick is now, to take a business service and analyse which of the application services youi need to compose a service that will bridge the gap. This composite service can be multi-layered, and usually is.

One of the paradigms we use in grouping and discovering services is domain driven design (see Eric Evans marvellous work). Stated shortly: we define business domains (top-down here) consisting of delimited data and corresponding services (multi-level services: from data manipulation to business level services). This domain (and therefore its data) can only be accessed through the services. These are the building blocks for the future business functionality.

The attentive reader will have seen the problems looming on the horizon. The main issue is, that the domains we have recognized will be spread over the existing application infrastructure. So, how are you going to delimit the domains? That's the hardest part. In a greenfield it's easy, because you can define and build the domains. In real life situations you will usually build a part of the domain and integrate it with the existing applications. So you will need composite domain services which will present a unified view towards its consumers, and will integrate the different data sources on the back-end.

How hard this is, usually depends on the openness of the systems involved. Even though a lot of systems are walled of, there are usually several strategies you can employ. This goes from the use of service adapters, wrappers, up to direct database calls and file interfaces. You'd be surprised how far you can get.

Service Discovery, part 2: Top-down

Time for part II. In Part I I explained that the Bottom-up approach is valid in itself, but it is does have its limitations.

A logical alternative is the Top-down approach. In short: this approach looks at the processes in the business, decomposes them into subproces/tasks/functions until 'logical units of work' are defined, which we call services.

Is that all there is to it? In principle: yes. However, there's more to it than meets the eye. A pure BPM (Business Process Management) approach, for example as advocated by IDS Scheer, will deliver a 4-layered process structure for the entire organization. Since the introduction of the Oracle BPA Suite you are then able to convert the lowest level of process to an executable process (BPEL) in which individual tasks become service-operations.

The decomposition of the process in itself is not enough to discover services, because you will have to convert the 'candidate-service' into a real-world executable and discoverable service. This means you will have to compare the functionality of the service with the available infrastructure (which consists of more services, hopefully). Quite often, the service functionality is delimited by specific functionality not being available. For example, suppose you run a very specific proprietary financial system where you cannot interact with either the business logic or the data store. If this is the ultimate source that you will have to use, the chance of being able to fulfill the service functionality needs are quite slim.

Does this invalidate this approach? Of course not. I find this approach working very well in those areas where SOA is used to add new functionality to an existing infrastructure or in a complete greenfield situation (which are quite rare). What this approach does for you is that it's easy to involve the business into designing the new services. The defined services are very recognizable for the business, but not always easy to implement.

Using tools to convert services that are discovered during the process analysis, means that the process analysis has to be done very thoroughly. And, even more important, the process analyst should have knowledge of the limitations of the execution platform. Otherwise you'll end up with services and orchestrations that work rightly on paper, but will never execute in reality. How to address that will be explained in part III.

Oracle ACE Director

I'm very proud to announce that I've been awarded the Oracle ACE Director status. I'd like to thank Clemens Utschig and Jürgen Kress for their nomination. If anything, this will intensivy my activities around SOA in general and Oracle SOA in particular. Thanks guys!

Does SOA Deliver?

I read this very interesting article by Max Pucher. He builds up a quite convincing case about why SOA is, as he calls it, an overhyped buzzword. Every vendor is jumping on the bandwagon, afraid of losing out on this hype.

He does pose some very interesting and helpful considerations, that every SOA architect should keep in mind. Let's not forget that SOA is NOT a silver bullet. SOA is also NOT a technology, but an approach to build architectures which are more tailored to the needs of the business (which sometimes includes educating the business).

BPA on the go

Today I've finished a 4-day course on ARIS. For those of you who don't know ARIS, it's a Business Process Modelling (BPM) tool from IDS Scheer. You might wonder why I'd follow such a course? For several reasons, really. One is that BPM is a part of my daily work. Most architectural studies I do, also address the organization and especially the processes within. Another reason is that Oracle started to work with IDS Scheer about a year ago, to create the Oracle BPA Suite (Business Process Analysis). This suite is meant to bridge the gap between designing Business Processes (BPM) and developing Executable Business Processes (BPEL).

With the BPA Suite it has become possible to automatically generate BPEL Outlines from BPM processes, modelled as EPC (Enterprise Process Chains). So finally we have integration between business design and IT implementation! The good part is, we already have customers ready to use it, so I can bring my new found expertise into play.

Service Discovery, part 1: Bottom Up

Discovering services is one of the more challenging parts of designing an SOA. I will examine several strategies and try to assess the merits and dangers of each and every one of them.

One of the easiest (but not necessarily the best) approaches is: bottom-up. Using this approach one analyses the existing application infrastucture for all available functions that can be exposed (in any way) to the environment. For example: flat files, database procedures, external interfaces, callable forms and even database tables & views. For each function you need to determine whether or not the exposed functionality is independent of other functionality. Any dependency means that there's additional functionality needed to be able to freely expose this function.

This approach will give you a large number of available functions which you can expose as services. However, there's a catch: are these the services my business needs? The answer is: No. These are the services which can be made available from the current applications within the company. This approach will NOT tell you anything about their usability.

The good thing is: it does give you a feel for the needs of the company. These applications are not here by accident.

There's one dangerous aspect to this approach. It is a technical approach to the use and discovery of services, but it is NOT SOA! Stay tuned, I will get back to this!