SOA Magazine

I'm very proud to announce my first publication in SOA Magazine, on preventing failures in an SOA environment by implementing good governance.

It's not quite a standard article. I wrote it together with Dave Berry but it really originated from a discussion we had on his/this blog and in real life as well. We ended up doing a fictitious interview with an architect. The architects' answers reflect our real-world experiences rather closely.

Enjoy, and let me know what you think of the article!

SOA Manifesto revisited

The publication of the SOA Manifesto generates a lot of discussion. As my fellow ACE Director Hajo Normann states on his blog:

I wish for the SOA Manifesto to serve as a guiding foundation for many proposals and solutions in SOA space – similarly as the Agile Manifesto served beautifully well for many to start thinking Agile.

I really hope that will become true. There's been so much discussion about SOA lately, and I still feel that there's still no consensus on what it really means. The SOA manifesto is but a first step in aligning SOA practitioners around the world. I think Thomas Erl and Anne Thomas Manes put it quite clearly during their 'exorcism of the evil SOA': it's the vendors' fault.

Initially, SOA has been hyped by just about all the vendors, in a way to push their middleware technology. Truth is: a lot of middleware technology is based on service orientation, but that does not mean it will give you SOA. SOA is a mindset, a paradigm you can use when building or integrating applications.

The manifesto itself is nice and compact, but will need additional refinement and explanation to make it really worthwile. I hope the working group for the SOA manifesto will be seen by many as experienced SOA practitionists if not SOA thought-leaders. That will broaden its acceptance.

If nothing else, it will facilitate the discussion around SOA. Hopefully we can bring it to an end soon. Thanks to the SOA Manifesto ....

The Future of IT

This month I've visited both Oracle Open World as well as the SOA Symposium. The one hot item that links both conferences is ....... Cloud Computing. That seems to be the newest fad at the moment.

I had several discussions with advocates and opponents to this idea.I've been thinking about Cloud computing a lot. So here's what I think about it.

I'm an advocate of SOA, as you all know. According to Gartner, we're climbing away from the 'Trough of Disillusionment', meaning SOA is rapidly becoming more mature and mainstream. I agree. Already we see SOA being used with EDA and CEP, which is a good thing too. A logical next step would be Cloud Computing. I see Cloud Computing as the logical next step for 'location transparancy'. It's a combination of location transparancy and SaaS, I guess. There will come a time when a consumer of a service doesn't  care where the service is hosted. He will care about who will host it. Companies will look for best-of-breed service producers.

It reminded me a bit of my own history. When I started out as a COBOL programmer in 1984, my first task was to maintain an existing financial system that was built within the company. That's (almost) unheard of these days. NOBODY builds his own financial systems anymore. There are very good, complete and transparant financial systems (and even services) available in the market. Everybody wins, in my opinion. The customer gets real value for less money, the provider earns more money by selling his services to more consumers. The catch: it needs to be very, very, very secure. A service provider needs to make sure that the data and the service is protected, without lowering the service level. One outtage, or one leak can mean the end of the provider.

A lot of companies will hold back. People want to 'see' where their money goes to. Having servers physically present, having a box with software makes them feel they got something for their money. Running software on a cloud doesn't give the same feeling. However, this will change. Companies will have to change. What if your competitor does his manufacturing using a cloud-based infrastructure for less than 30% of the costs, and you don't? You'll be out of business soon.

So yes, I think it's going to happen. It will take a long time, but it will definitely happen. Of course we will see different types of clouds (public vs corporate), but I think this is good news for smaller companies that can't hardly afford IT personnel, let alone a server park. But it's also good news for the best-of-breed service providers, with the additional effect that services will have to become better and better at the same time.

But, getting to the point of this blogpost, what does this mean for the 'average IT-guy'? I'm foreseeing a shift in our line of business. IT personnel within a company will focus on orchestrating and integrating services from different vendors, and maybe - just maybe - maintain one or two left-over applications that are very specific for the enterprise. Custom made software will become outdated. Consultancy firms will specialize in either business/process consultancy and/or data integration (including warehouses) because that's all that will be left. 

Only the larger service providers will have lots of IT staff, as it's their core business. What will be your place in that future?

SOA Manifesto presented

Yesterday at the closing of the SOA Symposium in Rotterdam, the SOA Manifestowas presented to the audience. The working groups have worked very hard to finish the Manifesto to be presented here. A - not complete picture (some people had to leave early) - of the working group:



The whole manifest can be found on the SOA Manifestowebsite.

Open World Update #2 - What's New?

We're into the third day of Oracle Open World. Time to have a look at a few of the changes in the product portfolio. As Thomas Kurian already said: over 2000 product changes. So I'm not going to cover all, but point out some that in my opinion will have an impact.

Some of you will already have looked at Oracle SOA Suite 11gR1 and will be interested to know that 11gR1 PS1 (Patch Set 1) will be out shortly. In this patchset there will be a number of changes. One important thing to note is that PS1 is a new install, not a real patch. The patchsets coming after this one will be patches in the real sense of the world. This one just has too many consequences. One of the limitations that we have seen in 11g is the lack of domains. BPEL domains were a nice feature to be able to discern different deployemnts. For example you could have a domain per department or per tuning requirement, or even for OTAP purposes. With the introduction of SCA we have the SOA domain, but .... you can have only one: default. This is not going to be fixed in PS1 or even PS2, however PS2 will show the introduction of Composite folders where you can at least group different composites. Oracle Product Management is aware of the use cases of this feature and is currently looking into the possible revival of the domain. However, the repercussions of this feature are massive, if you stop to think about it. For now, we'll wait and see.

So what does PS1 bring? For one: it will be possible to have a BPEL process participate in a pub/sub EDN event. I'm sure you can see the advantage of that. It's the 'marriage' of SOA and EDN starting to get shape. Another nice BPEL feature is Monitor Express, where you're have BAM dashboards out of the box for a specific process or process step.

Another feature that has come back is the web-based rule designer and DVM (Domain Value Map) editor. With the MDS available for all configuration items this is just one of the first steps in opening up the MDS for runtime editing. I can't wait for the rest to come!

PS1 also includes some preview functionality that is to come in PS2. I really like the Spring component which enables you to include sping context into an SCA composite. The best part: it works two ways, inbound and outbound.

On to BPM. At this time, BPM is 'lagging behind'. Product management ensures us that after the 11g release of BPM early next year, BPM will adjust its schedule to conform with all the other 11g components. That's good news. What will 11g BPM bring us? Well, we will finally have a Unified Process Engine that's capable of running BPEL, BPMN, Human workflow and business rules. As BPM is a separate layer on top of the SOA Suite infrastructure, it will NOT impact any BPEL instances, even though the UPE is unified. With the introduction of BPM 11g, BPEL can leverage BPM as well.

In my opinion - and I guess it's Oracle's opinion as well - BPM is going to stand much clsoer to the business user. So I expect BPEL to become more integration focused whereas BPM will be more process/workflow focussed.

So far I've only scratched the surface of all the changes that I've noticed or told about. Before I end however, there's two last things I'd like to present to you. The first one is BPEL Roundtrip. To be honest, this was never one of my favourites. In the new setup it will be possible to roundtrip from BPA to either BPEL or BPMN, which opens up new opportunities.

Always good to go out with a bang, so I've saved the best for last. With the introduction of BPM 11g comes the Process Composer. This is an web-based tool aimed at the business users, where they are able to modify any process before it's deployed. Based on available rules, services and process activities, they can modify the process to their needs. This will make imtroduction of new products (based on a default process) an undertaking that can be handled purely by the business, without the need for IT.

The Process Composer initially will have limited functionality (only to be used BEFORE deployment) but will get additional features like runtime editing etc soon thereafter. Way to go guys!

Open World Update #1 - Boys and their Toys

Oracle Open World 2009 was opened last night by a presentation from Scott McNealey, the - still - current CEO of SUN. As always, Scott has a free style of presenting, laced with a lot of humour. I got a sad feeling from him, as these are his last days as SUN CEO. One time he mentioned: this will probably the last time for a while that I'm able to present for such a large crowd. However, there was also a lot of pride in what was achieved during his 26 years at Sun.

As was to be expected, Larry Ellison entered the stage as well, to fend off the IBM attack. Boy, is it going to be a clash of titans. Oracle has really blown away IBM's world record TPC-C benchmark. And how! The new Sun Exadata Database machine, coupled with the new T5100 Flashfire is not only 20% faster, but - more impressively - has 16x better response times, while using a fraction of the energy (Green is HOT!): 300 watts versus 40.000 watts!

But really, it felt like the boys were loose, running in the yard playing with their toys. It sets a tone of what to expect in the next few months. IBM played it hard and got clobbered in the first round. Not the end of the war though!

Architecture: Burden or Blessing?

I'm a big fan of architecture. I really think it makes a big difference whether or not you have an architecture in place to build upon. However, I get into discussions with program- and projectmanagers, who think that architecture is nothing more than a burden for the project. Often I try to explain to them what the benefit is from architecture, even teaching them that architecture and programmanagement have the same goals: realizing business goals (by implementing software).

What does seem to help, is to approach it from a different perspective: what if .... you do not use architecture to guide your development? Lessons learned from the past show us that this will lead to:
* applications with loads of peer-to-peer connections (building a convoluted network of interapplication dependencies)
* unpredictability of IT projects due to the coupled nature of the application landscape. One never knows which application will fall over when a change is applied.
* costly projects due to the unpredictability. Project risks increase, costly measures (extreme testing) have to be implemented. Changes will take longer and longer at higher cost.
* diverse landscape of different applications, tools, languages, hardware etc.
* increased IT costs due to maintaining this diverse landscape and keeping knowledge up-to-date.
* overlapping functionality, multiple implementations of the same business functionality (like customer registration etc)
* loss of insight into the entire IT landscape.

Need I go on? I daresay that architecture can tackle all these issues and deliver a clean and feasible solution. However, it does mean for (some) architects that they will have to come down from their ivory tower and participate in building the new application landscape. They have to be the missionaries to spread the word ....

To 11g or not to 11g

I know, it's cheap to paraphrase a well-known writer. However, it doesn't make the question less valid. Usually the migration to a new major release of a software product is quite an undertaking. This is certainly true for Oracle SOA Suite 11g. That said, there are more than enough reasons to go for 11g, as it delivers a whole lot of new functionality that you might need. That new functionality is not the subject of this post, but you can find any information you need on OTN.

What I expect to see happening however, is a limited amount of back-porting of 11g functionality into the current release (going for 10.1.3.4 or even 10.1.3.5). So, depending on your needs it is imperative to have a good look at your current and future situation, to determine what will be the right strategy for you to take. Don't hesitate to contact me if you have specific questions about the way forward to 11g.

Podcast on SOA Governance

A while ago I was at Oracle HQ talking to Dave Berry, product team manager of Oracle Fusion Middleware, about governance. We started out a discussion on our blogs and we felt it would be a nice idea to convert our discussion into an article.

Somehow Bob Rhubart, manager of the Architect Community within Oracle, got wind of our discussion and invited us to give our take on Governance in general. The result is two podcasts on Governance:

* OTN Arch2Arch Podcast: SOA Governance: It's Cultural
* OTN Arch2Arch Podcast: SOA Governance Perspectives.

Let me know what you think of it!

Holidays

This is my last day before I take a well-deserved 3-week holiday. Meaning I won't be posting any thoughts or comments in the next 3-4 weeks. Probably won't have much access to the Internet and besides, who cares about my holiday pics anyway?

So, see you all in August!

Oracle Fusion Middleware 11g launched. Did you miss it ?

Yesterday Oracle launched the long-awaited 11g release of Fusion Middleware. Perhaps you missed it? Well, nothing lost! You can go to OTN and see a replay of the launch. You'll also find all the downloads and additional information that will help you get started!

Go for it! It's definitely worthwhile!

Process Engines vs State Engines

Process Engines, usually BPEL engines or Workflow engines, execute proces-steps based on a predefined process. This is a sound practice, however not for long-running processes.

Long-running processes have the nasty habit to change over time. This will affect all active and running processes. Why? Because the process engines usually store the process state and corresponding data in a persistence store. The effect of a change in the process requires you to migrate all those saved instances, unless you are able to leave them running the old version of the process.

A different approach is a State engine, where the next step in the process is not determined by the predefined steps in the process, but on the current state of the subject and the event that has been received. This leads to more decoupling between process and services. However, so far I haven't seen any Process engine that works like this.

In theory there's not much difference between the two approaches. You could state that a process engine uses implicit states as defined in the process, whereas a state engine uses explicit states. The end result could/should be exactly the same. A state engine would give some more flexibility, but also needs more functionality to be able to have an overview and control of the process.

Anyone knows of any tool that works like a state engine? I'd really like to know.

ODTUG Update: SOA Symposium

Yesterday at 8 pm we started the SOA Symposium at ODTUG (yes, Sunday and Father's Day to boot). ODTUG is well known for its technological content, but as SOA becomes more and more mainstream, SOA will become a integral part of ODTUG.

The symposium was organized by three Dutch Oracle Ace Directors: Lonneke Dikmans, Lucas Jellema and myself. The setup we choose was a little different then one would expect from a symposium. From the start, we aimed to get a very interactive symposium where the goal is to meet other SOA adepts and to exchange experiences. We managed to achieve this by doing just 3 presentations (to create a mindset) followed by two workshops and a panel discussion.

The day was divided in two parts. In the morning we approached SOA from a business point of view, in the afternoon we discussed technology. The average knowledge level and experience of the participants was rather high, which made discussions very very interesting. It was good to see that there were a lot of Oracle Ace Directors present.

We've made several very interesting conclusions which will be put on the Oracle Wiki.

ACE Briefing @ Oracle HQ

Today we have a briefing of all ACE and ACEDs (Ace Directors). Unfortunately we are not yet allowed to disclose anything that's been said.

Interesting? ABSOLUTELY. It will not surprise anyone that we've seen and heard an awful lot about the soon to be released 11g versions. All I can see for now is to stay tuned!

If you are a serious Oracle professional, the upcoming launch of 11g is something you should be part of. Don't miss out and register here. I think you will be as impressed as all the Ace's and Ace Directors present today.


For those who are attending ODTUG as well, be sure to visit the keynote by Steve Miranda. It's rumoured that we'll see some interesting demo's there around Oracle Fusion Apps.

Accessing Oracle SOA documentation

Ever tried to find that one document about the ESB on OTN? Did you quite remember where you found that BPEL Developers Guide? Well, life's been made easy for you. Marc Kelderman (Oracle Consulting) has made a list of just about all you need to start out with SOA from an Oracle perspective. Go and have a look at Marc Kelderman's shortlist. Way to go, Marc!

SOA: Process driven, message driven or event driven?

For a long time I thought that SOA is process driven, meaning that the functional requirements were discovered by working downwards from a business process perspective to a comprehensive set of services to deliver the business value.

However, the more solutions I see, the less I believe it. Saying so, I realize I need to clarify that a bit more. Whenever you're building an SOA from the Business process downwards, it is smart to discover possible candidate services from your existing legacy environments (building upwards) so you can execute a 'meet in the middle approach'. This usually results in orchestrating business processes (with BPEL) and -simplified - composing composite services (based on atomic services from your legacy).

Is that bad? Not in itself, I think. It depends on the type and the number of processes. Suppose you have - on average - 50.000 running instances of a process. If you have to change that BPEL process you defined, what will you do? The first question to answer is: what do I need to do with my current active processes? Can they continue using the existing process or do the need to switch? Does it depend on the state they are in or not?

This problem gets worse the longer a process runs. It increases the chance that a process change will occur during the time it runs. Besides, not only functional changes impacts the running processes, but an update of the underlying BPEL engine may do so as well. Of course, if the number of instances increases, your problem gets worse too. Ever tried to restart a BPEL engine with 150.000 instances?

If it's a short running process, you could stop all incoming transactions for the specific process and wait for the running processes to finish before you upgrade the process to it's new version. Unfortunately life is usually not that simple.

Let's get back to the initial thought: is an SOA process driven or not? If you follow the scenario I sketched above, you might end up with one that is. What you should keep in mind while designing an SOA application, is that you have to look ahead to see what possible situations you need to able to handle. If you know beforehand that you will have a (very) large number of active processes, that they will be long-running and that they will very likely be subject to change, you might want to consider a different approach. Why? Because migrating running processes in a BPEL environment is a very hard thing to do (and expensive too). You will need to make allowances in your architecture to prevent problems.

In that scenario, a more event-driven or message-driven approach may be the answer. By loosely coupling the stages a process will go through, using either events or messages, you are a lot more flexible when it comes to migrating to new versions of a process(step). This will mean that your 'highest level' process will probably not be visible as a process in your BPEL engine, but the underlying steps may be. Again depending on the duration of the processes.

Oracle Fusion Middleware 11g Launch

Finally the date is known. July 1st will bring us Oracle Fusion Middleware 11g, immediately followed by a huge training program.

What will be delivered:

Oracle WebCenter Suite 11g: the long awaited Portal solution

Oracle WebLogic Suite 11g: BEA Weblogic Server the new middleware foundation

Oracle Identity Management 11g: Next level of security and compliance

Oracle SOA Suite 11g: a new, very tightly integrated environment to build service oriented applications.

What's missing? Oracle BPA Suite 11g, which will probably appear within 1 month.

The delayed delivery (we expected 11g to be released about a year ago) is mainly due to effort that went into the integration of BEA. Too bad that we had to wait, but in the end it is delivering us a whole lot more value. It was worth it! Enjoy 11g!

My Take on Governance: People, Process and Tools

I was reading Dave Berry's blog on how to achieve the ROI on SOA by implementing Oracle SOA Governance tooling. Dave starts with a simple example of how customers - using Oracle BPEL PM - have no clue as on how to achieve reuse. In his blog he states that 'one of the main values of SOA Governance is to deliver savings by enabling this reuse of existing services'. From there on he moves towards governance tooling, enabling visibility, control and monitoring of all SOA artifacts.

My first response was to point out that tooling for reuse is nice, but there's also something about culture. What about the 'not invented here' syndrome? After that, i started thinking (well done, Dave, you got me thinking!) about the elements of SOA Governance. To me governance is about people, process and tools. People, because it's all about the professionals on the job and the need to govern their actions to make sure the right things happen at the right time. To do that, you need a governance process in place that ensures that all artifacts are reviewed and judged to be either confirming to the architecture, or rightly not so. But, that should be done, before anything get's built! Finally it will really enable to process and the people to have good governance tooling available. For to be sure: we do need visibility, control and monitoring to enable reuse. And we do need reuse to reap the major profits of SOA.

For instance, I know of a customer case where the people involved were SOA adepts. However, they decided not to use an existing service, even though it was accepted by the Architecture Board. The main reasons being, that the service was developed by a different department, it's quality and availability could (maybe) not be guaranteed and they could probably do a better job. Unfortunately, this happened not just once, but four times for the same artifact. The company ended up having 4 almost equal services doing almost the same thing. Being able to govern this process would have saved them 3 times building the same component! Tooling in itself would probably have made this problem more visible, but would not have prevented it.

So what are the typical governance processes you see being advocated either internally or by vendors? Let me know!

Oracle Service Bus Explained

There's a lot of confusion towards the positioning of the Oracle Service Bus. In this post I'll try to clear up the issue as best as I can.

First of all, Oracle has already shown a convergence of BEA and Oracle FMW. This will continue even stronger in the upcoming releases, 11g in particular. The strategic platform as it will be introduced with 11g will consist of two main components: Mediator and Oracle Service Bus.

The mediator is an intra-composite mediation component within an application. It is responsible for brokering communications between components that make up a composite (conform Service Component Architecture - SCA). It will enable transformation, routing, event delivery and payload validation. The mediator is almost exclusively based on Oracle ESB (yes, the old Oracle Enterprise Service Bus).

The Oracle Service Bus (OSB) provides service bus capabilities for the entire company, again including standard functionality as transformation, routing, event delivery and payload validation. It's main function is to decouple intra-application communication from inter-application. Endpoint changes will not affect the internals of composite applications. The OSB is based on the Aqualogic Service Bus, augmented with key features from the Oracle ESB, especially JCA adapters, DVM, X-ref and JDev based design-time.

In the long run, I expect the distinction between these implementations to disappear, but I do like the current setup, as it differs between development of application and integration. I wouldn't be surprised to see this pattern appear more.

Oracle Acquires Sun

Today Oracle announced the acquisition of Sun, only 2 weeks after IBM cancelled the acquisition talks with Sun. I'm not sure what this will mean, as Oracle has been moving away from Sun for a long time. This might impact the close relationship Oracle has cultivated with both HP and Dell. It looks like Oracle wants to broaden its horizon into hardware as well, which was already indicated last year with the introduction of the HP Oracle Database Machine.

MySQL, only just acquired last year by Sun, becomes the latest addition to Oracle's database portfolio. Will this mean MySQL will no longer be OSS? Personally I don't think so.

ODTUG SOA/BPM Symposium

Ever hear of the ODTUG Kaleidoscope? It's the yearly conference that's organized by the Oracle Developers Technical User Group (ODTUG). This year there will be a number of symposiums at the day before ODTUG starts.

One of them is the SOA & BPM Symposium, that's organized by Lonneke Dikmans, Lucas Jellema and myself. It will not be your average symposium. It will be a very interactive meeting of minds. The symposium is about working together, exchanging experiences and trying to determine the best, most practical way, to go about implementing an SOA.

So, if you feel you have anything useful to bring to the table or you want to hear what recognized specialists in the industry do and think, why not join us at the ODTUG SOA/BPM Symposium?

This symposium is organized by a large number of Oracle ACE Directors. Expect a large number of them to be there and participate actively! Come and join us!

Crisis, what crisis?

I'm seeing a lot of companies cutting their IT budgets, in an attempt to reduce costs. I do wonder if that's always the best strategy. Yes, IT budget is usually large, and therefore suspect. However, it is usually a fraction of the total costs of an organization. Just getting rid of external consultants (which is sometimes a good idea, anyway) will save you money in the short run, but what does cutting your IT budget mean for the long term? I think that in the long run, it will cost the organization more.

I read an article in my newspaper (NRC Handelsblad) last Saturday about how companies perform, depending on their investment strategy in times of crisis. Recent research shows that organizations that focus on cutting costs and reducing investments will have a harder time to get out of the crisis. Organizations that keep up investments (though at a lower level, more focused) are able to take advantage of the dwindling crisis (we're not there yet, though) a lot sooner.

As with all business decisions it's imperative to keep the longer term in view. Yes, you will have to look closely at all expenses you have as a company. But, sometimes you can do more with less! Possible ways to cost management can be process optimization, integration and consolidation.

The smart thing to do - again, just my humble opinion - is to keep investments at a certain level, but focus on investing in areas that will give you value for your money. For instance: optimizing your process to deliver more value for your customer (keep your customer happy, even in bad times), decrease the number of unwarranted faults, reduce the human tasks (less people), increase intimacy with your chainpartners, etc. The basic premise for these investments are: the return on investment must be visible within 6-9 months.

If at the same time your focus is on creating a more agile (system and application) infrastructure you are a lot more flexible whenever the marketplace allows new initiatives.

Just cutting the IT budget only means that you will not be able to deliver new functionality as the major part of your budget goes to maintaining the installed base. It will not gain you anything other than (short term) continuity.

SOA Marketing is dead! Long live service orientation

Most of you have probably read the blogpost by Anne Thomas Manes about SOA being dead. This coming from an analyst, doesn't surprise me. But I do like the very nice picture she put in.

SOA has been pushed by middleware vendors and analysts alike. But ... they've been pushing SOA as a technology, what it really is NOT. Too bad really, because it makes convincing our businesses a lot harder.

She does have a point in that the term SOA has turned sour. In a way, that's good. It forces us to focus again on what we need to do: we need to build solutions that our business needs. And if we need service orientation to do that, we'll use it.

Look back at all the hypes behind us. They're gone, but the underlying mechanisms (structured programming, object orientation, BPM, decoupling, etc) are still with us and will be for the next years. The same will go for service orientation. So, let's get back to work!

Confusing SOA

It seems that a lot of people forget the A in SOA. A is for Architecture. However, SOA has been pushed by vendors and analysts as being the technological panacee for almost all business issues.

There are parts of the architecture where technology comes in, but mainly architecture is about a vision for the company, about business drivers and goals, about aligning the whole company towards the vision, about implementing the right organizational units, processes, procedures and applications.

What this means, is that a lot of people confuse SOA with technology. I hear people ask 'where can i buy SOA' or state 'we're doing SOA, we've just implemented our first webservice'.

The way I see it, there's architecture and there's technology. You can use service orientation concepts in both of them. So:

* SOA = architecture characterized by the use of service orientation concepts
* SOC = service oriented computing (using service orientation techniques)
* SOI = service oriented infrastructure