Mastering Architecture

Ace and Architect ?

2012-01-16T11:18:00.000+01:00

As of January 1st 2012 I have left the Oracle Ace Program.

I will always be proud of having been part of such an impressive group, and I will surely miss being an Ace Director! Let me try to explain this apparent contradiction.

First of all, I have never been a very technically oriented guy. Nevertheless, I was very lucky to be involved in doing the first BPEL implementation in The Netherlands, back in 2005. That opened up a lot of doors (mainly because we had a lot of issues to fix and needed a lot of support :P). Having a succesful implementation and a drive to spread the vision of service orientation led to an invitation to the Ace Program by Clemens Utschig, with massive support from Jürgen Kress (thank you both!). Being a part of the Ace Community has been very rewarding for me. I have met so many good people in the program, have had very interesting discussions, certainly learned a lot, but above all: got many new friends.

Over the last 2 years I have felt more and more guilty of not living up to expectations. The course of my career changed when I started my own company (MShift) in 2009, as an independent architect. Ever since that time I have been active doing Enterprise & Business Architecture. My involvement with technology declined even further. It meant I was no longer staying abreast of all the new developments around SOA Suite etc, not doing any presentations for the community, nor participating in the forums. I was participating more on OTN ArchBeat (thanks Bob!). All in all, it feels like I am no longer adding to the community.

There's another factor involved, though. I could have stayed Ace just for the fun of it, and for being able to travel to San Francisco every year to meet my fellow Ace Directors. That in itself was tempting, but ... being an independent architect becomes harder when you are affiliated in any way with a vendor. In my case, being an Oracle Ace Director often raises eyebrows on my independency and integrity. People who know me will know that has never been (and never will be) an issue. Regardless, it is more and more often that I find it to be working against me. Commercially it is better for me to become a real independent architect.

I have been thinking about this for a long time, and when we were asked to reevaluate our own position I decided to do what I felt was right: to leave the program. So there it is, in a nutshell. I have become an Ace Alumnus.

Some last words: I would like to use the opportunity to thank Oracle, the Ace Program (especially Vikki Lira and Lillian Buziak) for all their good care, friendliness, input, support, knowledge and a lot of fun. I have no doubt the Ace Program will keep on being succesful and I will try to keep in touch as much as I can.

Consensus, Leadership and Agility

2011-10-19T15:43:00.001+02:00

Many organisations in The Netherlands are culturally consensus based, which means there's often no clear leadership, management is often seen as purely facilitating and everybody more or less goes his or her own way. It's basically the way we (the Dutch) are.

Leadership is resisted. The boss says 'Let's do this my way', and there's bound to be a lively discussion about 'better' ways to do it. Everybody (boss included) accepts this as normal behaviour.

One of the hard parts of applying architecture in such a culture is to get everybody following the strategy as it is set out in the Enterprise Architecture. How are you going to do that if leadership is resisted? What happens? A lot of architectural implementations fail due to resistance from projects.

The introduction of Agile development has increased this problem. An often heard complaint is that too much upfront architecture is a form of over-specification, which is waste in agile terms. That's true, but the argument is too often used to totally ignore architecture and just focus on the user stories for this sprint. This can result in well working applications, which do not quite fit in the strategic direction of the company. Talking about waste ...

I'm not quite sure where the solution to this problem is. One way is to define architecture in such a way that there's something in it for everybody, not just for 'the business' or 'the architect'. Another way is to evangelize, to become a thought leader within the company. Not to push the architecture through, but to convince, stimulate and enthuse all participants.

Normative Restriction of Design Freedom

2011-09-26T14:26:00.001+02:00

I've been working on creating an environment to document an architecture. The customer I'm working on at the moment, has a lot of documentation, but it's all spread over powerpoint presentations, word documents, enterprise architect models, archimate models, visio diagrams, excel spreadsheets and wiki pages. You can imagine it's a tough job to find all the information you need to make choices. So, I have suggested to rationalize all these documents into a Wiki, which will then be the 'Single Point of Truth' concerning the company's architecture.

That's easier said then done. The most important question here is what to document. Document too much and you'll never get anything done, document not enough and people will not know what to do or how to do it. I looked at all (well, not all, but a lot) the architecture metamodels around to see which one of them would give me the structure I needed. Unfortunately, most architectural models ignore concepts like business goals, principles and construction regulations. To me that's quite strange, as I find that these are essential parts of any architecture.

About two weeks ago a NAF workgroup (Dutch Architecture Forum) presented its results. A book on Architecture Principles. In a very lively presentation the authors Danny Greefhorst (ArchiXL) and Erik Proper (Tudor) presented their book, accompanied by two organizations who participated in the research. I'm not going to discuss the book here, as I'm still in the process of reading it, but it did trigger me to re-think the work I had done for documenting architectures.

One of the essential conclusions of the book (and hence the presentation) is, and I quote:

The meaning of enterprise architecture is that it provides a normative restriction of design freedom toward transformation projects and programs

This is a very interesting concept. The more I thought about it, the more I think it is a concept that I could use in my quest for the optimum architecture documentation metamodel.

Luckily I was able to discuss both my quest as well as the book during an IT-eye Open Space. After more than hour of (sometimes) heated discussion we came up with the model you see here on the right hand side. In that model you will find the 3 basic artifacts I think every architecture needs:
1. Business Goals - what needs to be achieved in the next 5 years
2. Principles - what are the (business, information, technical, etc) principles that guide any change within the organization?
3. Construction regulations - what do we have to comply with when constructing artifacts (like processes, datastructures, business units, software,etc).

I have added the 'Generic Components' as well, as every organization has reusable artifacts (reuse is not limited to software!). The fun part of this model is, that you can look upon each and every relationship that's drawn in it as being of the type 'Normative Restriction of Design Freedom'. Business Goals, usually the highest level, is (often severely) restricted by the existing environment. Change comes harder if you have to replace existing constructs. Business Goals themselves restrict Principles. It won't do to have principles that go outside the set goals.

So now I have a potential metamodel which I'm already applying at my customer. It will probably need some more finetuning, especially taking in account specific characteristics of Wiki functionality.

What Happpened to the Designers ?

2011-08-22T09:08:00.000+02:00

A couple of years ago a lot of job names in The Netherlands changed. One day it seems, it was no longer acceptable to call the lady who cleans your house 'cleaner' but 'interior caretaker'. A farmhand became an 'agrarian assistant'. I really don't know why. At the time it looked like it was aimed to change the perception of less-liked jobs. Maybe to increase interest?

These days, everybody is an architect. Process Architect, Software Architect, Information Architect, Infrastructure Architect .....

I experience a devaluation here. I see lots of 'architects' designing applications (making process models, logical data models and use case models, etc), abusing the PSA (for those without IT background: Project Start Architecture) for the job. Even worse: a lot of people and organizations think that's what architecture is all about.

Depending of the level of architecture, an architect should set out the framework, the guidelines for the solution, not the lowest details of the solution itself. He (She included) should focus on the consistency of the total solution for the enterprise, making sure that no effort goes to waste, but delivers to the long term goals of the organisation.

That also means you don't need a whole army of architects. Just a few (depending on the size of your organisation), assisted by a minor army of designers, would do the job!

What or Who's Driving Architecture ?

2011-08-17T10:05:00.000+02:00

All too often I'm hired by the IT Department. First thing I usually have to do then is argue my way into the 'business side' of the company that hired me. After all, Architecture should be a business issue, not to condone IT strategy. Usually IT management grudgingly agrees.

But, when architecture is initiated by IT management, your work is a lot harder. I find it is really one of the hardest part in doing architecture: business often doesn't really care, and IT cannot sell the idea of architecture. You will have to overcome the suspicion of the business. That will take some doing, as nobody is really waiting for you. Make sure you're not on a mission to sell the IT strategy to a business that is a) not interested and b) doing a whole lot of other - more interesting - stuff.

So you really need to make clear to the business that what you're trying to achieve is aimed at achieving (long term) business goals. At the same time you need to stay on good terms with IT management, as they're picking up the bill for your hours. Another kind of trade-off than we're usually used to. The hard - but fun - part lies then in finding the balance between business and IT goals.

Sometimes you're in luck. Business is so pressured into change that it will welcome anyone who can assist them in realizing that change. But don't get used to it....

Why Enterprise Architecture Doesn't Work

2010-09-07T13:49:00.000+02:00

I really think Enterprise Architecture does not work. First of all, I have never seen a working example. All Enterprise Architecture Departments I know (a lot!) are very good at generating enormous amounts of paper, but generally not so good on achieving results. The added value of the architecture function is marginal.

Let me give you one example, at a large bank in the Netherlands, where the EA team consists of about 40 (sic!) architects. Just a few months ago they presented the latest version of the Enterprise Architecture to the management that's supposed to be responsible for implementing it. The presentation consisted of over 100 slides, the architecture documents ranged up to 400 pages. Guess what? The majority of the management dropped the handouts (classified material) in the paper bins when they got back at the office. Back to business.

There are several reasons why I think EA does not work:

architecture is a means to an end. Architecture provides steering and control information, but is not in control!
enterprise architecture generally takes up too much time. Companies are in need of fast turnarounds, quick solutions. No time to spend 9 to 12 months creating a baseline architecture.
trying to fit an entire organization (depending on size ofc) within one single framework is often too much asked. It is possible to define one for smaller companies (say up to 1000 employees), but are we still talking about enterprise architecture then?
enterprise wide scope of activities are usually doomed to fail, due to scope, size and consequences, but often due to distance between daily operations and the project itself
EA often becomes a goal, instead of a means, especially when the teams get over 3 architects

I really think we - as architects - should be a bit more humble. We are a supporting function for the business, helping them decide what course to take and how to implement that course. Giving guidance on what works best, in practice, not on paper. Aiming for solutions that we can implement this year, not within the next five years.

I prefer to use what I call JITJEA: Just In Time, Just Enough Architecture. Yea, very original, I know. Call it Agile Architecture, where you spend very little time on the big picture, and only work out the parts you are going to need in the next couple of months. Combine this with the notion of emerging architectures, where you - as the lead architect - function as a catalyst to let the architecture emerge, and you have a recipe for an active, innovative, involved and loyal community, organization-wide! They will carry the torch where you can't or shouldn't. The architecture that emerges will be supported by many more people then you ever could've hired to staff your enterprise architecture group.

Governance Causes SOA Projects to Fail?

2010-05-04T08:36:00.001+02:00

I got spammed with a mail about a Methodology for SOA that contained a very intriguing thought:

.. it is not poor governance that causes SOA failures but rather governance itself is a single point of failure and a pessimistic organizational structure that causes SOA to fail.

It reminded me about a presentation I attended at the LAC 2009 conference, by Frank Schalkwijk (Atos Origin), on Emerging Architectures. He argues that it's better to 'engage' our specialists than to 'govern' them. That way an architecture can 'emerge' with a lot more support in your organization. Hmmm.

Architecture and Agility

2010-05-04T08:23:00.001+02:00

Last night I had an inspiring dinner with one of my old colleagues, Ronald Doelen. We both see more and more companies changing their project methodology from waterfall to agile. We're both very enthusiastic about that, even though there are some hurdles to take, like architecture for example.

All and good to go Agile, but if you're stuck in an analysis-paralysis situation with your architecture, you will not reap the benefits that Agile brings. You might get away with ignoring the architecting issue - at least for the short term - but it will catch up with you in due course.

So, the answer: Agile Architecture. That got a nice ring to it, but what and how? It all depends on how you look upon architecture and the role of the architect in the company. At the end of our meal - not quite as good as our discussion, tbh - we came up with a number of statements that we'll take with us to work on in the near future. Let me share a few of our thoughts:

The role of the architect in your organization is often determined by the culture of your organization. The architectural style (ivory tower, magician, counsellor, architectus reloadus) determines how you need to adapt your agile projects to align with architecture. The more agile the architect, the less impact.

Architecture is nothing more (or less) than a (limited!) set of choices and the accompanying motivations. Often architects worry about making the best choice. In case you didn't know it yet: there IS NO BEST CHOICE! There's just choices, all with consequences. Use risk-analysis to determine short and long term implications of your choices and base your trade-offs on them. But limit the choices to those you really need NOW.

I'll spend some more time thinking about Agile architecture and especially how to align Enterprise Architecture with Agile projects. I still see some challenges, but will take about those later.

Here Goes the Sun

2010-03-22T15:06:00.000+01:00

A lot of people have been wondering lately what will happen to the different products from Sun. Will they be incorporated into the Oracle stack, or phazed out altogether. It is one of the questions I have had myself. According to the webcasts on the Sun acquisition it seemed that:

GlassFish will continue as the Java EE reference implementation

GlassFish Server included in every Oracle WebLogic Server offering

Sun Java System Web Server will be part of the new Oracle Web Tier offering.

As the Glassfish ESB becomes more and more popular, one of my current customers was very interested in using the Glassfish ESB. Looking over the Glassfish website we got the impression of a fully supported product.

Nothing surprised us more however, when we asked Oracle for a solution package of Glassfish ESB. We received .......... an offer to use Oracle SOA Suite. Further inquiries to Sun learned us that "we're not allowed to offer Glassfish ESB solutions anymore".

So it seems that the choices have become clear, at least within Oracle.

Managing Data and System Integrity in an SOA environment. Are You Prepared?

2010-02-15T14:27:00.002+01:00

One thing that worries me a bit, is that you hardly find any discussion on data and system integrity in an SOA environment. At least, I don't see many. To me, data and system integrity is the most challenging issue we face today and the days to come. We will see more and more combinations of Services, SaaS, legacy apps, which will make a 'standard solution' to this problem even more relevant.

Lots of vendors I talk to tend to minimize the problems of integrity and robustness. They point at their infrastructure and say: well, our infrastructure is WS-Transaction compliant, you will not lose any messages. OK, that may be true, but will it fix failures? You can guess the next comment: with our high availability strategem, we can ensure a 99,99% availability, so you need not to worry.

Despite their reassurances, I still tend to worry: 99,99% availability is not 100%. Things can still (and will!) go wrong.

Let me try to explain the issue as I see it. A very simplistic example:

Service A calls Service B to execute a process. During execution of Service B it calls upon Service C to handle financial details.

Suppose Service B needs to be restored to a certain point in time because of an internal failure (does it really matter what caused the need for in-time recovery? I think it does ...). What does this mean for Service A, B and C? What kind of functionality do we need to have in place to make sure the entire system will not lose its integrity? How do I make sure that Service B 'catches up' with Service A and C? One might argue that due to the statelessness of a service, this shouldn't be a problem, but it is (besides the fact that there's loads of statefull services out there).

This used to be no problem in our legacy application environment. You just rolled-back the whole system and started all over again. However, our boundaries have become much smaller and larger at the same time. It is still a valid approach within a service boundary, but not in an SOA environment (which has no clear boundaries to begin with). Especially as you use services that might not be under your control (SaaS vendors, chainpartners, etc).

My worries come from the fact that most companies I visit, do not have a strategy to maintain this integrity. Mostly, they do not even acknowledge this problem, until they are confronted with it in real life. Suddenly it's become a major problem, because it is very hard to determine what to do, but there's a lot of pressure to fix it right this very minute!

What we need to keep in mind here, is that it is not just a technological problem. It is functional as well. How does the business wants to respond to failing (internal or external) components?

Luckily, having a good middleware infrastructure mitigates the problem somewhat, so it is possible to reduce the problem a lot, but especially in high-volume environments you really need to have a well thought and tried-out strategy in place. There's no falling back to manually fix things when you're processing thousands of transactions a minute.

SOA Magazine

2009-11-18T10:16:00.001+01:00

I'm very proud to announce my first publication in SOA Magazine, on preventing failures in an SOA environment by implementing good governance.

It's not quite a standard article. I wrote it together with Dave Berry but it really originated from a discussion we had on his/this blog and in real life as well. We ended up doing a fictitious interview with an architect. The architects' answers reflect our real-world experiences rather closely.

Enjoy, and let me know what you think of the article!

SOA Manifesto revisited

2009-10-26T15:08:00.002+01:00

The publication of the SOA Manifesto generates a lot of discussion. As my fellow ACE Director Hajo Normann states on his blog:

I wish for the SOA Manifesto to serve as a guiding foundation for many proposals and solutions in SOA space – similarly as the Agile Manifesto served beautifully well for many to start thinking Agile.

I really hope that will become true. There's been so much discussion about SOA lately, and I still feel that there's still no consensus on what it really means. The SOA manifesto is but a first step in aligning SOA practitioners around the world. I think Thomas Erl and Anne Thomas Manes put it quite clearly during their 'exorcism of the evil SOA': it's the vendors' fault.

Initially, SOA has been hyped by just about all the vendors, in a way to push their middleware technology. Truth is: a lot of middleware technology is based on service orientation, but that does not mean it will give you SOA. SOA is a mindset, a paradigm you can use when building or integrating applications.

The manifesto itself is nice and compact, but will need additional refinement and explanation to make it really worthwile. I hope the working group for the SOA manifesto will be seen by many as experienced SOA practitionists if not SOA thought-leaders. That will broaden its acceptance.

If nothing else, it will facilitate the discussion around SOA. Hopefully we can bring it to an end soon. Thanks to the SOA Manifesto ....

The Future of IT

2009-10-26T13:46:00.002+01:00

This month I've visited both Oracle Open World as well as the SOA Symposium. The one hot item that links both conferences is ....... Cloud Computing. That seems to be the newest fad at the moment.

I had several discussions with advocates and opponents to this idea.I've been thinking about Cloud computing a lot. So here's what I think about it.

I'm an advocate of SOA, as you all know. According to Gartner, we're climbing away from the 'Trough of Disillusionment', meaning SOA is rapidly becoming more mature and mainstream. I agree. Already we see SOA being used with EDA and CEP, which is a good thing too. A logical next step would be Cloud Computing. I see Cloud Computing as the logical next step for 'location transparancy'. It's a combination of location transparancy and SaaS, I guess. There will come a time when a consumer of a service doesn't care where the service is hosted. He will care about who will host it. Companies will look for best-of-breed service producers.

It reminded me a bit of my own history. When I started out as a COBOL programmer in 1984, my first task was to maintain an existing financial system that was built within the company. That's (almost) unheard of these days. NOBODY builds his own financial systems anymore. There are very good, complete and transparant financial systems (and even services) available in the market. Everybody wins, in my opinion. The customer gets real value for less money, the provider earns more money by selling his services to more consumers. The catch: it needs to be very, very, very secure. A service provider needs to make sure that the data and the service is protected, without lowering the service level. One outtage, or one leak can mean the end of the provider.

A lot of companies will hold back. People want to 'see' where their money goes to. Having servers physically present, having a box with software makes them feel they got something for their money. Running software on a cloud doesn't give the same feeling. However, this will change. Companies will have to change. What if your competitor does his manufacturing using a cloud-based infrastructure for less than 30% of the costs, and you don't? You'll be out of business soon.

So yes, I think it's going to happen. It will take a long time, but it will definitely happen. Of course we will see different types of clouds (public vs corporate), but I think this is good news for smaller companies that can't hardly afford IT personnel, let alone a server park. But it's also good news for the best-of-breed service providers, with the additional effect that services will have to become better and better at the same time.

But, getting to the point of this blogpost, what does this mean for the 'average IT-guy'? I'm foreseeing a shift in our line of business. IT personnel within a company will focus on orchestrating and integrating services from different vendors, and maybe - just maybe - maintain one or two left-over applications that are very specific for the enterprise. Custom made software will become outdated. Consultancy firms will specialize in either business/process consultancy and/or data integration (including warehouses) because that's all that will be left.

Only the larger service providers will have lots of IT staff, as it's their core business. What will be your place in that future?

SOA Manifesto presented

2009-10-24T20:59:00.001+02:00

Yesterday at the closing of the SOA Symposium in Rotterdam, the SOA Manifestowas presented to the audience. The working groups have worked very hard to finish the Manifesto to be presented here. A - not complete picture (some people had to leave early) - of the working group:

The whole manifest can be found on the SOA Manifestowebsite.

Open World Update #2 - What's New?

2009-10-14T22:30:00.003+02:00

We're into the third day of Oracle Open World. Time to have a look at a few of the changes in the product portfolio. As Thomas Kurian already said: over 2000 product changes. So I'm not going to cover all, but point out some that in my opinion will have an impact.

Some of you will already have looked at Oracle SOA Suite 11gR1 and will be interested to know that 11gR1 PS1 (Patch Set 1) will be out shortly. In this patchset there will be a number of changes. One important thing to note is that PS1 is a new install, not a real patch. The patchsets coming after this one will be patches in the real sense of the world. This one just has too many consequences. One of the limitations that we have seen in 11g is the lack of domains. BPEL domains were a nice feature to be able to discern different deployemnts. For example you could have a domain per department or per tuning requirement, or even for OTAP purposes. With the introduction of SCA we have the SOA domain, but .... you can have only one: default. This is not going to be fixed in PS1 or even PS2, however PS2 will show the introduction of Composite folders where you can at least group different composites. Oracle Product Management is aware of the use cases of this feature and is currently looking into the possible revival of the domain. However, the repercussions of this feature are massive, if you stop to think about it. For now, we'll wait and see.

So what does PS1 bring? For one: it will be possible to have a BPEL process participate in a pub/sub EDN event. I'm sure you can see the advantage of that. It's the 'marriage' of SOA and EDN starting to get shape. Another nice BPEL feature is Monitor Express, where you're have BAM dashboards out of the box for a specific process or process step.

Another feature that has come back is the web-based rule designer and DVM (Domain Value Map) editor. With the MDS available for all configuration items this is just one of the first steps in opening up the MDS for runtime editing. I can't wait for the rest to come!

PS1 also includes some preview functionality that is to come in PS2. I really like the Spring component which enables you to include sping context into an SCA composite. The best part: it works two ways, inbound and outbound.

On to BPM. At this time, BPM is 'lagging behind'. Product management ensures us that after the 11g release of BPM early next year, BPM will adjust its schedule to conform with all the other 11g components. That's good news. What will 11g BPM bring us? Well, we will finally have a Unified Process Engine that's capable of running BPEL, BPMN, Human workflow and business rules. As BPM is a separate layer on top of the SOA Suite infrastructure, it will NOT impact any BPEL instances, even though the UPE is unified. With the introduction of BPM 11g, BPEL can leverage BPM as well.

In my opinion - and I guess it's Oracle's opinion as well - BPM is going to stand much clsoer to the business user. So I expect BPEL to become more integration focused whereas BPM will be more process/workflow focussed.

So far I've only scratched the surface of all the changes that I've noticed or told about. Before I end however, there's two last things I'd like to present to you. The first one is BPEL Roundtrip. To be honest, this was never one of my favourites. In the new setup it will be possible to roundtrip from BPA to either BPEL or BPMN, which opens up new opportunities.

Always good to go out with a bang, so I've saved the best for last. With the introduction of BPM 11g comes the Process Composer. This is an web-based tool aimed at the business users, where they are able to modify any process before it's deployed. Based on available rules, services and process activities, they can modify the process to their needs. This will make imtroduction of new products (based on a default process) an undertaking that can be handled purely by the business, without the need for IT.

The Process Composer initially will have limited functionality (only to be used BEFORE deployment) but will get additional features like runtime editing etc soon thereafter. Way to go guys!

Open World Update #1 - Boys and their Toys

2009-10-12T19:37:00.004+02:00

Oracle Open World 2009 was opened last night by a presentation from Scott McNealey, the - still - current CEO of SUN. As always, Scott has a free style of presenting, laced with a lot of humour. I got a sad feeling from him, as these are his last days as SUN CEO. One time he mentioned: this will probably the last time for a while that I'm able to present for such a large crowd. However, there was also a lot of pride in what was achieved during his 26 years at Sun.

As was to be expected, Larry Ellison entered the stage as well, to fend off the IBM attack. Boy, is it going to be a clash of titans. Oracle has really blown away IBM's world record TPC-C benchmark. And how! The new Sun Exadata Database machine, coupled with the new T5100 Flashfire is not only 20% faster, but - more impressively - has 16x better response times, while using a fraction of the energy (Green is HOT!): 300 watts versus 40.000 watts!

But really, it felt like the boys were loose, running in the yard playing with their toys. It sets a tone of what to expect in the next few months. IBM played it hard and got clobbered in the first round. Not the end of the war though!

Architecture: Burden or Blessing?

2009-09-22T12:10:00.004+02:00

I'm a big fan of architecture. I really think it makes a big difference whether or not you have an architecture in place to build upon. However, I get into discussions with program- and projectmanagers, who think that architecture is nothing more than a burden for the project. Often I try to explain to them what the benefit is from architecture, even teaching them that architecture and programmanagement have the same goals: realizing business goals (by implementing software).

What does seem to help, is to approach it from a different perspective: what if .... you do not use architecture to guide your development? Lessons learned from the past show us that this will lead to:
* applications with loads of peer-to-peer connections (building a convoluted network of interapplication dependencies)
* unpredictability of IT projects due to the coupled nature of the application landscape. One never knows which application will fall over when a change is applied.
* costly projects due to the unpredictability. Project risks increase, costly measures (extreme testing) have to be implemented. Changes will take longer and longer at higher cost.
* diverse landscape of different applications, tools, languages, hardware etc.
* increased IT costs due to maintaining this diverse landscape and keeping knowledge up-to-date.
* overlapping functionality, multiple implementations of the same business functionality (like customer registration etc)
* loss of insight into the entire IT landscape.

Need I go on? I daresay that architecture can tackle all these issues and deliver a clean and feasible solution. However, it does mean for (some) architects that they will have to come down from their ivory tower and participate in building the new application landscape. They have to be the missionaries to spread the word ....

To 11g or not to 11g

2009-08-11T11:00:00.004+02:00

I know, it's cheap to paraphrase a well-known writer. However, it doesn't make the question less valid. Usually the migration to a new major release of a software product is quite an undertaking. This is certainly true for Oracle SOA Suite 11g. That said, there are more than enough reasons to go for 11g, as it delivers a whole lot of new functionality that you might need. That new functionality is not the subject of this post, but you can find any information you need on OTN.

What I expect to see happening however, is a limited amount of back-porting of 11g functionality into the current release (going for 10.1.3.4 or even 10.1.3.5). So, depending on your needs it is imperative to have a good look at your current and future situation, to determine what will be the right strategy for you to take. Don't hesitate to contact me if you have specific questions about the way forward to 11g.

Podcast on SOA Governance

2009-08-11T10:53:00.009+02:00

A while ago I was at Oracle HQ talking to Dave Berry, product team manager of Oracle Fusion Middleware, about governance. We started out a discussion on our blogs and we felt it would be a nice idea to convert our discussion into an article.

Somehow Bob Rhubart, manager of the Architect Community within Oracle, got wind of our discussion and invited us to give our take on Governance in general. The result is two podcasts on Governance:

* OTN Arch2Arch Podcast: SOA Governance: It's Cultural
* OTN Arch2Arch Podcast: SOA Governance Perspectives.

Let me know what you think of it!

Holidays

2009-07-14T09:33:00.003+02:00

This is my last day before I take a well-deserved 3-week holiday. Meaning I won't be posting any thoughts or comments in the next 3-4 weeks. Probably won't have much access to the Internet and besides, who cares about my holiday pics anyway?

So, see you all in August!

Oracle Fusion Middleware 11g launched. Did you miss it ?

2009-07-02T11:59:00.004+02:00

Yesterday Oracle launched the long-awaited 11g release of Fusion Middleware. Perhaps you missed it? Well, nothing lost! You can go to OTN and see a replay of the launch. You'll also find all the downloads and additional information that will help you get started!

Go for it! It's definitely worthwhile!

Process Engines vs State Engines

2009-06-30T14:34:00.002+02:00

Process Engines, usually BPEL engines or Workflow engines, execute proces-steps based on a predefined process. This is a sound practice, however not for long-running processes.

Long-running processes have the nasty habit to change over time. This will affect all active and running processes. Why? Because the process engines usually store the process state and corresponding data in a persistence store. The effect of a change in the process requires you to migrate all those saved instances, unless you are able to leave them running the old version of the process.

A different approach is a State engine, where the next step in the process is not determined by the predefined steps in the process, but on the current state of the subject and the event that has been received. This leads to more decoupling between process and services. However, so far I haven't seen any Process engine that works like this.

In theory there's not much difference between the two approaches. You could state that a process engine uses implicit states as defined in the process, whereas a state engine uses explicit states. The end result could/should be exactly the same. A state engine would give some more flexibility, but also needs more functionality to be able to have an overview and control of the process.

Anyone knows of any tool that works like a state engine? I'd really like to know.

ODTUG Update: SOA Symposium

2009-06-22T23:20:00.003+02:00

Yesterday at 8 pm we started the SOA Symposium at ODTUG (yes, Sunday and Father's Day to boot). ODTUG is well known for its technological content, but as SOA becomes more and more mainstream, SOA will become a integral part of ODTUG.

The symposium was organized by three Dutch Oracle Ace Directors: Lonneke Dikmans, Lucas Jellema and myself. The setup we choose was a little different then one would expect from a symposium. From the start, we aimed to get a very interactive symposium where the goal is to meet other SOA adepts and to exchange experiences. We managed to achieve this by doing just 3 presentations (to create a mindset) followed by two workshops and a panel discussion.

The day was divided in two parts. In the morning we approached SOA from a business point of view, in the afternoon we discussed technology. The average knowledge level and experience of the participants was rather high, which made discussions very very interesting. It was good to see that there were a lot of Oracle Ace Directors present.

We've made several very interesting conclusions which will be put on the Oracle Wiki.

ACE Briefing @ Oracle HQ

2009-06-20T01:13:00.004+02:00

Today we have a briefing of all ACE and ACEDs (Ace Directors). Unfortunately we are not yet allowed to disclose anything that's been said.

Interesting? ABSOLUTELY. It will not surprise anyone that we've seen and heard an awful lot about the soon to be released 11g versions. All I can see for now is to stay tuned!

If you are a serious Oracle professional, the upcoming launch of 11g is something you should be part of. Don't miss out and register here. I think you will be as impressed as all the Ace's and Ace Directors present today.

For those who are attending ODTUG as well, be sure to visit the keynote by Steve Miranda. It's rumoured that we'll see some interesting demo's there around Oracle Fusion Apps.

Accessing Oracle SOA documentation

2009-06-15T22:47:00.002+02:00

Ever tried to find that one document about the ESB on OTN? Did you quite remember where you found that BPEL Developers Guide? Well, life's been made easy for you. Marc Kelderman (Oracle Consulting) has made a list of just about all you need to start out with SOA from an Oracle perspective. Go and have a look at Marc Kelderman's shortlist. Way to go, Marc!

SOA: Process driven, message driven or event driven?

2009-06-03T14:51:00.002+02:00

For a long time I thought that SOA is process driven, meaning that the functional requirements were discovered by working downwards from a business process perspective to a comprehensive set of services to deliver the business value.

However, the more solutions I see, the less I believe it. Saying so, I realize I need to clarify that a bit more. Whenever you're building an SOA from the Business process downwards, it is smart to discover possible candidate services from your existing legacy environments (building upwards) so you can execute a 'meet in the middle approach'. This usually results in orchestrating business processes (with BPEL) and -simplified - composing composite services (based on atomic services from your legacy).

Is that bad? Not in itself, I think. It depends on the type and the number of processes. Suppose you have - on average - 50.000 running instances of a process. If you have to change that BPEL process you defined, what will you do? The first question to answer is: what do I need to do with my current active processes? Can they continue using the existing process or do the need to switch? Does it depend on the state they are in or not?

This problem gets worse the longer a process runs. It increases the chance that a process change will occur during the time it runs. Besides, not only functional changes impacts the running processes, but an update of the underlying BPEL engine may do so as well. Of course, if the number of instances increases, your problem gets worse too. Ever tried to restart a BPEL engine with 150.000 instances?

If it's a short running process, you could stop all incoming transactions for the specific process and wait for the running processes to finish before you upgrade the process to it's new version. Unfortunately life is usually not that simple.

Let's get back to the initial thought: is an SOA process driven or not? If you follow the scenario I sketched above, you might end up with one that is. What you should keep in mind while designing an SOA application, is that you have to look ahead to see what possible situations you need to able to handle. If you know beforehand that you will have a (very) large number of active processes, that they will be long-running and that they will very likely be subject to change, you might want to consider a different approach. Why? Because migrating running processes in a BPEL environment is a very hard thing to do (and expensive too). You will need to make allowances in your architecture to prevent problems.

In that scenario, a more event-driven or message-driven approach may be the answer. By loosely coupling the stages a process will go through, using either events or messages, you are a lot more flexible when it comes to migrating to new versions of a process(step). This will mean that your 'highest level' process will probably not be visible as a process in your BPEL engine, but the underlying steps may be. Again depending on the duration of the processes.

Oracle Fusion Middleware 11g Launch

2009-06-03T10:28:00.003+02:00

Finally the date is known. July 1st will bring us Oracle Fusion Middleware 11g, immediately followed by a huge training program.

What will be delivered:

Oracle WebCenter Suite 11g: the long awaited Portal solution

Oracle WebLogic Suite 11g: BEA Weblogic Server the new middleware foundation

Oracle Identity Management 11g: Next level of security and compliance

Oracle SOA Suite 11g: a new, very tightly integrated environment to build service oriented applications.

What's missing? Oracle BPA Suite 11g, which will probably appear within 1 month.

The delayed delivery (we expected 11g to be released about a year ago) is mainly due to effort that went into the integration of BEA. Too bad that we had to wait, but in the end it is delivering us a whole lot more value. It was worth it! Enjoy 11g!

My Take on Governance: People, Process and Tools

2009-05-11T20:04:00.004+02:00

I was reading Dave Berry's blog on how to achieve the ROI on SOA by implementing Oracle SOA Governance tooling. Dave starts with a simple example of how customers - using Oracle BPEL PM - have no clue as on how to achieve reuse. In his blog he states that 'one of the main values of SOA Governance is to deliver savings by enabling this reuse of existing services'. From there on he moves towards governance tooling, enabling visibility, control and monitoring of all SOA artifacts.

My first response was to point out that tooling for reuse is nice, but there's also something about culture. What about the 'not invented here' syndrome? After that, i started thinking (well done, Dave, you got me thinking!) about the elements of SOA Governance. To me governance is about people, process and tools. People, because it's all about the professionals on the job and the need to govern their actions to make sure the right things happen at the right time. To do that, you need a governance process in place that ensures that all artifacts are reviewed and judged to be either confirming to the architecture, or rightly not so. But, that should be done, before anything get's built! Finally it will really enable to process and the people to have good governance tooling available. For to be sure: we do need visibility, control and monitoring to enable reuse. And we do need reuse to reap the major profits of SOA.

For instance, I know of a customer case where the people involved were SOA adepts. However, they decided not to use an existing service, even though it was accepted by the Architecture Board. The main reasons being, that the service was developed by a different department, it's quality and availability could (maybe) not be guaranteed and they could probably do a better job. Unfortunately, this happened not just once, but four times for the same artifact. The company ended up having 4 almost equal services doing almost the same thing. Being able to govern this process would have saved them 3 times building the same component! Tooling in itself would probably have made this problem more visible, but would not have prevented it.

So what are the typical governance processes you see being advocated either internally or by vendors? Let me know!

Oracle Service Bus Explained

2009-05-11T15:52:00.003+02:00

There's a lot of confusion towards the positioning of the Oracle Service Bus. In this post I'll try to clear up the issue as best as I can.

First of all, Oracle has already shown a convergence of BEA and Oracle FMW. This will continue even stronger in the upcoming releases, 11g in particular. The strategic platform as it will be introduced with 11g will consist of two main components: Mediator and Oracle Service Bus.

The mediator is an intra-composite mediation component within an application. It is responsible for brokering communications between components that make up a composite (conform Service Component Architecture - SCA). It will enable transformation, routing, event delivery and payload validation. The mediator is almost exclusively based on Oracle ESB (yes, the old Oracle Enterprise Service Bus).

The Oracle Service Bus (OSB) provides service bus capabilities for the entire company, again including standard functionality as transformation, routing, event delivery and payload validation. It's main function is to decouple intra-application communication from inter-application. Endpoint changes will not affect the internals of composite applications. The OSB is based on the Aqualogic Service Bus, augmented with key features from the Oracle ESB, especially JCA adapters, DVM, X-ref and JDev based design-time.

In the long run, I expect the distinction between these implementations to disappear, but I do like the current setup, as it differs between development of application and integration. I wouldn't be surprised to see this pattern appear more.

Oracle Acquires Sun

2009-04-20T14:58:00.002+02:00

Today Oracle announced the acquisition of Sun, only 2 weeks after IBM cancelled the acquisition talks with Sun. I'm not sure what this will mean, as Oracle has been moving away from Sun for a long time. This might impact the close relationship Oracle has cultivated with both HP and Dell. It looks like Oracle wants to broaden its horizon into hardware as well, which was already indicated last year with the introduction of the HP Oracle Database Machine.

MySQL, only just acquired last year by Sun, becomes the latest addition to Oracle's database portfolio. Will this mean MySQL will no longer be OSS? Personally I don't think so.

ODTUG SOA/BPM Symposium

2009-04-09T13:57:00.004+02:00

Ever hear of the ODTUG Kaleidoscope? It's the yearly conference that's organized by the Oracle Developers Technical User Group (ODTUG). This year there will be a number of symposiums at the day before ODTUG starts.

One of them is the SOA & BPM Symposium, that's organized by Lonneke Dikmans, Lucas Jellema and myself. It will not be your average symposium. It will be a very interactive meeting of minds. The symposium is about working together, exchanging experiences and trying to determine the best, most practical way, to go about implementing an SOA.

So, if you feel you have anything useful to bring to the table or you want to hear what recognized specialists in the industry do and think, why not join us at the ODTUG SOA/BPM Symposium?

This symposium is organized by a large number of Oracle ACE Directors. Expect a large number of them to be there and participate actively! Come and join us!

Crisis, what crisis?

2009-04-09T13:35:00.003+02:00

I'm seeing a lot of companies cutting their IT budgets, in an attempt to reduce costs. I do wonder if that's always the best strategy. Yes, IT budget is usually large, and therefore suspect. However, it is usually a fraction of the total costs of an organization. Just getting rid of external consultants (which is sometimes a good idea, anyway) will save you money in the short run, but what does cutting your IT budget mean for the long term? I think that in the long run, it will cost the organization more.

I read an article in my newspaper (NRC Handelsblad) last Saturday about how companies perform, depending on their investment strategy in times of crisis. Recent research shows that organizations that focus on cutting costs and reducing investments will have a harder time to get out of the crisis. Organizations that keep up investments (though at a lower level, more focused) are able to take advantage of the dwindling crisis (we're not there yet, though) a lot sooner.

As with all business decisions it's imperative to keep the longer term in view. Yes, you will have to look closely at all expenses you have as a company. But, sometimes you can do more with less! Possible ways to cost management can be process optimization, integration and consolidation.

The smart thing to do - again, just my humble opinion - is to keep investments at a certain level, but focus on investing in areas that will give you value for your money. For instance: optimizing your process to deliver more value for your customer (keep your customer happy, even in bad times), decrease the number of unwarranted faults, reduce the human tasks (less people), increase intimacy with your chainpartners, etc. The basic premise for these investments are: the return on investment must be visible within 6-9 months.

If at the same time your focus is on creating a more agile (system and application) infrastructure you are a lot more flexible whenever the marketplace allows new initiatives.

Just cutting the IT budget only means that you will not be able to deliver new functionality as the major part of your budget goes to maintaining the installed base. It will not gain you anything other than (short term) continuity.

SOA Marketing is dead! Long live service orientation

2009-02-04T13:25:00.006+01:00

Most of you have probably read the blogpost by Anne Thomas Manes about SOA being dead. This coming from an analyst, doesn't surprise me. But I do like the very nice picture she put in.

SOA has been pushed by middleware vendors and analysts alike. But ... they've been pushing SOA as a technology, what it really is NOT. Too bad really, because it makes convincing our businesses a lot harder.

She does have a point in that the term SOA has turned sour. In a way, that's good. It forces us to focus again on what we need to do: we need to build solutions that our business needs. And if we need service orientation to do that, we'll use it.

Look back at all the hypes behind us. They're gone, but the underlying mechanisms (structured programming, object orientation, BPM, decoupling, etc) are still with us and will be for the next years. The same will go for service orientation. So, let's get back to work!

Confusing SOA

2009-02-04T13:18:00.002+01:00

It seems that a lot of people forget the A in SOA. A is for Architecture. However, SOA has been pushed by vendors and analysts as being the technological panacee for almost all business issues.

There are parts of the architecture where technology comes in, but mainly architecture is about a vision for the company, about business drivers and goals, about aligning the whole company towards the vision, about implementing the right organizational units, processes, procedures and applications.

What this means, is that a lot of people confuse SOA with technology. I hear people ask 'where can i buy SOA' or state 'we're doing SOA, we've just implemented our first webservice'.

The way I see it, there's architecture and there's technology. You can use service orientation concepts in both of them. So:

* SOA = architecture characterized by the use of service orientation concepts
* SOC = service oriented computing (using service orientation techniques)
* SOI = service oriented infrastructure

Modifying running BPEL instances

2008-12-02T09:23:00.003+01:00

Yahoo! Soon we will be able to modify running BPEL instances.

Ever since I started to work with BPEL, we ran into the problem that when you upgrade a BPEL process to reflect a changed situation, you will have to think about how to handle the already running instances. There were not many options: you either let them run their course or aborted them to restart them with the new version.

In our case we had a lot (15.000+) of rather long-running processes (up to 14 months). So you can imagine both options were not very attractive. Yesterday I had an upgrade about the upcoming Oracle SOA Suite 11g. The very good news was that in Oracle BPEL 11g you are able to modify already running instances. That is very very good news indeed.

It will have some limitations however, but that has more to do with the structure of the process. You will have to think about where you will allow any modification on running instances.

A Must Read!

2008-10-13T09:59:00.001+02:00

Last week, at the International SOA Symposium in Amsterdam, Thomas Erl launched two new books which I think are a must read for any serious SOA practitioner. The first one to be launched was Web Service Contract Design & Versioning for SOA, which I've started reading already. It's a massive book (again), but so far I'm impressed. Easy to read, good examples, definitely worth the time reading it.

I'm even more eager for the second book that was announced: SOA Design Patterns. Unfortunately this book is still on its way from Amazon to my home, so I cannot give any insights, but I do expect a lot from it. As Thomas explained during his presentation at the International SOA Symposium, it consists of 85 real-world, real-practice design patterns. One of the nice things with this book is, that there's a website along with it, containing all the 85 design patterns, but also the candidates that didn't make it into the book. If you have a design pattern you think should be in the book, publish it on the site, and maybe it'll be in the next release ....

For more information go to Thomas Erl's SOA Books website.

Will Dell Be Oracle's Next Acquisition?

2008-09-27T03:37:00.002+02:00

Oracle has taken over a lot of companies during the last few years. Looking back (hindsight is always easy) it shows a very clear strategy. All - or at least most - of the acquisitions have added one or more capabilities to the SOA technology stack that Oracle was building. To say the least: I'm very impressed with that strategy.

The announcement of the Database Machine and the Oracle Exadata Storage made me wonder where Oracle is taking this now. Will they not only acquire for technology but now for hardware as well? Michael Dell might start to get worried .....

The Future According to Oracle

2008-09-27T03:28:00.003+02:00

I'm getting very mixed messages from Oracle, especially here during Oracle Open World. To say the least: I was very surprised when Oracle announced the conversion/migration from Forms to APEX, while at the same time stating that APEX is 'only' for localized applications and while pushing forward on SOA.

In a way I do understand the reasoning behind it. There's so many people out there still relying on Forms and sort of afraid of going down the SOA path. To get these customers move over to APEX will at least mean that they will stay customers of Oracle. However, APEX is no SOA, in fact it is contradictory to SOA, as it is based on a very strong coupling from the front-end to the database.

This development, coupled with the fact that a lot of small and medium businesses find the Oracle product line a bit too expensive for their taste, might lead to a kind of splitting of the market. SMB goes APEX, the rest goes SOA? Or am I just being too cynical .....

Service Contracts

2008-09-24T19:46:00.003+02:00

A while back I wrote a blog about the WSDL, mainly about the fact that it is not quite what we need to describe the service contract. Last night we had a get-together of Oracle ACEs and Oracle ACE Directors, where we discussed a lot of topics. One of the topics was about the Service Contract.

The discussion focussed on whether or not you need to have things like binding, security and policies in a Service Contract. Soon we found out that you need to forget the WSDL when you're entering such a discussion, because it muddles the discussion.

At 1.30 this morning we quit the discussion, without having reached a consensus. However, it did clear my mind as to my opinion on this matter.

What we do need in a Service Contract is:
- Definition of the interface
- Definition of the Service (what functionality will be delivered, ie what does it do?!)
- Service quality (QoS, SLA, etc)
- Applicable policies
- Allowable bindings

However, this is about the external contract. This does NOT mean that it is all IN the service. For example: adherence to company policies is obviously a matter of importance. However, security and policies are things you should not worry about in a service, but should add to the service before deploying it. It should be decoupled, so that changes in company (security) policies do not lead to changes in the implementation of a service.

As the discussion is not finished, expect me to get back on this real soon. I urge all my readers to comment on this! Would be nice if we could come up with a definition.

Safe Harbor

2008-09-24T19:43:00.001+02:00

One of the running gags here is 'May or May not'. There's hardly any statement made by an Oracle employee that doesn't start or end with it. Of course I'm exaggerating, but we do hear it a lot these days.

Especially around the Database there's some buzz going around. So I'll expect some announcement to be made later today during Larry's keynote. Well, whether or not he does, a Larry Ellison keynote is usually fun to see. Not to be missed!

OOW off to a good start!

2008-09-22T20:56:00.003+02:00

This morning Oracle Open World started off with keynotes from Charles Phillips and Chuck Rozwat. Again it's fabulous weather during OOW, thousands of attendees on the streets between Marriot and Moscone, and the smell of expectancy in the air. I do expect some announcements these days about new releases and new strategies. Time will tell!

Anything interesting that comes up in the next few days, I will certainly report on here. Right now, I've got to hurry to get to my next assignment: ACE Office Hours in the OTN lounge. Maybe I'll see you there?

Mud Slinging and the Enterprise Service Bus

2008-09-11T08:39:00.003+02:00

The last few weeks I've been reading up on the Enterprise Service Bus (ESB). I can tell you: it was an amazing journey. I started out with the book by David Chappell and from then on went to IBM DeveloperWorks which is usually a good spot to get technical info. These two sources combined give a nice overview of what an ESB could offer you, and how to put it to use.

That was not enough. I needed more information, especially on what an ESB really is and how to fit it into our architecture. So I looked at a couple of Blogs around the ESB and the Mediation patterns. What surprised me was the number of discussions and especially the tone of them. It seems the whole ESB discussion has degraded to a kind of mudslinging between different fundamental groups. One group states that the ESB is an architectural monstrosity which is not needed whenever the initial design of the service constructs adhere to standards of good design. The other group argues that there will never be such an environment. Oh btw, both groups think they hold the truth, the whole truth and nothing but the truth. Well, what else is new?

I think I understand where this argument is coming from. Just have a look at the ESBs that vendors today deliver. Probably the only thing they have in common is the name and the fact that it can do some routing and transformation. There the resemblance ends. In short, there's no clear definition of what an ESB is or does.

If I look at the architectural concepts that I found during my search, especially mediation, integration and security, there's a lot to be said about using these patterns. If you can find a tool (whether or not it is called an ESB) that fits your needs now and in the near future, why not use it?

So in my opinion, there's definitely a case to be made about an 'ESB'. Just don't know yet exactly what should be in there and what not .....

Oracle ACE Office Hours

2008-08-22T09:56:00.002+02:00

Will you be visiting Oracle Open World this year? When you do, make sure to free up some time to visit the ACE's. Every day they will hold office at the OTN Lounge in Moscone West (3rd level). Sort of like a roundtable, advice session or a "live" discussion forum thread-- however you want to view it.

What: Oracle ACE Office Hours
Where: OTN Lounge, Moscone West, 3rd Level

Oracle ACE Office Hours:
Monday, September 22
12:00 – 1:30 and 4:00 – 5:30pm

Tuesday, September 23
12:00 – 1:30 and 4:00 – 5:30pm

Wednesday, September 24
12:00 – 1:30 and 4:00 – 5:30pm

Thursday, September 25
11:00 am – 12:00 pm

I'll be there!

Growing Pains

2008-07-28T11:52:00.001+02:00

It’s not hard to build an SOA. That is, technically. Once the domains are clear and the services defined, it’s kinda easy to implement. Sure, there’s lots of work to be done, but no rocket science. But, what happens after you’ve done your Proof of Concept and your first and maybe even your second real-life project?

By then you’re sitting on top of a lot of services, of different granularities and of different types, all mixed together. If you’re lucky, the semantics are still correct (no mismatches yet!). Maybe you even have an ESB running. Slowly but surely the need for additional tooling becomes evident. How are you going to manage this whole environment? How are you going to make sure the next project reuses the generic services that have been so painstakingly defined?

A logical first step would be to get a UDDI, where you can at least register your services. Usually this is not a tool that you would implement in your first project, as you do not yet have the need for location transparency or runtime service discovery. That may change, however. But the main reason for the introduction of a UDDI is the discovery of services: what are the reusable assets my organization has?
If you grow even bigger and you get more and more services, a UDDI will not be enough. You will also need to implement a Service Life Cycle Management environment.

Managing a handful of services (say up to 50) can be done manually, without the need for advanced tooling. As the people involved know about the upcoming changes and are involved in bringing the changes into effect, you can manage a whole lot of services. But .. up to a point. There will be a moment in time when you will have to formalize the procedures and support those with tooling. This is where a Service Repository comes in.

What will a Service Repository bring you? A lot! First of all it will give you a tool for implementing Service Life Cycle Management, as it will support discovery (through visibility), impact and depency analysis, monitor compliance (assets, policies & standards), collect metadata, automate lifecycle staging, etc, etc. What I really like about a repository is that it will become the central focus of development, test and deployment. In the repository you will find all the information you will need, from service contracts, SLA’s, documentation, dependencies, tests, policies, service usage (requires closed loop data gathering). Your repository will be the single point of truth in your SOA!

Unfortunately there’s not many repositories around. According to Forrester (The Forrester Wave: SOA Service Life-Cycle Management, Q1 2008) there are just 3 good repositories available at the moment: Oracle Enterprise Repository (rebranded from BEA AquaLogic Enterprise Repository), Software AG’s CentraSite Governance Edition and LogicLibrary’s Logidex.

Oracle Enterprise Repository, formerly known as BEA ALER, will be rebranded and integrated into the Oracle SOA Suite as well as certified against the Oracle BPA Suite. This is one of the reasons I’m very happy with the BEA acquisition by Oracle. OER is by far the most sophisticated repository around and will be making my life a lot easier!

Oracle vs Open Source

2008-07-08T09:49:00.002+02:00

I work a lot with Java developers and software architects. Often there's a discussion in the projects about which software to use for what purpose. Increasingly i find people to be very rigid about this choice: it's all OSS or nothing. In a way it reminds me of the old anti-Microsoft atmosphere.

Therefore I'd like to start a poll, to find out if this issue is alive or that it's just me :p.

Do me a favour and fill in the poll on the side?

So, is WSDL the standard for service contracts?

2008-07-07T13:17:00.001+02:00

A Service Oriented Architecture is based upon the use of services (hence service orientation). It's implementation is based upon standards. One of which is the way services interact, namely by messages (XML messages usually).

Standardization is the most important thing for SOA to become a working concept. XML has become the de facto standard for the definition of messages between services. That's fine, but in my opinion we are missing one critical item. THERE IS NO STANDARD FOR SERVICE CONTRACTS.

EH??? Some people will argue that WSDL is THE standard for the service contract, but that's not quite true. A WSDL is an implementation of a service contract, specifically for web services. However, a service does NOT have to be a webservice!

Let's approach it theoretically. What do I need to know to access and use a service? I need several different things at different times. When I need a service to fulfill a specific task, I need to know what services are available, what their service consist of, what data it uses (in) and delivers (out), depencies (if any), security constraints and QoS.
At runtime I need to know where it is located, how it is accessible, what credentials to use, how it is secured, etc.
To manage it I need to know: where it is located, its resource requirements, what technical depencies it has (services it uses, but also db access), what security profile is active, SLA requirements (availability), etc.

All we have now is a WSDL, being the Web Service Definition Language, which is used for webservices at runtime. It does NOT have all the information that is listed above.

So, to answer my own question: Sorry to say, but NO, WSDL is not the standard we need!

Oracle & BEA: A Match Made in Heaven?

2008-07-01T20:15:00.006+02:00

Today Charles Phillips and Thomas Kurian disclosed Oracle's strategy concerning the integration of BEA and Oracle. Lots of changes, though most are not surprising. Oracle's vision towards SOA has been clear for the last few years, and I look upon this merger as one of the highlights. I am surprised that there's relatively little overlap between BEA and Oracle's offering. In fact, it is an example of realworld synergy, leading to a best-of-breed SOA technology stack. In that sense I fully agree with the closing words of Thomas Kurian:this is the best middleware suite in the industry. So, the answer is yes: it's a Match Made in Heaven!

Some highlights
Not surprisingly, Oracle has integrated WebLogic into both JDeveloper and the Application Server, replacing OC4J. This will be good for performance and stability. But, it gets better. Oracle has announced Oracle Service Bus, in fact replacing ESB with AquaLogic Servicebus (ALSB). Customers will have several years to decide to stay on current installation of ESB or ALSB, and migrating later. JRockit, the world's leading VM will be the standard VM for Oracle, Coherence will be the central component for dehydration. So we get zero latency in the VM and unparallelled performance for the dehydration store. I feel like I'm in a candy store!

Oracle Service Bus
Thomas presented a roadmap towards Oracle Service Bus. This does not mean that the ESB and the Aqualogic Service Bus are discontinued, but they will converge towards the OSB. No forced migration here, but the OSB will consist of the strong points of both ESB and AL-SB.

The Oracle SOA offering will look like:
Oracle BPA Suite
Another interesting development is the integration of BPM Studio into the BPA Suite. It's fair to state that the Oracle BPA Suite, in conjunction with the Oracle SOA Suite (BPEL PM in particular) is the world's best-of-breed for procesmanagement, -design and execution. The added functionality from BPM Studio will create a comprehensive set of tools to support both process architecture, design and execution in a fully supported roundtrip engineering environment. This will also mean that from the new version onward, XPDL will be supported.
Integrating all the tools will take some time, but in the end we will have the BPA Suite for architectures and high level proces modeling, the BPM Suite for building processes in XPDL/BPMN and BPEL to execute them. The roundtrip engineering will be between BPM Suite and BPEL. Oracle aims to deliver a web-based Proces Designer, although the details about this are not yet fully clear.

The Road to 11g
So, when will 11g be available? Well, JDeveloper 11g will probably go GA towards the end of the year (might even be launched during OOW 2008). Fusion Middleware will take some longer, due to the ongoing integration. I think it's worth waiting for. The newly added components JRockit, Coherence and BPM Studio will make the whole Suite a lot better. Besides, it also gives Oracle time to finish WebCenter and WebCenter Spaces, products that will revolutionize the way we build web apps.

IDE
And the winner is ....... JDeveloper. Yes, JDeveloper will continue to be the central IDE for Oracle. All the components in the stack will have a plugin in JDeveloper to increase production and quality. Of course, Eclipse will still be supported, but not for BPA and BPEL.

If you want to read more about Oracle's strategy, go to http:www.oracle.com/goto/july1. The webcast will be made available soon too.

SOA Components

2008-06-30T12:28:00.004+02:00

What do you really need in an SOA technology stack? To answer that question, let's first have a look as to what SOA really is. SOA is an architectural style, based on service orientation. Ok, so what's architecture? If you Google that term, you'll find 1 gazillion different definitions. I like to look upon architecture as the structure a company gives to its vision. It addresses all the choices (trade-offs) an organization has to make regarding its structure, the way it is organized, how it addresses customers, suppliers, partners and personnel, how and where resources are used (one of them being IT) ... and a lot more. So it's not even about IT, even though a lot of people think so.

If you start working with architecture and especially SOA, sooner or later you will apply it to IT as well. From that point of view, let's see what components you need. Service orientation, as it name says, is based on the use of (business) services.

Let's start at the bottom: services usually make use of data, which will be stored in a database (usually). Next, services exchange data thru messages, so you need something to transport your messages. As different services use different messages (alas!), you will also need a component to transform messages from one format to another. Finally you will need a platform to support a user interface to which the service can be exposed, even though this really is outside the SOA stack. So, basically what you need for an SOA platform is:
- database
- routing & transformation (ie a 'bus')
- UI platform

However, there's more to it. Life isn't that simple anymore. One of the strengths of an SOA is the ability to combine services into a new service: orchestration. So we want to add an orchestration engine to the stack. What else do we need/want? Well, building an SOA is one thing, managing one is another. As you can imagine, the number of services might get large very soon. So you will need at least two additional components: a sort of repository where you can register your services and a management tool that shows you how your services are doing.

Are we done yet? Nope! Managing services is more then just taking care of availability, we also have to take care of security. Who is allowed to use what service? To maintain the reusability of a service, we want a non-intrusive form of security, as a separate layer.

Ok, let's step back and see what we have now:
- database
- routing & transformation (ie a 'bus')
- UI platform
- orchestration engine
- service repository and registry
- management tools
- non-intrusive security

To top it all off, I would like to add a rules engine, to be able to reuse business logic in services, orchestrations, UI, routing and transformations.

Luckily you don't need the whole stack when you start out with SOA, especially when you start small and only within your company. But as SOA is especially very good at organizing processes within your company but also between companies, it will grow fast.

Unfortunately there are not many vendors around who can offer you the full stack. For each and every component I've mentioned there are commercial best-of-breeds and Open Source alternatives. However, integrating tools from just a single vendor can be a challenge, and integrating a lot of best-of-breed tools from different vendors and OSS can be a nightmare (and usually is :p). This is one of the reasons I am very enthusiastic about the Oracle SOA Suite, as it delivers a well integrated suite of components. Most of these are even best-of-breeds, especially the Oracle BPEL Process Manager. I think the integration with BEA will make the Oracle stack even more complete and robust.

Also good to see is that there are a number of OSS projects like GlassFish, who are delivering good SOA components. Unfortunately there are not enough standards yet to overcome the integration challenges. Enough work to do!

How hard is it?

2008-06-30T12:20:00.001+02:00

As you can see, I'm quite an evangelizer for SOA. The main reason being that it delivers (in my humble opinion) the alignment between IT and Business. Delivering business services (recognizable and usable services for the business community) is what IT is all about. However, whenever I get into discussions with IT personnel, I usually meet a lot of (initial) resistance. Of course, change is stressfull and can even be threatening, I understand that. But what I really don't understand, from a technical point of view, is: isn't it great to have all those new technological challenges out there? Isn't it marvellous to see you can build services so easily and building return on investment?

Service discovery, part 3: Meet in the Middle

2008-05-05T16:35:00.002+02:00

Finally I found some time to finish this series. My apologies to those who've been waiting for part 3, but ..... here it is.

I have outlined two different approaches so far: bottom-up and top-down. As is usually the case, both have merits and drawbacks. In fact, the real solution is to do both, and not just one. That's what I call 'Meet in the Middle'. The top-down approach is very important to achieve the business goals, where the bottom-up approach is just as important to utilize the investments made in the current systems for the last few years. By combining these two approaches you get the best of both worlds: integration with the current application infrastructure (in a service-oriented way) and preparedness for more process driven functionality which supports the business needs.

But (of course there's a but) there's a little more to it. Matching the two approaches is not as easy as it sounds. There will be quite a gap between the business services as described in the top-down approach and the application services discovered in the application infrastructure. The trick is now, to take a business service and analyse which of the application services youi need to compose a service that will bridge the gap. This composite service can be multi-layered, and usually is.

One of the paradigms we use in grouping and discovering services is domain driven design (see Eric Evans marvellous work). Stated shortly: we define business domains (top-down here) consisting of delimited data and corresponding services (multi-level services: from data manipulation to business level services). This domain (and therefore its data) can only be accessed through the services. These are the building blocks for the future business functionality.

The attentive reader will have seen the problems looming on the horizon. The main issue is, that the domains we have recognized will be spread over the existing application infrastructure. So, how are you going to delimit the domains? That's the hardest part. In a greenfield it's easy, because you can define and build the domains. In real life situations you will usually build a part of the domain and integrate it with the existing applications. So you will need composite domain services which will present a unified view towards its consumers, and will integrate the different data sources on the back-end.

How hard this is, usually depends on the openness of the systems involved. Even though a lot of systems are walled of, there are usually several strategies you can employ. This goes from the use of service adapters, wrappers, up to direct database calls and file interfaces. You'd be surprised how far you can get.

Service Discovery, part 2: Top-down

2008-02-11T13:49:00.001+01:00

Time for part II. In Part I I explained that the Bottom-up approach is valid in itself, but it is does have its limitations.

A logical alternative is the Top-down approach. In short: this approach looks at the processes in the business, decomposes them into subproces/tasks/functions until 'logical units of work' are defined, which we call services.

Is that all there is to it? In principle: yes. However, there's more to it than meets the eye. A pure BPM (Business Process Management) approach, for example as advocated by IDS Scheer, will deliver a 4-layered process structure for the entire organization. Since the introduction of the Oracle BPA Suite you are then able to convert the lowest level of process to an executable process (BPEL) in which individual tasks become service-operations.

The decomposition of the process in itself is not enough to discover services, because you will have to convert the 'candidate-service' into a real-world executable and discoverable service. This means you will have to compare the functionality of the service with the available infrastructure (which consists of more services, hopefully). Quite often, the service functionality is delimited by specific functionality not being available. For example, suppose you run a very specific proprietary financial system where you cannot interact with either the business logic or the data store. If this is the ultimate source that you will have to use, the chance of being able to fulfill the service functionality needs are quite slim.

Does this invalidate this approach? Of course not. I find this approach working very well in those areas where SOA is used to add new functionality to an existing infrastructure or in a complete greenfield situation (which are quite rare). What this approach does for you is that it's easy to involve the business into designing the new services. The defined services are very recognizable for the business, but not always easy to implement.

Using tools to convert services that are discovered during the process analysis, means that the process analysis has to be done very thoroughly. And, even more important, the process analyst should have knowledge of the limitations of the execution platform. Otherwise you'll end up with services and orchestrations that work rightly on paper, but will never execute in reality. How to address that will be explained in part III.

Oracle ACE Director

2008-01-08T08:57:00.001+01:00

I'm very proud to announce that I've been awarded the Oracle ACE Director status. I'd like to thank Clemens Utschig and Jürgen Kress for their nomination. If anything, this will intensivy my activities around SOA in general and Oracle SOA in particular. Thanks guys!

Does SOA Deliver?

2008-01-07T09:48:00.000+01:00

I read this very interesting article by Max Pucher. He builds up a quite convincing case about why SOA is, as he calls it, an overhyped buzzword. Every vendor is jumping on the bandwagon, afraid of losing out on this hype.

He does pose some very interesting and helpful considerations, that every SOA architect should keep in mind. Let's not forget that SOA is NOT a silver bullet. SOA is also NOT a technology, but an approach to build architectures which are more tailored to the needs of the business (which sometimes includes educating the business).

BPA on the go

2007-12-06T16:04:00.000+01:00

Today I've finished a 4-day course on ARIS. For those of you who don't know ARIS, it's a Business Process Modelling (BPM) tool from IDS Scheer. You might wonder why I'd follow such a course? For several reasons, really. One is that BPM is a part of my daily work. Most architectural studies I do, also address the organization and especially the processes within. Another reason is that Oracle started to work with IDS Scheer about a year ago, to create the Oracle BPA Suite (Business Process Analysis). This suite is meant to bridge the gap between designing Business Processes (BPM) and developing Executable Business Processes (BPEL).

With the BPA Suite it has become possible to automatically generate BPEL Outlines from BPM processes, modelled as EPC (Enterprise Process Chains). So finally we have integration between business design and IT implementation! The good part is, we already have customers ready to use it, so I can bring my new found expertise into play.

Service Discovery, part 1: Bottom Up

2007-12-06T15:38:00.000+01:00

Discovering services is one of the more challenging parts of designing an SOA. I will examine several strategies and try to assess the merits and dangers of each and every one of them.

One of the easiest (but not necessarily the best) approaches is: bottom-up. Using this approach one analyses the existing application infrastucture for all available functions that can be exposed (in any way) to the environment. For example: flat files, database procedures, external interfaces, callable forms and even database tables & views. For each function you need to determine whether or not the exposed functionality is independent of other functionality. Any dependency means that there's additional functionality needed to be able to freely expose this function.

This approach will give you a large number of available functions which you can expose as services. However, there's a catch: are these the services my business needs? The answer is: No. These are the services which can be made available from the current applications within the company. This approach will NOT tell you anything about their usability.

The good thing is: it does give you a feel for the needs of the company. These applications are not here by accident.

There's one dangerous aspect to this approach. It is a technical approach to the use and discovery of services, but it is NOT SOA! Stay tuned, I will get back to this!

Off to a New Start: Changes Ahead!

2007-12-04T20:57:00.001+01:00

Doh, never thought it would be so hard to keep a Blog. Day to day business takes up so much time I never seem to get around to post new stuff. But, last night I gave a presentation to my co-workers about the things we're going to do around SOA, that I got triggered again to write some more.

What was it that triggered me? For that you need to understand the environment I work in. The company I work for, Inter Access, has a business unit Oracle Technology, where I am an architect. Within that business unit, we have 3 lines of business: Technology (mainly SOA projects), EBS (Oracle E-business Suite, focussing on Healthcare) and Managed Services. During my talk last night I realized that this will change in the next few years. Why? Well, I've been to Oracle's Open World in San Francisco a couple of weeks ago and I've been to several sessions about Application Integration Architecture (AIA). AIA is supposed to be 'SOA out of the Box'. It's a set of predefined business processes (industry reference models) linked with a canonical data model to integrate all the different kinds of ERP solutions that Oracle offers. I expect that we will be able to extend AIA to also include other existing applications, as well as custom made applications.

That will mean that AIA will become the heart of the solutions we are going to offer. This will effectively lead to our separate skills on Fusion Middleware and Fusion Applications will merge. Is this a good thing? In my opinion it is, as we will be able to support a larger audience. It will be a lot easier to implement an SOA when you have a lot of standard processes and default interfaces ready on the shelf.

We'll see.... it will take some time!

Business driven?

2007-09-10T16:08:00.000+02:00

I've been attending several seminars on SOA lately, and all presenters agree on one issue: SOA should be Business Driven. I really do agree with that, but it also made me wonder: why is it that it's mostly the IT vendors who are pushing SOA?

The principle of SOA can be stated as: separation of concerns. In my mind, that's the central theme of SOA. Separation of concern isn't anything new to IT. Ever since I started in the early 80s, we've been trying to solve problems by decomposing them into smaller pieces where each piece addresses a distinct part of the problem. Service orientation, which is the means of SOA, defines the characteristics of these pieces, like loose coupling, autonomy and statelessness. So it is no real surprise to see the IT vendors jumping to it.

But SOA is more than just the technological approach. The separation of concern can also be applied to the business. Look at Business Process Management for instance. One of the techniques that's used here is Process Analysis, which consists mostly of decomposing processes into smaller, clearly defined sub-processes.

The one thing that differs in SOA from the 'old-skool' approach of decomposition is that it also adds the concept of Orchestration. Orchestration is the means by which business functionality (whether or not automated) is structured and executed. And isn't that where the business value comes from .... ?

Why Service Orientation ?

2007-09-01T16:00:00.000+02:00

Service orientation is rapidly becoming mainstream. What makes it so valuable? Is it just another new hype that will blow over soon? SOA is supposed to be business driven, so how come it's being pushed so hard by IT vendors? Are there any people out there doing SOA, or is it just talk? Means using webservices you have an SOA? Where does technology comes in? What's the best technology for SOA? Is there a methodology for SOA?

I will be posting my thoughts, but more important: my experiences, upon these questions in this blog. Not that I have all the answers, though. It will be interesting to see who's got the answers, or how we go out to get them!