The Enterprise SOA Manifesto

Recently, the SOA camp has entered into a period of self-reflection and evaluation. Through this endeavor, the camps are dividing. There are those who feel that SOA aligns more closely with the concepts of enterprise architecture and there are those who feel that success can only be driven by more agile, quick hit wins. The optimal answer is likely somewhere in the middle... an approach that blends the best of EA with the best of Agile.

In the vein of the "Agile Manifesto", I'll post my position and invite my colleagues to offer their ideas.

First, it is worth defining Enterprise SOA. The definition that has evolved from working with our customers is,

Enterprise SOA is:

• An Enterprise IT Strategy that encompasses a set of business, process, organizational, governance and technical methods.


• It enables business agility through the use of loosely coupled services that are used as building blocks to develop composite applications that can be reused and recombined to address changing business priorities.
  

Complementing this definition is a set of supporting philosophies. Note that these philosophies are about Enterprise SOA, not just 'service oriented architecture', which has already been covered.

The Enterprise SOA Manifesto

1. Communities over Silos
   Enterprise Architecture focuses on creating portfolios of integrated software to fulfill a business mission. Although application silo's can often be created more quickly, the long term process of integrating silo's of logic, data and policy create spaghetti architecture, demoralize teams, stagnate innovation and increase long term maintenance costs. Communities (or application portfolio domains) should adhere to enterprise standards while each zone tailors the localized rules and regulations.



2. Balanced Planning over The Extremes
   Enterprise SOA attempts to balance 'planning' versus the extremes (too little/too much). The popularity of the Agile movement was largely a knee-jerk reaction to the frustrations with "waterfall planning". Enterprise SOA blends long term planning with tight iterations. Think, "planning in the large, agility in the small".



3. Governed Delivery over Ad-hoc Delivery
   The enterprise must prioritize the needs of the many over the needs of the few. Applications must be architected to fit into an ecosystem of applications. This will require adherence to guidelines and policies on technical standards and software processes, employed to protect the long term interests of the community.



4. Sharing and Reuse over Building from Scratch
   Portfolios of applications will have many common functional requirements. An implicit non-functional requirement in Enterprise SOA is to design for sharing and reuse where appropriate.



5. Business Priorities over the Enterprise SOA Manifesto
   I.T. systems are either a reflection of the business today or a projection of where the business is heading tomorrow. The I.T. approach must not become a religious battle fought at the expense of the business. On occasion, it will be in the best interest of the business to violate the principles of the Enterprise SOA Manifesto for the purpose of 'doing the right thing' for the business. Appropriate planning, governance and leadership should make this the exception, not the rule.
   

JackBe Closes $9.5 Million in Growth Funding

JackBe, a leading provider of mashup software, has closed on a new round of funding. This is a Series C round and should provide them the resources to continue building out their platform and sales engine.

I had the opportunity to speak with JackBe CEO, Luis Derechin, earlier this week. First, I have to publicly congratulate him on running (and finishing) his first marathon! I'm sure there are plenty of parallels to be drawn with running his business...

A couple interesting topics were discussed in our meeting. The first was the inevitable maturation of the market from widgets to platforms. Like many RIA vendors, JackBe initially focused on AJAX widgets and presentation layer accessories. Their business has grown into server-side mashup engines capable of integrating data from a variety of sources (databases, legacy systems, Web Services, etc.) and transforming the transports, payloads and data into easy-to-consume formats for Web 2.0 developers.

I believe that the mashup layer will quickly become an essential element in User Experience Reference Architectures (UE-RA's). The focus of older portal technologies is 'on-the glass integration'. Here, disparate data sources are pulled together as a set of portlets (or windows). Conversely, the Mashup Layer focuses on what I've been calling 'before-the-glass integration'. This enables new functionality to be inserted between consumers and providers (think clients and services), aimed at providing mediation and transformation services 'just in time' based on the context of the consumption.

Because this software is positioned squarely as a presentation layer-to-service go-between, it has the unique advantage of also enabling wide scale governance through policy visibility and control mechanisms. Think of it this way. Today we have 'mash-ups' tomorrow we'll have 'crash-ups'. Service interfaces will change, as will formats and protocols. This places new challenges on software configuration management and the overall governability of the Web 2.0 solutions.

We saw a similar trend in the SOA world where vendors who provided intermediaries had to make a decision. Are you the source of governance policies or the target? This is a fundamental question that each vendor will have to resolve. Said another way, will your tagline be "Mashup Governance" or "Mashup Enablement". That said, a key attribute of Mashup Enablement will be providing Policy Enforcement Points (PEP's), fed by governance vendors.

Over the last 5 years, many of us SOA-guys have put extensive thought into every aspect of the 'service provider' aspect, while neglecting the 'service consumer'. However, an acknowledgement that more advanced UE-RA's are needed will hopefully draw out other Enterprise Architects to surface a more in-depth discussion on this very important topic.

Quit the Clowning

Joe McKendrick is confused. In his latest blog post, "Enterprise SOA Concept Falls out of Favor", he implies that 'enterprise SOA is failing' which couldn't be farther from the truth. I believe that his bad information comes from people who don't know SOA, don't do SOA and in some cases, caused the mess that SOA cleans up.

In my position, I have visibility into HUNDREDS of SOA programs. Many are active clients - many are past - many are just companies that I've had the privilege to speak with. And through my partners, (the leading SOA infrastructure companies), I have excellent visibility into their historical sales and future demand pipeline. Between these vehicles, I am able to see commitments related to SOA strategy, training, governance, architecture, infrastructure, organizational design, change management, packaged application enablement, service integration and composite application development. That said, I feel like I'm in an excellent position to comment on what is really going on with regard to SOA.

First, SOA is nowhere near the mythical Gartner beast called, "The Trough of Disillusionment". And I think it's funny that people keep trying to force us in that phase so that we can then say we've moved past it. Nope. We're not even close.

Second, the idiots that are running around yelling "guerrilla SOA" have to be put in their place. Many of these individuals are the ones responsible for silo-oriented thinking in the first place. They proposed small (agile) projects where we captured just enough requirements to begin coding and releasing. Guess what? This style of development doesn't jive with the concept of shared services. It is the cause of the problem, not the solution.

We love to compare EA to city planning (plotting out neighborhoods, identifying common infrastructure, etc.) Yes - this requires using your brain and creating a plan. Enterprise SOA involves long term planning coupled with short term results.

Don't get me wrong - I'm a fan of 'controlled agile'. The rules of agile by themselves are so incredibly destructive to large organizations that they have done immense harm. Shouting 'agile manifesto' at people who have to build real 'application communities' is a non-starter.

I am proud of the companies that I work with who take the time to think about what SOA means to them. They plan their community. They consider the common infrastructure. They create policies and rules for their citizens. They identify practices to build the structures (reference architectures). And once they've figured out how to build a community they go do it. They don't do it for the entire enterprise all at once - that isn't what 'enterprise SOA' means. Instead, they partition their enterprise into a set of communities and attack them, often in parallel.

Let's be clear. Enterprise SOA is, by far, in the strongest position it has EVER been in. The jokers who feed columnists bad information need to go away. But this won't happen on its own - it requires the columnist to call them the clowns that they are.

Consolidation in the Software Industry

In June of 2003 I noted that, according to Yahoo Financials, Microsoft was larger than the next 40 largest software/service companies:

Quest Software, Inc. + Tibco Software, Inc. + Documentum, Inc. + Take-Two Interactive + Activision, Inc.+ Hyperion Solutions Corp. + Sybase, Inc. + Macromedia, Inc. + RealNetworks, Inc. + Business Objects + Cognizant Technology Sol. + Red Hat, Inc. + Satyam Computer + J.D. Edwards & Company + Autodesk, Inc. + CSK Corporation + Network Associates, Inc. + Konami Corporation + Compuware Corporation + Trend Micro Incorporated + Cognos Incorporated + Mercury Interactive Corp. + VeriSign, Inc. + Citrix Systems, Inc. + Cadence Design Systems + BMC Software, Inc. + Amdocs Limited + BEA Systems, Inc.+ Synopsys, Inc. + Siebel Systems, Inc. + Check Point Software Tech + PeopleSoft, Inc. + Infosys Technologies Ltd.+ Symantec Corporation + Adobe Systems Incorporate + Intuit Inc. + Electronic Arts Inc. + VERITAS Software Corp. + Computer Associates + SAP AG + Oracle Corporation

As I scanned this list, it occurred to me how many of them have been acquired...

1. Documentum --> EMC
   
2. Veritas --> Symantec
   
3. Hyperion Solutions --> Oracle
   
4. Macromedia --> Adobe
   
5. Business Objects --> SAP
   
6. J.D. Edwards --> PeopleSoft / Oracle
   
7. Network Associates --> McAffee
   
8. Mercury Interactive --> HP
   
9. BEA Systems --> ??TBD - Oracle??
   
10. Siebel --> Oracle
    
11. PeopleSoft --> Oracle
    

It is interesting to look at the list of companies that didn't get acquired. Let's remove the non-enterprise I.T. companies (entertainment software, semiconductor, consulting). The odd balls include: Sybase, Tibco, Cognos, Adobe, Citrix, BMC, Red Hat and Computer Associates. In my opinion, these companies need to find a home - and soon.

Companies that have had significant growth since my last post include:
VMware, Inc. and salesforce.com, however you should note that their PE's are 269xTTM and 1,245xTTM - and in my humble opinion - this is just a tad bit inflated! At some point these companies are going to have to increase their top line numbers. Regardless, we've entered into a period of massive consolidation where few new companies can truly shake the market.

SAP TechEd - We Have SOA (for real this time)

I've been attending the SAP TechEd in Las Vegas conference for the last few days. Overall, I'm impressed with the reality of the story that SAP has put together. In the past, SAP had a SOA-by-PowerPoint story. Today - they have real products and guidance.

At the core of the SAP SOA story is the Enterprise Service Repository (ESR). It is actually a combination of both registry and repository. The registry is a UDDI 3.0 implementation and has been tested to integrate with other registries such as Systinet. But the bulk of the work is in their repository. Unlike other commercial repositories, the first thing to notice is that SAP's is pre-populated (full, not empty). It contains gobs of information on global data types, schemas, wsdl's and similar artifacts relating to the SAP modules.

Both the registry and the repository are designed to embrace service metadata that is housed by SAP as well as service information that might be in other platforms (IBM, Microsoft, etc.) And although the registry will do metadata interchange with other registries, we're not so lucky with the repository. Apparently the current version is designed to be a single instance across the entire enterprise. Of course, this isn't realistic in most organizations due to mergers & acquisitions, etc.

SAP has also customized the TOGAF enterprise architecture to meet the needs of large customers. I was hoping to learn more on this but the speaker failed to show up (bummer). However, it was clear that SAP is relying on IDS Sheer to supply EA modeling tools.

I felt like SAP really got it. Unfortunately, I felt like most of the people at the conference didn't. The gap between vendor and customer has grown significantly. Many of the SAP customers have avoided technology topics, favoring black box implementations of large monolithic applications. SAP is introducing a fundamental change that will take time to sink in. They are doing a great job of evangelizing and providing community guidance but this is a huge leap for many of the 'ABAP' programmers of today.

A Watched Pot Never Boils - SOA Style

Literally - I was trying to boil some water today. While staring at it, I began thinking, "My God this is taking forever!". A Watched Pot Never Boils. Sadly, too many of my life experiences end up relating to SOA...

Some journalists and bloggers have been critical to the success of SOA. Personally, I view it as a "watched pot". Impatiently staring at an SOA effort and constantly asking, "How many services do we have?" or "What's the ROI?" - doesn't really help.

SOA is a long term change to I.T. - it will take time. I've seen HUGE changes take place in some of the world's largest corporations. I've also seen companies with poor I.T. leadership whine about how they failed. Hell, I could have told them that they would have failed at about anything they tried.

I guess what I'm trying to say is that I don't think that the journalists / bloggers should ask EVERYDAY, "are we there yet?" It's annoying and not helpful. And the LOSERS who threw in the towel - well, they're losers.

Budgeting for SOA

Well, it's that time again - I.T. teams around the nation are hunkered down working on their 2008 budgets. All of them are asking the same questions: How much do I need? What will it be spent on? Where can I cut? Unfortunately, I can't answer all of these questions in a blog post, but I can provide some investment categories to consider:

1. SOA Foundation - A good SOA Foundation program is typically a 3-6 month process utilizing internal and external resources. The initiative rolls together a number of common deliverables: SOA Strategy & Roadmap, SOA Methodology Updates, SOA Reference Architecture, Standards Development and SOA Governance Planning. This will typically cost about $200-400k from external resources, and will eat up significant time of 2-4 internal resources.

2. SOA Infrastructure Realization - Over the last several years, software vendors have been perfecting their SOA solutions. Most large organizations evaluate and acquire their software in 4 different stages, often a year apart.

Stage 1 software is typically a combination of Registry, Repository, Mediation and Web Service Management. For an enterprise license this is typically $750k - 1.25, depending on the size of the organization.

Stage 2 is typically a combination of security and integration. This often includes SOA firewalls and other edge devices. For integration, most organizations are looking at ESB's, orchestration engines and service engines / adapters. Plan on $400-800k for Stage 2.

Stage 3 is typically a combination of EA and Advanced Integration. Organizations that don't have EA modeling tools are quickly bringing them in house and are laying the foundation for their process, service and information modeling needs. Advanced Integration is usually a combination of Data as a Service tools (EII, MDM) as well as legacy host integration. Stage 3 can be quite expensive depending on what all you need. If you're starting from scratch, plan on $1 to 2.5 million for all of it.

Stage 4 is primarily focused on using the services. This includes client side platforms (AJAX, Web 2.0, Next-Gen Portal, Composite Application Tools, etc.) as well as the quality tools to verify the services. From a quality perspective, organizations are buying SOA testing platforms as well as asset governance tools. Comparatively, this area is a bit cheaper, plan on $250 to 800k.

3. SOA Governance Team - After the SOA strategy and roadmap have been defined, the next step is to create an organization that moves SOA forward. Activities include: Program Management, Service Portfolio Management, SOA Infrastructure Architect, SOA Infrastructure Administrator, Service Product Management and typically a couple SOA consultants to engage in active projects and review deliverables. Again, this is typically a combination of internal resources and external SOA consultants. Total combined costs are typically in the $.5 to 1 million range. Also, expect this team to provide significant guidance in the selection of the SOA Infrastructure and to mature the documents defined in the SOA Foundation Program.

4. EA Domain Analysis - All of the expenses on SOA planning, infrastructure and governance is wasted if you don't actually DO SERVICES. It surprises me how many organizations forget this point. Domain analysis is typically performed by enterprise architects who have a strong background in process modeling and service design. They pick a domain area (Sales, Supply Chain, etc.) and perform Process Reengineering, Process Modeling, Service Identification, Service Analysis and Composite Application Requirements Gathering. It is common for these activities to be driven by a Global Process Reeningering effort, or by Application Rationalization / Consolidation efforts. These efforts vary significantly in size, but rarely can you analyze an enterprise domain for under $300k. It is typical for an organization to be analyzing multiple domains in parallel. Most EA teams I've met with are already fully utilized, have minimal budget and don't have time to perform this work. Plan on either adding permanent members to the team or bringing in on-demand external resources.

5. SOA Training and Change Management - Unfortunately, we're not born with SOA skills, we must be trained. Managers, analysts, architects, developers, QA professionals and operational support personnel must all be trained in their piece of the solution. Even after training, you'll find that some people didn't make the change (Silo Oriented Architects), and you will need to provide some degree of change management to either get them on the right page or get them out of the way. Most organizations that we're dealing with are sending 100-300 people to training and are sending IT leaders to conferences. I'd plan on $150k to 400k for getting the teams up to speed.

5. SOA Build and Integration Teams - As organizations continue business as usual, they are constantly bringing in new packaged applications as well as building new systems for their business customers. Going forward, these systems will be sent through the SOA Governance Center. In some cases, the Governance team will determine that they shouldn't be services and will pass them through. In other cases, the systems will be required to adhere to the Governance standards. Packaged applications will often require 'service enablement' and 'service oriented integrations'. New systems will require 'SO-analysis, design, construction and testing'. If you think that your offshore teams will be building your first services, well, you're probably wrong. Service design and construction, like anything new, will most likely be done by in-house teams or on-shore development centers. After the art of SOA Build and Integration is turned into more of a repeatable discipline, you should strongly consider moving this to your favorite commodity development center. For planning purposes, I recommend that you first get a non-service orieneted cost using your internal estimating scheme. Then add on an extra 20-35% to turn the the software into hardened, reusable shared services.

---

I've said it before and I'll say it again... SOA Transformations cost big bucks, take years to complete but in the end are worth the investment. I hope that this off-the-cuff analysis is valuable to you. Again, it is impossible for me to provide anyone with accurate advice without knowing their situation. I'm currently running around the country helping large organizations put together their precise 2008 SOA budgets. If you need an extra hand looking at your situation - feel free to reach out to me: e-mail me

Terry White on SOA

I just caught a great comment that Terry White, EDS Fellow, left:

A good approach is to develop a taxonomy that is easily understood by all involved. We recently developed an Enterprise Architecture that was SOA based. It didn’t really look that much different from any other Enterprise Architecture that is modular and effectively layered.

In developing this architecture we involved people from both the IT and business areas of the company. When we first started the context was purely business process – re-engineering and decomposition. We moved them to a services view for looking at the business processes. Then we added the IT “application” details and show how business services would be supported by application services. We decomposed the existing application landscape and re-ordered it to align with the business services.

Business processes have owners, or people responsible to manage them as assets. Applications have owners. Now the move is for services, both business and IT application, to have owners and be managed more like products. One potential problem I noticed is that management tends to count the number of applications supporting their business and work to reduce that number. When we move to services and managing services, there is an automatic increase in things you are managing because before these were all just application functions buried within the application. Aggregating the services can help but it is a bit of a mind shift.

I couldn't agree more. This is very similar the process that we do at MomentumSI. I'm glad to hear that others are having success as well.

Using SOA to Align I.T. with Itself

After much consideration, I've come to the conclusion that SOA is best suited to facilitate "I.T. and I.T. Alignment" (not Business and I.T. Alignment). That is to say, SOA (from an enterprise architecture perspective) is better suited to align internal I.T. efforts with other internal I.T. efforts. This might sound like common sense (and hopefully it is), but SOA is fundamentally about sharing common logic and data while facilitating accurate and complete client side consumptions.

I had a conversation with a gentleman the other day. I'll paraphrase his comments... he asked me to imagine an enterprise without computers or software systems. Instead, it had one Filing Room that people went to when they needed to store or retrieve data.

At the front of the Filing Room were people working the Service Counter to fulfill your requests. In the back of the room were Filing Clerks who kept the filing system organized.

In this model it was assumed that the people in the Filing Room did a good job of organizing their files, as to ensure that when a customer asked for "all customers", they didn't have to go to 5 different Filing Cabinets. It was the responsibility of the Filing Clerk to facilitate Master File Management. The Chief Filing Officer was responsible for making sure that the Filing Cabinets stayed organized and on occasion were reorganized.

The "Service" in SOA is the new filing cabinet. Our SOA Governance Teams will work the front counter taking requests and also verify that they filing clerks do their job correctly. They must ensure that the portfolio of filing cabinets stay organized and avoid duplicate filing systems. And ultimately, the CIO must be held responsible for the state of the Filing Room.

SOA is not a holistic EA framework, but it will provide the taxonomy and organizational structure to become the foundation for a single enterprise system of services.

SOA Consolidation - Updates

I finally updated the SOA consolidation list:
http://www.momentumsi.com/SOA_Acquisitions.html

The clock is ticking on some of the smaller firms...

Software M&A Rumors

I noticed two items of importance this week:
1. The Dow hits an all time high
2. Rumors of VERY LARGE M&A activity in the I.T. space are off the chart

I'll avoid perpetuating any of the rumors that are going around; talk is cheap. What seems evident is that both I.T. buyers and providers seem bullish on the idea of massive consolidation. Our industry has been in constant flux for a long time, exhibiting all the signs of an immature industry.

Despite our strong economy, many of the preeminent brands continue to bleed money while failing to create a differentiated product portfolio. The buying community has rewarded platform commoditization and 'design to standardization'. Cloners and imitators have shown their ability to rapidly re-create innovator's products using open source and competitive pricing models.

Most large ISV's have already executed on a significant number of acquisitions and have had to become experts in integrating their own products. As their ability to refactor, integrate and generally absorb new products grows, so does their desire to capture additional market share through acquisition.

The unanswered question is where is this all leading? I'm confident that we'll see a few models emerge. However, the one that seems obvious is the one-stop shop. One company provides hardware, software infrastructure, packaged applications and professional services. With the growing popularity of SaaS, we can anticipate that multi-tenant hosted application support will also be a key element. The other trend that can not be discounted is the use of offshore resources to provide low cost labor.

I believe that massive consolidation is both inevitable and necessary. Information and technology providers must reach the next level maturity. They must be able to provide end-to-end solutions and take full responsibility.

Service Lead Management

When a customer or prospect inquires about a product or service we call this a "lead". Typically the information given in the initial call is not enough to understand what the customer hoped to achieve. Therefore it is necessary to go back and talk to them about their goals and plans. Today, modern software enables this process. For instance, if you go to the MomentumSI web site, I know you visited within 30 seconds. Via our web marketing automation system (and a network of cookies), I am informed about visits.

A similar system is needed in the SOA world. The SOA registry / repository must be viewed first and foremost as a shopping catalog and should employ modern techniques for capturing visits, searches and users. The service librarian must view all hits to the catalog as "leads". The process of following up on these leads is considered "Service Lead Management".

Here is a sample follow up
- Who are you? What department? What project?
- What kind of service were you looking for? (let me help you look)
- Do you want me to tell you more about Service X? (you found one)
- We don't have Service X. Do you want me to talk to the Service Portfolio Manager about adding one to the Shared Services Group?
- etc.

Remember, when you first get your Service Catalog going, it will be EMPTY. The goal is to find out what people need and to determine if they are good candidates for shared services. Kill "Empty Registry Syndrome"!

Using the Zachmann Framework to Manage Schneider Oriented Architecture

If you haven't seen Bill Zachmann's attempt to throw water on the SOA fire, it is worth a read.

Bill basically says that SOA has been overhyped, is the same-ole architecture and that it is like CORBA, etc. He goes on to say that the new thing is XML and you can already do that in Microsoft .Net. He also states that, " SOA is a matter of good, modular, object-oriented design..." No Bill, SOA is a matter of good Service Oriented Design.

I'm at a loss for words. I can understand his distaste for the amount of attention that SOA is getting. He actually states that he doesn't like: "IT guru firms that peddle high-priced snake oil as "expert advice" and use high-sounding, yet vague and obscure terminology to cloak the utter banality and limited practical value of what -they're saying." My God Bill - have you looked in the mirror? At least we didn't name it after ourselves!!!

This article, written in the Microsoft Redmond Journal, is the biggest bunch of shit I've seen written since Nicholas Carr put pen to paper.

Advanced SOA Governance - Negotiation

As we split our monolithic applications into right-sized units of work (services) and pull them back together again with composite applications, we realize that the sharing of assets introduces benefit (subsidized costs, consistent logic, single source of the truth, etc.) but it also introduces new pain, namely change management.

In the last few years, most large enterprises have moved to some variation of the three tiered architecture. Here, the system typically has 4 or 5 layers or tiers:


The stateful interactions that monitor the business processes have been factored out of the business logic, and distributed joins (EII) have become first-order citizens of the architecture.

More advanced Service Oriented Enterprises have adopted this tiering model as the heart of their technical service taxonomy.



This diagram depicts the relationship that services have with the composite applications that consume them. As you can see, some services are required by more than one composite application. This is a core feature of SOA - the sharing of assets across the enterprise to solutions that need them.

Modern SOA Governance practices focus on passing the 'service baton' from one group to the next while verifying that best practices are adhered to and that the baton remains in motion. This movement throughout the software life cycle is managed according to new service/operation requests as well as for change requests. The Service Life Cycle Governance and the Evolution Governance become the primary framework for monitoring change.



The needs (features and functions) of a service will vary by the consuming application. The teams that deal with each software life cycle will need to be aware of the multiple masters (or owners) that they must serve. And on occasion, we can expect that the masters will have disagreements. They will disagree on what it should do, who pays for it, when it should be rolled out, etc. These questions are at the heart of SOA Governance!



As you can see, consuming applications need to approve, prioritize, fund, plan and communicate the changes to shared services. The Service Oriented Enterprise realizes that the services are an enterprise asset, shared for the benefit of the organization. And each business or process owner will, and should, argue for their needs. Ultimately, the group needs to come together to resolve the differences.

SOA will force more occasions where departments and business units will need to find a common ground. I.T. shops have had the need to negotiate for shared infrastructure in the past. If you move forward with SOA, this activity increases significantly. There are no magic answers to SOA Governance. My only recommendation is to make sure that the tools, processes, roles and committees are in place to make the negotiation process as efficient as possible. Said another way, a competitive advantage for the Service Oriented Enterprise is the ability to efficiently negotiate differences and take action.

SOA is an I.T. Strategy

I've read some stuff lately where people have called SOA a "business strategy". I'll go on a limb and estimate that in 99.99% of the cases where SOA is being used, it isn't a business strategy but rather an I.T. strategy. And you know what? That's ok. There is absolutely nothing wrong with acknowledging that SOA is about making I.T. better. The better I.T. works, the better it can serve the business.

The notion that everything I.T. does is "about the business" is just a little bit silly. Sure, at some secondary or tertiary level it's all about the shareholder. But let's get real... buying rackmount servers is about a more efficient I.T.; standardizing on one or two operating systems is about creating a more efficient I.T - - and sharing common application logic and data across the enterprise is about creating a more efficient I.T.

SOA is first and foremost an I.T. strategy. If you need money from the business to fund SOA, then talk to them using words they understand. But don't kid yourself - this is an I.T. problem and you have to clean up your own mess. If you've created a silo-oriented, sphaghetti integrated, inefficient architecture that slows down your ability to server your internal customers then it's your problem to clean up the mess. Whatever you do, do not insult the business by telling them that you've got a new business strategy called SOA.

Did the SOA Community Fail You?

Yesterday, I had an interesting discussion with another SOA veteran on the subject of barriers to successful SOA. The conversation led to a fairly obvious conclusion that several parties aren't pulling their weight. The parties we discussed included: the Venture Capital Community, Standards Bodies, I.T. Analysts, the Press, the Academics, SOA Infrastructure Vendors, Consulting and Training Vendors, Packaged Application Vendors and Design Tooling Vendors. Here is my personal view on their success:



Venture Capital Community
The VC's did their job. They invested early; many of the investments were made in the days following 9-11 which was a tough time for anyone to invest, but they did it. And when they did, they funded deep - but not too deep. It wasn't crazy dot-com investment. As their investments grew up, they did a good job helping them to find parent companies. On a scale of 1 - 10 stars, I'll give the VC a solid 10 on SOA.

Standards Bodies
Many will argue that a key enabler of SOA was the creation of Web Service standards. The creation of SOAP, WSDL, UDDI and the WS-* stack has been both a blessing and a curse. Often the concepts of SOA are tied to implementations and the limitations of those implementations are then attached to SOA. This is a tough one for me to score. IMHO, the real standards work was done behind closed doors at IBM and Microsoft and then handed to the standard bodies for cleaning and revising. In some cases the standards bodies did more harm then good. Overall, I'll give them 6 stars.

I.T. Analysts
The I.T. analysts got on the SOA bandwagon pretty early. Overall they did a good job of covering an extremely broad subject. Boutiques like ZapThink, Burton Group, CBDI and Macehiter Ward-Dutton showed precision analysis and deep insight. While Gartner, Forrester and IDC all did a sincere job covering the space, their insights often lagged the blogs and mainstream media. I'm giving the I.T. Analysts 8 stars.

The Press
The press has been overly generous on the topic of SOA. Most of the large publications have a regular column on SOA and even an SOA blog. Some, like Infoworld, have even held SOA conferences. I think that the reporting has been fair, but often light. Maybe I'm too old, but I remember back when magazines would do more lab testing and comparing products. This just doesn't seem to happen very often anymore. Now, we vote on 'hot products'; how lame is that? If the press would bring back the labs (or more case studies), I'd give them a solid 10. But lacking this I have to give them a 7.

The Academics
I have no expectations for the academics to do anything. And in my opinion they haven't. They're just where I like them. Hence, they get a solid 10.

SOA Infrastructure Vendors
Infrastructure such as: XML Appliances, ESB's, Orchestration Engines, Smart Intermediaries, Web Service Management/Monitoring, Registries and Repositories became mainstream in the SOA era. For the most part, I was impressed with the ingenuity that the startup's demonstrated in forging their products. And even some of the big guys like IBM showed that they have game. I'd love to give these guys a solid 10 but I can't. It's because of the big lie. These vendors have told customers that their products work with each other and for the most part they don't. HP/Systinet promoted the Governance Interoperability Framework as a solution, but in my personal opinion, they completely failed to deliver. And the rest of the vendors have sat on their lazy asses acting like the problem doesn't exist. I'm giving them 7 stars. How ironic is it that SOA vendors lose 3 stars for having a lack of interoperability?

Consulting and Training Vendors
As a consulting and training vendor - this is a hard one to do. There are a few companies like MomentumSI that I believe are doing a good job of helping organizations make the transition to SOA. The problem is that there are only a few doing a good job and a very large percentage who are not. Consulting companies should be able to quickly deliver SOA best practices, methodology adjustments, job descriptions, project mentoring and deep knowledge on tough subjects like governance and change management. For the most part, the big guys (Accenture, BearingPoint, CSC, EDS, SAIC, WiPro, etc.) are still very immature. There are a few notable exceptions, in my opinion Infosys, CGEY and IBM Global Services are all making sincere attempts to build out their SOA competency, with IBM leading the pack. I'm going to have to give this category 6 stars.

Packaged Application Vendors
First, as long as this category is called 'packaged apps' and not 'packaged services' I'm probably going to bash it. I remember back when we used to talk about the end of 'big bang implementations'. This was the idea that you wouldn't spend 3 years installing and configuring a huge monolithic application like SAP. Instead, you'd do it one service at a time. Ha! The packaged app companies were quick to get on the SOA bandwagon. They realized that this paradigm offered a genuine threat to their model and were quick to say that they embraced it. The question is, "embraced what?" That they'd web service enable BAPI's and hide the ABAP code? You've gotta be joking? And there is a reason why people refer to the Oracle strategy as conFUSION. Without a doubt, the best thing that these guys have done is made promises about SOA indicating to customers that SOA is the future. Yes, because of their marketing efforts I am forced to give this category 2 stars.

Design Tooling Vendors
In every paradigm change, we require new tools to facilitate the new architecture. In past generations we saw wonderful products like Rational Rose, ER/Win and TogetherJ rise to the occasion. I am so disappointed in the lack of progress in this category. The EA modeling vendors have demonstrated 'pompous ignorance' and failed to deliver anything resembling modern SOA tooling. Notational standards for service models remain in their infancy. I firmly believe that the lack of SOA modeling standards and tools will defer the adoption cycle. This category deserves zero stars.

---

Now the burden is on the buyer. Did the SOA Community fail you or will you fail it? It is the responsibility of every buyer to push the community to do a better job. Money talks.

Talking to the Business about SOA

I often recieve the question, "How do you go about talking to the business about SOA?"

Here's my advice. Don’t lump all non-I.T. functions into a single bucket called the business! We must know our audience and create a message that they can relate to.

FINANCE
There are many definitions of SOA, but my favorite is to describe SOA as an 'I.T. economic model focused on cost savings through increased utilization of existing enterprise assets'. First and foremost, shared services should be viewed as a subsidized and highly leveraged asset that lends itself to supply and demand economics.

From a slightly different perspective, we can say, “SOA is an I.T. investment model that allows you to view your enterprise I.T. assets from a portfolio perspective enabling precision investing.” Here, the focus is on the granularity of the investment. By breaking large, monolithic investments into smaller units we have increased our ability to specify and evaluate individual investments.


PROCUREMENT
In the past, it was difficult for I.T. to divide up large systems into smaller units and have each smaller unit sourced individually. In order to do this successfully, you need to be able to specify how they will reintegrate by using common standards. Recently, I.T. has made a significant advance in these standards and we are now able to employ a multi-sourcing strategy for very large systems. In fact, buying large pre-integrated systems that are proprietary is now considered a worst practice.


LEGAL
I.T. has finally learned that we need to specific software requirements using precise terminology. We’re now using “digital contracts” to specify the functionality of the systems that we will build or buy. SOA makes I.T. focus on knocking out the contract before we start doing the work. This will allow us to hold internal and external parties accountable for their work products. We’ve also adopted a model for incorporating Service Level Agreements so that on-going satisfaction can be reviewed.


CORPORATE DEVELOPMENT & GROWTH
Corporate development groups must evaluate the risk and costs of performing acquisitions. Integrating the I.T. systems of acquired companies is generally considered a high risk and a costly endeavor. SOA enables the newly acquired systems to be leveraged in a shorter time frame facilitating business and system integration. In essence, SOA is the I.T. strategy for enabling mergers and acquisitions.

The assumption here is that the I.T. department is identifying ‘master services’ and ‘master data’. Newly acquired I.T. systems are rolled underneath these services.


PROCESS IMPROVEMENT
Most mature SOA programs have adopted a ‘process driven’ approach to business and service analysis. By analyzing business capabilities, processes and activities, SOA services and operations can be discovered. This enables I.T. to promote (and often fund) the use of formal process analysis. In addition, since services have been broken into smaller units of work, we now have greater visibility into each unit. This is a key enabler of business process management and monitoring. Now, process improvement specialists can utilize a ‘process driven, service oriented’ approach to monitoring the business and recommending improvements.


THE P&L OWNERS
At the end of the day, someone owns the P&L. This could be the CEO or a business unit head. Organizational leaders need to hear:
- You have a new I.T. strategy that benefits the entire company.
- The strategy can be successful through incremental investments but it will require an initial out-of-the-gate investment.
- If successful, you’ll be able to significantly reduce I.T. costs without sacrificing results.
- The entire I.T. and software industry is heading down this path, so eventually we’re going to be impacted by this software model. The only question is do we want to invest now to get ahead of the curve.

Unlike previous I.T. paradigms, SOA is not about changing out your hardware or creating a new ubiquitous user interface. For the most part, SOA is invisible to the users. This paradigm is harder to explain than others but if done correctly, it's easier to justify.

Try to avoid lumping all non-I.T. personnel into a unit called, “the business”; it’s not ‘us and them’. Know your audience and what they understand and value. SOA is a complex model that has many advantages. The key is to know which advantage to pull out and when!

Composite Service Analysis

A common scenario that I see is the case where an organization wants to build a new application with the intent of 'finding services' in the implementation of the project. This is common because current I.T. governance favors 'project funding'. This begs the question, what does the process look like for capturing requirements for a silo-oriented application in a service oriented, mashup, process driven world?

First and foremost, we must recognize that we are not in the object oriented world where we captured requirements as if there were only one type of logic. We live in a world of domain specific languages and self describing interfaces. Capturing requirements as if we didn't know that kind of input our downstream partners need is just plain silly (and obsolete).
A simplified view of the new process looks like this:


Today, we leverage our downstream partners (UI, design, architecture) to help drive full and useful requirements. The analyst leverages the UI/Mashup/Composite specialist to create mockups, prototypes and even working user interfaces. The UI's are then presented back to the business users to help validate the requirements. UI's typically are good at conveying a single users role in a larger process but fail to paint the big picture. To bring breadth to the table, the analyst will call on a process and integration specialists to clarify process steps, human workflow and even the flow of information between legacy systems. In most cases, the 'composite services', 'integrations' or 'process logic' calls atomic services. This is where fine grained business rules, calculations and constraints are housed. The anlayst will leverage the skills of a Service Designer to stub out or mock up an interface (WSDL) along with a dummy implementation.

Our goal is to use a combination of rapid prototyping and extreme programming to quickly create releases that can be presented to the business community for feedback. Modern analysis embraces the following key concepts:
1. It acknowledges that some functions will be shared (as services)
2. It embraces various types of logic (presentation, process, data, etc.) and documents those requirements in a manner that makes sense to the people who have to build it
3. It leverages the techniques we've learned in spiral, iterative, RAD and agile methods limit risk and involve the user

Remember - SOA is only a piece of the puzzle.

Service Domain Analysis

Today, most organizations have someone who owns an application. The ownership is often split between two areas. One group will own the budget and control of features while the other group owns engineering and staffing decisions. In the service oriented world this will likely stay the same. One potential difference is the scope of ownership. With the popularity of mashups and composite applications new customer facing systems are being designed around the needs of the user rather than around a specific domain. Many of these systems are 'process oriented' and cross several business domains. This makes it hard to determine who owns it.

In a service oriented enterprise, you have a 'owners' of services and 'owners' of composite applications. The service owners are domain focused (manufacturing, sales, etc.) while the composite application owners will be process (or problem) focused. Service owners focus on creating consistent logic and having 'a single version of the truth'. Composite app/mashup owners work with the end users to understand their day-to-day computing needs. Their job is to deliver the right information at the right time to their users. They leverage the service teams to provide the right information.

Today, many organizations have not split their groups into 'service teams' and 'client teams'. This typically requires a re-org and as I've mentioned before is a SLOW process. In the meantime, I'm suggesting the following process:



In this scenario we have a project analyst who is working with a business user to determine their needs. They will employ a variety of techniques to uncover the requirements. The analyst will often be trained in rapid prototyping (Web 2.0 style) or work with others with that skill set. They will also use Service Oriented Analysis techniques to identify potential services and scope them out. The analyst will document their findings in a form different than legacy silo systems. They will use a combination of Service Cases and Operation Cases. These cases are used to fully describe that portion of the system which would be shared and controlled as a service. The portfolio analyst works with the project analyst to verify that the candidates are valid (reusable, distinct, etc.) and create a placeholder for the service, know as a Slot or a Service Slot.

There are a variety of techniques for performing Service Oriented Analysis. At MomentumSI, we utilize a set of practices known as Harmony™. In the coming months, we'll be discussing these practices in detail.

Starter SOA, Part 3

So far we've discussed a few concepts that organizations can take on without too much pain:
1. Get the project disciplines to talk with the enterprise guys about SOA
2. Create a SOA Steering Committee to start a dialogue between enterprise disciplines

Number 3 is a bit harder for some organizations. It deals with money, or the lack thereof. The heart of the problem is that the funding process in most I.T. organizations still doesn't facilitate the SOA paradigm. In the past, systems were generally funded as a standalone 'application' that had one owner. Going forward, funding will be broken into:
- Shared Services
- Composite Apps / Clients
- Non-Shared Services (services with one consumer/client; temporarily dedicated)
- Shared infrastructure (reg/rep/esb/wsm/etc.)
- Non-Shared Infrastructure (service container, app server, etc.)

Within each of the aforementioned areas, the dollars will be broken down by the various support groups (engineering, operations, etc.)

The SOA Steering Committee must be capable of recommending financial models that support the future paradigm. Most organizations already have some mechanism to make a request for shared investments. The SOA initiatives must work together to create a standard model for joint investment as it deals with services. Initially, the request looks like an exception to the rule but after time shared investment will become the norm and non-shared investments will be critically reviewed.

Starter SOA, Part 2

Many organizations seem to hit the brick wall on SOA when one of four things happen:
1. Their isn't a critical mass of SOA enthusiasts
2. No one will pay for SOA infrastructure
3. No model for funding or governing shared services exists
4. Doing SOA right requires a reorganization of I.T.

Simple SOA is primarily directed at the 4th problem. Most I.T organizations have groups that "own applications" or dare I say, they "own monolithic, tightly coupled applications" - they don't own "shared services" (business or technical). Doing SOA right usually requires a new I.T. reporting structure but this can be a SLOW process.

So, the second concept I'm promoting is the use of an SOA Steering Committee.


As I mentioned in my last post, an early (and easy) step to take is having project teams leveraging the enterprise disciplines. Application architects should be working with their counterparts in EA, etc. Once the project teams begin communicating with the enterprise disciplines, the natural progression is to get all of the enterprise disciplines talking to each other.

The agenda for a typical SOA Steering Committee is:
10 minutes - The status of the SOA Roadmap: what is done and what isn't
20 minutes - A review of any new artifacts or SOA deliverables
20 minutes - A discussion on problems that the project teams are encountering
10 minutes - Recommended changes to the SOA Roadmap

It's a simple meeting that is cross-discipline. The team uses a program plan (SOA Roadmap) to keep a list of initiatives visible. The roadmap remains 'agile' and reacts to findings in the field. A fundamental goal of the team is to address issues that pop up quickly and to take decisions back to their respective teams.

In the beginning the team will likely meet often (every 2-4 weeks) and as the program matures the team will probably meet quarterly or until stabilization is realized.

Starter SOA

Many large organizations are trying to figure out their new processes around SOA. I've been involved in a handful of these efforts. Typically this will involve a cross-discipline team who reviews current processes (RUP, ITIL, COBIT, etc.) to determine what must change in a service oriented world. Although I fully support this model, I've also noticed that this is a LONG and difficult road. This got me thinking... how do you do "starter SOA"?

Of course, I'm less interested in the technology/architecture side of SOA and I'm more interested in the people/process side. We have literally hundreds of modified SDLC processes that depict changes to support SOA. It's great stuff. But I've determined that it is too much to convey the basic idea of what we need to get done. This led me to create a simple picture about what I believe is at the heart of the SOA issue.

The basic idea that I'm suggesting is this:

1. Don't roll out all of the updated disciplines at once - pick a few key disciplines and insert them into the process.

2. Attack three specific areas: Portfolio Mgmt, Enterprise Architecture and Information Management.

Again, this won't solve all of your SOA problems but it will solve many of the problems related to 'Master Service Management' and basic SOA Governance. As this simple process becomes part of the regular work stream you can begin bringing in more and more disciplines like Operations, BPM specialists, Shared Service Groups, etc. My primary warning is don't wait until you have ALL your ducks in a row to get started - it may never happen.

ERS - Empty Registry Syndrome

I've met a number of organizations recently that seem to have a bad case of ERS, or Empty Registry Syndrome. This disease must be taken seriously. It is known to be fatal to service oriented initiatives.

Common causes of ERS include:
- Not having a 'shared services group'
- Not monitoring the project pipeline and identifying service early
- Silo-Oriented requirements gathering
- Buying a bad registry

Luckily, ERS is completely curable. By changing habits an organization can usually treat the disease in 3-9 months. It should be noted that the treatment can be painful and takes real commitment.

Common treatments:
- Give the development organizations goals related to service creation. The SOA initiatives MUST move beyond enterprise architecture and integration centers
- Revise the I.T. governance, funding and portfolio management processes to find services early
- Train business analysts and application designers to 'identify, analyze and design' services; people were not born with this skill
- Verify that your registry doesn't stink. Here's the test: If you search for a service that doesn't exist does it return with:
A) No results found
or
B) No results found, would you like to request a new service?

If the answer is "A" please call your vendor and let them know that their software is spreading a disease known as ERS.

Hattrick Software & Choreography

Steve Ross-Talbert has his fingers in a couple start-ups:
Hattrick Software focuses on Web Service Choreography:
http://www.hattricksoftware.com/

He blogs about some of his reasoning here: http://pi4tech.blogspot.com/

The original WS-CDL specification was less than impressive, however, the concepts were right on. I haven't gone back to revisit the specs but I will. It will take people some time to understand the fundamental 'centralization' problem associated with BPEL. Until then, alternatives will largely be ignored.

Lustratus Research on SOA

I just ran across this site which provieds SOA insight:
http://www.lustratusresearch.com/store/default.aspx

Service Analysis and Design

I've kicked off a new discussion group specific to SOA Analysis and Design. We're just getting things going but please stop by and sign up.

http://tech.groups.yahoo.com/group/SAD4SOA/

The group will not be discussing architecture (mediation, ESB's, SOI, etc.) nor will it be discussing construction (Java, .Net, SCA, etc). The focus is purely on analysis and design. Hope to see you!

The Blog Tag

I've been blog tagged by Paul Brown and Brenda Michaelson. So, here it goes... 5 things you probably didn't know about me:

1. I constantly listen to music - favoring "classic punk rock" and gospel... an odd combination.

2. Before I got fat, I was a long distance runner and a racquetball instructor.

3. Initial funding for MomentumSI came from personal savings and a $40k loan from my mom (10k at a time). "Uhhh... Mom... any chance..."

4. I intend to write at least one more book - most likely on "strategy digitization".

5. I was a double major in Computer Science and Psychology with the intent to pursue artificial intelligence.

I know I'm supposed to tag 5 more people, but it's the holiday season... :-)

New Podcast on SAP and SOA

Scott Campbell, SAP ESA guru, speaks on the state of SAP and SOA in the enterprise:
http://searchsap.techtarget.com/originalContent/0,289142,sid21_gci1234356,00.html


It's an excellent overview of the current state and future roadmap.

The Jon Udell Challenge

Jon Udell of InfoWorld has announced a career change. He is going to work for Microsoft. When I heard this I almost fainted. Jon at Microsoft? And then I remembered a conversation that I had with him a few months back. He spoke of 'affecting change'. His position was that the divide between the turbo geeks and the average consumer of technology has grown to an unacceptable level. We, the software development community, have gotten so wrapped up in the technology that we forgot about who it serves and why.

For Jon, I am happy - he deserves good things in life. For myself, I am sad. There is a part of me that enjoys going to read about Jon's latest geek adventure. I realize that I do so for one reason. He is so incredibly smart that it makes me feel stupid. That feeling of stupidity motivates me to learn more. I really hope that Jon doesn't become one of those mindless Microsoft snobs who views the world from a purely Microsoft perspective. This might sound insulting but I've lost too many friends to the Microsoft brain washing machine. It has taken down some good men.

That said, I am issuing a challenge to Jon:

1. Make a difference at Microsoft. Create a list of ten things that Microsoft has to change and then be ruthless in evangelizing what Microsoft must do to remedy their issues. Keep the list public and monitor the progress.

2. Eat your own dog food. If you are going to evangelize a new Microsoft technology, first show me how Microsoft uses it internally.

3. Be a good citizen. If you introduce a new concept, show me how it 'bridges cultures', as you mention in your podcast. The legacy of 'embrace and extend' will rightfully haunt Microsoft.

Congratulations to Jon for the new position. More important, congratulations to Microsoft for adding a team member that has the ability to actually make a difference.

Linthicum on SOA Costs

David seems to have a formula for the cost of SOA:

Cost of Data Complexity = (((Number of Data Elements) * Complexity of the Data Storage Technology) * Labor Units))

Number of Data Elements being the number of semantics you?re tracking in your domain, new or derived.
Complexity of the Data Storage Technology, expressed as a percentage between 0 and 1 (0% to 100%). For instance, Relational is a .3, Object-Oriented is a .6, and ISAM is a .8.

So, at $100 a labor unit, or the amount of money it takes to understand and refine one data element, we could have:

Cost of Data Complexity = (((3,000) * .5) * $100)

Or, Cost of Data Complexity = $150,000 USD Or, the amount of money needed to both understand and refine the data so it fits into your SOA, which is a small part of the overall project by the way.

------------------

I can't speak for David - but I can promise you, this is not how we estimate SOA efforts. I'm not even sure what David is attempting to estimate. This post has me so confused... I'd recommend deleting that post - real soon.

SOA Governance

Eric Knorr of InfoWorld asks the important questions:
http://www.infoworld.com/video/archives/2006/11/jeff_schneider.html

WebLayers Executes Asset Governance

Last week I saw the latest demo from WebLayers on their 'Policy Based Governance Suite'. The demo hit home for one simple reason. They've done a great job of laying out the extended SDLC from a roles / assets perspective and determining 'what' needs to be governed at each stage.

Most of the vendors in the space have approached the governance problem from a registry perspective which is an important aspect, but not exactly a holistic view. WebLayers takes a methodology / lifecycle perspective. Their tooling allows you to plug your own process with roles (Business Analysis, Application Architecture, Service Design, etc.) and identify 'what' needs to be governed in each area - then, define the policies for each asset or artifact.
Example: The Design Stage includes a "Service Designer"; this person creates a "WSDL"; and all WSDL's have a policy that "Namespaces must be used".

They do this by using an interceptor model. In essence, they've created a 'governance bus'. WebLayers provides intermediaries that sit between the asset creation tool (schema designer, IDE, etc.) and the repository that will store the asset (version control, CMDB, etc.) This allows their tool to inspect the newly created assets just after they've been created, but before they've been sent to production. The policies are applied to the assets and results displayed (pass, fail, etc.) to the author.

I've been calling this type of governance, "Asset Governance" because the emphasis in on looking at the final output that is created and determining if it complies with enterprise policies. IMHO, Asset Governance is an essential component of any SOA program that utilizes an offshore element ("WSDL is the Offshore Contract").

The product was lighter on the other two type of Governance that I look for: Process Governance and Portfolio Governance. We sum it up like this:
- Portfolio Governance focuses on finding right problem (prioritization)
- Process Governance focuses on ensuring that all the right steps are taken
- Asset Governance focuses on ensuring that the output of the steps were performed in accordance with policy

I talked with the WebLayers team about the other two types of governance and received feedback that traditional I.T. Governance & Project Management packages might solve the problem (see, http://www.niku.com/). Most of these vendors built their products prior to the SOA era and have not gone back and revisited the functionality. They have not killed the "application as the unit of work" and moved to "the service as the unit of work" nor have they updated ROI formulas based on "shared services" (thus reducing investment, increasing ROI).

IMHO, the SOA Governance space will eventually find a nice intersection that includes both classic I.T. Governance, and the more modern "asset & process governance". It will be interesting to see which of the vendors will have the courage to tackle the end-to-end governance problem.

Rejected Four Years Ago...

I was at the Infoworld SOA event this week and and someone asked by about the use of ontologies in SOA. I haven't been asked about SOA ontologies in a LONG time... I had an immediate flashback to getting rejected by Web Services Journal to write an article on the subject...


------------------
Hi Jeff,
I sent your proposal to Sean Rhody for his review. At this time, we regret that we will be unable to accept this article for the magazine.
Gail



Gail, here is the abstract:
=====================================

“Semantic Web Services”
The desire for computers to easily communicate has long been a goal of both computer scientists and businessmen, the latter recognizing the financial gain of seamless systems integration. Over time, this goal has been recognized through network standards like Ethernet, TCP/IP and HTTP. More recently the standardization has moved up the protocol stack. Now, XML is being used to add structure through tagging, which facilitates concept delineation and enumeration. Web Services build on this foundation and enhance the communication through additional features including object serialization/deserialization (SOAP), service registries (UDDI) and standardized service interfaces (WSDL).

Yet even with these advances computers still aren’t aware of the meaning of the text that is being sent, nor are they able to make any reasonable inferences about the data. Tim Berner Lee and the W3C have been tackling this problem through an initiative dubbed the “Semantic Web”. This initiative uncovers the semantic meanings of transactions allowing companies to use a common dictionary and also enabling like terms to be disambiguated (“Automobile == Car”).

The use of the Semantic Web for concept delineation and Web Services for interoperability is enabling a new bread of applications known collectively as, “Semantic Web Services”. This article will explore the state of semantic ontologies, business grammars and emerging commercial products.

=====================================
Ok, the year was 2002, and I did refer to SOAP as an object serilization mechanism - perhaps it was appropriate for them to reject the article ;-) Now that we're approaching 2007 I believe that we'll start to hear more and more on this subject - who knows, maybe I'll resubmit the abstract!

CIO Paul Coby is Waiting for VHS

British Airways CIO Paul Coby is quoted as saying, "We don't want to invest in Betamax when VHS becomes the standard. There's no point in BA trying to go its own way on that. We will wait and see what standards emerge." (referring to SOA)

ROTFL.

Uh, Paul - the VHS version of SOA came out 5 years ago. And SOA isn't about the stupid standards - it's a way of integrating business and I.T.

OASIS SOA-RM Passes

The news is out:

The ballots for approval of Reference Model for Service Oriented Architecture v1.0 as an OASIS Standard (announced at [1]) has closed. There were sufficient affirmative votes to approve the specification. However, because there was a negative vote, the SOA Reference Model Technical Committee must decide how to proceed, as provided in the OASIS TC Process, at http://www.oasis-open.org/committees/process.php#3.4. A further announcement will be made to this list regarding their disposition of the vote.

IMHO, the real value of this passing is that people can quit working on it. The RM is an abstract document that is used by professional conceptual RA developers (of which there are about 10 in the world). This particular standard was... very popular by the people who wrote it... and not so popular by everyone else.

Initially, MomentumSI was the sole No Vote, but we pulled the vote so that the committee could just put it to bed and move on. Unfortunately, someone else voted no saying that the RM was so generic that it served no purpose. Well, they do have an interesting point... here's an interesting test, see if Client/Server architecture passes the SOA litmus test as defined by OASIS... Hmmm....

Again - 'architecture by committee' is a hard thing to do. I don't envy these guys. The next test that these guys have is to make up their mind on the RA. Will it be a Conceptual RA or a Profile Based RA?

Reference Architecture Models

With the OASIS RM 1.0 up for vote, I've found myself discussing RM/RA vocabulary again... I found a Momentum view of how we differentiate between views and their usage.

I've heard people tell me that the OASIS Reference Model has not been helpful in creating their Customer Specific Reference Architecture. Well, that doesn't surprise me - it wasn't meant to be used in that way. A reference model establishes a scope, goals and a vocabulary of abstract concepts. It is used by professional Conceptual RA developers.

In RA land there is a process of moving from very abstract to somewhat concrete. It is an evolutionary process. However, if you skip layers you'll probably find yourself confused. Professionals from OASIS have informed me that they've lost some contributors in the process of creating their specification. It doesn't surprise me - RM's are theoretical work and not for everyone. I have also been told that they intend to publish a users guide which hopefully will set the stage for its use patterns and anti-patterns.

Inside-out or Outside-in

Just a snippet from a recent email:

Sun created a platform when they wrote an aggregated set of specifications. They made their specifications API centric. It was an inside-out view. The next generation platform must be an outside-in view centered on protocols, formats, identifiers and service descriptions. It MUST be written using RFC 2119 format.

SOA, BPM, AJAX, etc. require a new integrated outside-in standardized platform. Until then, expect limited adoption or alternatively, enterprise rework.

SOA Acquisitions (revised list)

I've updated the list of acquisitions in the SOA space:
http://www.momentumsi.com/SOA_Acquisitions.html

A few interesting notes:
1. I couldn't be more disappointed in Cisco and their lack of acquisitions. AON failed (past tense). This could go down as one of the biggest blunders in software / hardware history. IMHO, Cisco should have revisited their executive leadership around AON a long time ago.

2. I've taken Service Integrity off the list. It appears as though they've shut down and didn't move the IP. This is disappointing as well - from what I've heard, many of the SOA ISV's were never even notified that the IP was up for sale.

3. WebMethods acquired Infravio. HP/Mercury/Systinet couldn't be happier. WEBM stock has been in the gutter for a long time. Infravio has a great product and a great team. It will be interesting to see if WEBM realizes that they need to move aside and let the Infravio team run their SOA direction.

4. I'm keeping SOA Software on both 'buy side' and 'sell side'. These guys have put together a pretty interesting package that keeps them in the pure play SOA infrastructure space. It's too clean. Someone one will grab them.

5. I added a couple 'client-side' guys to the list yet: ActiveGrid and AboveAll Software.

6. I added RogueWave (strong SCA/SDO story), I will proabably add some more SDO / data service providers in the near future.

7. I added testing specialists iTKO and Parasoft (although I don't know who will buy them).

8. I added Logic Library. With the Flashline acquisition, these guys become an obvious target.

Shai on Enterprise SOA

An excellent article identifies Shai's take on what the enterprise needs to do to prepare for enterprise SOA and the next generation SAP platform:

http://www.sda-asia.com/sda/features/psecom,id,595,srn,2,nodeid,4,_language,Singapore.html

BPM 2.1.4.7.3.5.7.43.2.6.8.9.4.3.5.8.4.1

I noticed some guys blogging about BPM 2.0. It reminds me of some thought I authored on the subject a few years ago.

http://www.looselycoupled.com/opinion/2003/schnei-bp0929.html

So - I hate the tag line BPM 2.0; it's so 2003. Ok, shame on me.

Here's why...it isn't about the driving the software solution from a single angle. It isn't about the PROCESS. It isn't about the USER INTERFACE. It isn't about the SERVICES. It isn't about MODEL DRIVEN. It's about integrating all of these concepts without your head exploding.

BPM 2.0 places too much emphasis on the process.
Web 2.0 places too much emphasis on the UI and social aspects.
UML 2.0 ;-) places too much emphasis on making models.
SOA 2.0 places too much emphasis on the services.
Entperise 2.0 places too much emphasis on... hell, being a marketing term.
Did I forget any?

Design Time Agility over Runtime Performance Cost

Last week we were working with a client to define their core architectural principles. We had listed, "Agility over Performance" and this created substantial debate. The first question was, "why were services going to perform slower?" - and, couldn't we have "Agility and Performance"?

On occasion architects will get lucky and find new technologies and approaches that are not conflicting. It has been my experience that this is the exception not the norm. More common is the need to resolve competing interests such as 'agility' or 'performance'. And when they do compete, it is the job of the architect to give guidance.

The services found in your environment will likely be victim to two primary performance degrading elements:
1. They will be remote and will fall victim to all of the performance issues associated with distributed computing
2. They will likely use a fat stack to describe the services, like the WS-I Basic Profile.

Now that we've described the performance issues we have to ask ourselves, "will the system perform worse?" And the answer is, "not necessarily". You see, for the last few decades we've been making our software systems more agile from a design/develop perspective. When we went from C to C++ we took a performance/cost hit. When we went to virtual machines we took a hit. When we moved to fully managed containers we took a hit. And when we move to distributed web services we will take another hit. This is intentional.

A fundamental notion that I.T. embraces is that we must increase developer productivity to enable "development at the speed of business". The new abstraction layers that enable us to increase developer agility have a cost - and that cost is system performance. However, There is no need to say that it is an "agility over performance" issue; rather, it is a "system agility over performance cost" issue. By this I mean we can continue to see the same levels of runtime performance, but it will cost us more in terms of hardware (and other performance increasing techniques). Warning: This principle isn't a license to go write fat-bloated code. Balancing design time agility and runtime performance cost is a delicate matter. Many I.T. shops have implicitly embraced the opposite view (Runtime Performance Cost over Design Time Agility). These shops must rethink their core architectural value system.

Summary: The principle is, "Design Time Agility over Runtime Performance Cost". This means that with SOA,
1. You should expect your time-to-deliver index to get better
2. You should not expect runtime performance to get worse. Instead, you should plan on resolving the performance issues.
3. You should expect (performance/cost) to go down

MomentumSI on Reuse

We published this a few months back but I thought I'd republish it since reuse seems to be a hot topic these days...

http://www.momentumsi.com/SOA_Reuse.html


Feel free to use (or reuse) the graphics! If you choose not to REUSE the graphics you can SHARE this page by passing someone a link ;-)

Lessons from Planet Sabre

Back in the early days of Momentum I did quite a bit of hands-on architectural consulting. One of my clients was Sabre, the travel information company. One day I was reassigned from another Sabre project (I'll tell that story another day), to a project in distress called, "Planet Sabre".

Planet Sabre was an application that focused on the needs of the Travel Agent. It allowed an agent to book flights, cars, hotels, etc. As you might imagine, these booking activities look quite similar to ones that might be done over the web (Travelocity) or through the internal call center. Hence, they were great candidates for services (they had a good client-to-service ratio).

I was assigned as the chief architect over a team of about 30 designers and developers. (BTW, I was like the 4th chief architect on the project). The developers were pissed that they received yet-another architect to 'help them out'. Regardless, they were good sports and we worked together nicely.

At Sabre, the services were mostly written in TPF (think Assembler) and client developers were given a client side library (think CLI). The service development group owned the services (funded, maintained and supported users). They worked off a shared schedule - requests came in, they prioritized them and knocked them out as they could.

The (consuming) application development groups would receive a list of services that were available as well as availability estimates for new services and changes to existing services. All services were available from a 'test' system for us to develop off of.

So, what were the issues?

The reason why the project was considered 'distressed' was due to poor performance. Sounds simple, eh? Surely the services were missing their SLA's, right? Wrong. The services were measured on their ability to process the request and to send the result back over the wire to the client. Here, the system performed according to SLA's. The issue that we hit was that the client machine was very slow, the client side VM and payload-parser were slow as was the connection to the box (often a modem).

We saw poor performance because the service designers assumed that the network wouldn't be the bottleneck, nor would the client side parser - both incorrect. The response messages from the service were fatty and the deserialization was too complex causing the client system to perform poorly. In addition, the client application would perform 'eager acquisition' of data to increase performance. This was a fine strategy except it would cause 'random' CPU spikes where an all-or-nothing download of data would occur (despite our best attempts to manipulate threads). From our point of view, we needed the equivalent of the 'database cursor' to more accurately control the streaming of data back to the client.

Lesson: Client / consumer capabilities will vary significantly. Understand the potential bottlenecks and design your services accordingly. Common remedies include streaming with throttling, box-carring, multi-granular message formats, cursors and efficient client side libraries.

The second lesson was more 'organizational' in nature. The 'shared service group' provided us with about 85% of all of the services we would need. For the remaining 15% we had two options - ask the shared services group to build them - or build them on our own. The last 15% weren't really that reusable - and in some cases were application specific - but they just didn't belong in the client. So, who builds them? In our case, we did. The thing is, we had hired a bunch of UI guys (in this case Java Swing), who weren't trained in designing services. They did their best - but, you get what you pay for. The next question was, who maintains the services we built? Could we move them to the shared services group? Well, we didn't know how to program in TPF so we built them in Java. The shared services group was not equipped to maintain our services so we did. No big deal - but now it's time to move the services into production. The shared services group had a great process for managing the deployment and operational processes around services that THEY built. But what about ours? Doh!

Lesson: New services will be found on projects and in some cases they will be 'non-shared'. Understand who will build them, who will maintain them and how they will be supported in a production environment.

Planet Sabre had some SOA issues, but all in all I found the style quite successful. When people ask me who is the most advanced SOA shop, I'll still say Sabre. They hit issues but stuck with it and figured it out. The project I discussed happened almost 10 years ago yet I see the same issues at clients today.

Lesson: SOA takes time to figure out. Once you do, you'll never, ever, ever go back. If you're SOA effort has already been deemed a failure it only means that your organization didn't have the leadership to 'do something hard'. Replace them.

Supply Side and Demand Side SOA

I come from a manufacturing background, hence I tend to think in terms of products, inventory (the supply) and demand. I do the same thing with SOA.

Most of the companies that I've consulted to start with a 'supply side SOA strategy'. That is, they create a strategy to create a supply of services. As everyone in the manufacturing world knows, creating supply without demand is a really bad thing. Inventory that is not used is considered bad for a few reasons, the primary ones being:
1. You prematurely spent your money
2. As inventory ages, new demand-side requirements will be generated causing the current inventory to become outdated

Most manufacturing companies have moved to some variation of just-in-time production. They wait for customer demand before they build the products. You'd think that this would work for SOA but in many companies it isn't. The reason is simple. These companies do not have a demand-side generator (the sales and marketing engines). Demand-side SOA is a discipline that doesn't exist in many corporations.

Demand-side SOA requires a change in philosophy and process. For starters, you have to begin thinking about all of your services as products or SKU's. Products must be managed in a product portfolio. Products must be marketed to potential 'buyers'. Demand side application builders should have well defined processes to shop for SKU's with cross-selling capabilities. We need to kill the 'service librarian' and replace it with the 'service shopping assistant'. Obviously, we have to quit thinking in terms of 'registries'; 'catalogs' are better - but 'shopping carts' are probably even closer to target.

Demand-side SOA is currently at an infancy. Our ISV vendor community has largely failed us to date but it is only a matter of time before they catch up. In the mean time, it is the responsibility of the I.T. organization to begin changing philosophies and processes to think in terms of supply and demand economics.

Forking Web 2.0

Screw Web 2.0.

Seriously. I have no need for Web 2.0 as it is being defined.

A few months ago I attended the MS Web 2.0/SOA think & posture event where a bunch of smart people overloaded the term Web 2.0 to meet their own needs, myself included. I almost forgot about the event until I caught a post by Gregor Hohpe (who impresses the hell out of me). Gregor attended yet another 'what the hell is Web 2.0' event and blogged about some attributes and tenets:

I'm convinced that Gregor is a freakin genius, so I really doubt if he missed the conclusion. In fact, the thinking was very similar to the Spark conference so I'm not surprised.

What just occurred to me is Web 2.0, as the world defines it, bores the living hell out of me. It's simple - I don't work for a consumer company like Amazon, Google or Yahoo - and as I look across my clients most of them don't need Web 2.0 functionality (as people are defining it).

What do I need? How do I want to overload the term? Easy. I am a SOA dude. I need a bad ass client framework for my services. I was hoping that the Web 2.0 guys were going to create a new client model - but they aren't. They're creating a social computing model - good for them, but it doesn't meet my needs. I need... a Collaborative Composite Client Platform.

What's are the characteristics of a CCCP?
1. Obviously, it's client-side, asynchronous, message/service oriented and highly interactive.
2. It's designed for the Web but merges the application and document paradigm successfully (like http://finance.gooogle.com)
3. The componentized UI is self describing and viewable by a user (think 'view source' meets 'portlets') .
4. The services called by the clients can be identified and reused. If a user doesn't want the UI they should be able to identify the services the client calls and use them instead.
5. Collaboration is a core tenet, not a feature .
6. Services are pre-compiled and available as-is however, client compositions can be changed at runtime and the new configuration can be permanently saved (typical with modern portals).

If the Web 2.0 guys create their manifesto and it's a bunch of e-tail crap where the customer is king - I'm out. My interests are in creating a new programming model for the client that serves as a foundation for any domain. It's time to fork Web 2.0.

Decoupling the Client and Service Platforms

The realization seems to be hitting the masses that Service Oriented Architecture isn't just about the Services, rather it is about the consumption of those services. Dare I say, it is about the clients (and the services).

The SOA programs that I've seen stall out typically were because they failed to identify the composite applications that would consume the services. It sounds rather obvious - but it isn't about building or buying services. Value is created when business people use clients that leverage the services. I guess that's one of the reasons why I always try to call this paradigm 'Client-Service Computing', rather than SOA.

Recently I reviewed a few enterprise SOA reference architectures and noticed an unpleasant pattern. Architects were forgetting to put the 'client' on the architecture. I know - sounds silly. Really smart architects get so caught up in identifying the patterns, domains, interactions, practices and standards associated with services that they forget about the clients!

So - we have clients and services... and we decoupled them. I'll say it again - we decoupled them! This takes me to my next point. For legacy reasons architects are continuing to insist that the client platforms be tightly aligned to the service platforms (.Net on both, etc.) This is non-sense.

Many of the last generation client platforms were not optimized for service oriented computing. By this I mean that they don't easily accommodate the Web Service standards nor do they embrace 'contract first design' and in general - many of them just plain stink. The reason we use them is because analysts like Gartner told us to go with a single platform. It's time to decouple the client and the service platforms. The client platforms should be optimized around UI capabilities including collaboration and human-computer-interactions. This might mean using a strong Web 2.0 platform. My bottom line is that there is no need to continue building UI's using the same ole platforms. It's time to optimize for this computing paradigm.

5 Years and 1 Hour

Yesterday, I cranked out a fairly extensive document dealing with SOA governance. Not to pat my own back too much but it kind of rocked. I delivered it to the client and they expressed their happiness with the work. I mentioned to the client that I wrote it over lunch. They gave me a look of disbelief - as if I was lying.

It occurred to me that it really took me 5 years and 1 hour. It took 5 years of research and thinking about SOA and one hour to write it down.

Sometimes I cringe when I hear enterprise customers say that they're going to do their SOA effort internally and not learn lessons from people who've already put in the 5 years.

Am I trying to promote SOA consulting services? I guess I am. You know why?
1. I make my living by delivering SOA consulting services
2. SOA is the most extensive offering I've ever seen in I.T. and quite frankly, I think people who try to go it alone - or by reading a Gartner report are going to fail.

It's that simple.