Extended Service Portfolio Management

A few months back I came across a great blog entry:

"At a fundamental level, the radical shift to SOA calls for a different mindset – a dramatically different one at that. The adoption of SOA shall signify itself to be an important development in the IT world. Software will be described as a portfolio of capabilities and possibilities instead of modularized applications. Data models will be standard-based and externalized to enable interworking between services, and data will be considered to be like any other form of "digital content" ever ready for exchange and transformation between systems."

Guys like Frank Martinez, Carlos Perez and John Udell have discussed the value of creating services whereby they may be recombined in a manner that was unintended at design time, 'unintended recombinants' if you like. Services were meant to be used, reused and reused in manners which the design never intended. Services that were designed correctly hold a special latent value - they hold the value of recomposition, both at design time and perhaps more importantly at runtime.

As large organizations perform Service Portfolio Management, they are beginning to notice that certain service designs enable reuse and recomposition. Nothing new here. However, as the service ecosystem begins to grow the Portfolio Manager will spend less time on their current 'service capability view' and more on the 'service possibility view'. Here, the goal is to evaluate opportunity.

The Service Portfolio Manager must become a specialist not only in enterprise refactoring but also in shopping and procurement. Services will be built, bought, leased and borrowed. Recombining services, from anywhere, for value-add solutions is at the heart of a Service Oriented Enterprise.

10 Low Profile SOA Comapnies

Here is a list of 10 SOA centric companies that fly a bit lower on the radar:

Redberri - Redberri Integration Platform helps enterprises to leverage the power of service oriented architecture (SOA) to streamline processes within a reliable, scalable and secure infrastructure.

GridScope - GridScope’s vision of Simply Managing Services™ is to provide a simple, dynamic solution for federated management of web services that underlie your enterprise-critical business processes.

FiveSight - FiveSight Technologies provides open source WS-BPEL-based business process execution and service orchestration products along with related support, training, and consulting services.

Xcalia - Xcalia makes SOA easy to implement with agile business intermediation software that combines heterogeneous data with services to rapidly develop and deploy transactional composite applications.

SOAPKnox - SoapKnox is developing products in the area of Web Services (SOAP) to ease developers in their development and deployment of Web services.

Above All Software - Above All Software provides business integration software that allows our customers to maximize the business value of their service-oriented architecture (SOA).

Cast Iron Systems - Cast Iron Systems is the inventor of the Application Router, a simple yet scalable integration appliance capable of connecting, transforming, and routing data and events from source to target systems.

GT Software - GT Software is a leading provider of enterprise application and data solutions that integrate mainframe data and applications into client/server and Web applications using ODBC, JDBC, J2EE and Web services.

Neon Systems - NEON's continued growth in the Web services market provides customers and prospective customers with the most robust set of solutions to support Service-Oriented Architectures (SOA).

Kenai Systems - Kenai eXamineXT provides One-Touch Vulnerability Assessment for Web Services.


Did I miss your company? Send me a note! jschneider at momentumsi dot com

IBM Ships ESB

IBM has announced that they have an ESB - and will be shipping it within the month.

Yesterday was a busy day talking with IBM personnel about the ESB and their new WebSphere Process Server. Naturally, I voiced a few concerns about calling it an ESB but if anyone can create a category, it is IBM.

What I heard was a couple interesting comments:
- Don't concern yourself with SOAP over MQ
- We agree, it isn't about JMS
- Don't worry about the protocols, we'll add'em in later

Hmm. If a WS-startup were to have told me this I would have gone off - however, this wasn't - it was IBM. Big blue customers will buy this. They will build on endpoints and mediate in the middle.

I remain disappointed in the lack of clarity around the ESB but am happy to see IBM put some new weight behind their SOA marketing. Congratulations to the IBM teams for getting these products out the door on schedule.

Our customers have been asking us for an ESB evaluators offering which is now available. Send an email to webservices@momentumsi.com. In addition, we'll be putting the IBM ESB into the MomentumSI SOA lab as soon as we can. We remain committed to pushing vendors towards the next generation of standards and pointing out those that are stuck in the J2EE world.

Pop Quiz: WebSphere was the IBM J2EE/Component brand; What is the IBM SOA brand?

It is time for WSEE

When I looked at open sourcing my stuff as part of an ESB - I quickly determined that the ESB had so many definitions (many of which didn't contain orchestration) - that the ESB had already crashed.

Modern software infrastructure is based on standards. Standards are grouped into profiles. Profiles are implemented and become products. I don't always like it - but that's the way it works. The ESB 'pick your standard' circus has demonstrated that our customers need more than marketing umbrellas.

We need architectural profiles.

The successor to the ESB MUST be profile based. Think RFC 2119.

What is an ESB?

MomentumSI's Definition of the ESB:

"An ESB is a collection of platform specific (Java) API's front-ending a vast subset of the Web service protocols, formats and identifiers."

---

---

---

---

I almost forgot - I've been meaning to congratulate Sonic on inventing 'SOAP over JMS', also known as the ESB. Unofficially, MomentumSI has been predicting that the bus would crash since October of 2003.

Here is the really sad news: ESB isn't dead as a category. It won't die until a replacement category is created. In my humble opinion, the ESB 2.0 won't work. What is the replacement: The Enterprise Service Network? Application Oriented Networking? I don't know. What I do know is that in an unprecedented display of incompetence Gartner & Sonic took a shotgun to the ESB category and blew it away.

Annrai, Frank, Dan, Roman - uhh... what the hell are you guys waiting for??? I'm going to start referring to you guys as the "Ball-less-four".

Open Source ESB - Apache Synapse

It appears as though the Apache ESB project, Synapse, is moving forward. According to InternetNews, Blue Titan, Infravio, WSO2, Sonic and Iona are all supporting it.

For those of you who aren't familiar with ESB's, Microsoft does a pretty good job of describing one in their position paper. They basically say that an ESB consists of:
- Orchestration & Transformation
- Intelligent Routing and Mediation

The heart of the current code base was donated by Infravio. However, it was quickly noted:

Sonic Software itself an ESB vendor (as well as an initial committer to the Synapse project) said that Synapse wasn't a full blown ESB. "This project is related to ESB , but it is not in itself an ESB," Dave Chappell, Sonic Software's VP and Chief Technology Evangelist told internetnews.com,. "What Synapse bring to the table is a mediation framework that allows users to get in the middle between service requesters and providers and perform various tasks – including transformation and routing and that helps to promote loose coupling between services "

So, if I understand Dave correctly - Synapse has the routing and mediation but not the orchestration and transformation. Sounds like a problem! Perhaps I can help?

As of today - I am open sourcing our Orchestration IDE, BPEL engine and Transformation system.



But what good is an open source ESB if you can't get decent support? None! Sounds like a job for MomentumSI.

Done. Stay tuned. More to come.

Mark Baker Hates Architecting First?

Mark, "I won't rest until you REST", Baker kicks a bit of sand on architecture first.

Oh stop it Mark - you architect first, too.

MomentumSI recently finished a large transactional web system for a customer. The customer was a startup - and had NOTHING. Guess what - before writing a bunch of code, we gave them a "Web Architecture"; we talked to them about the ilities (scalability, integrity, availability, maintainability, etc.) We recommended the infrastructure: firewalls, routers, load balancers, operating systems, web servers, web analytic engines, etc. We also recommended some design patterns and some platform choices (J2EE, IoC/PoJo, SOA-IoC/PoJo, .Net, etc.) Then we talked to them about implementations. My point is that these are things that the company didn't have - and needed before a SINGLE LINE OF CODE WAS WRITTEN.

If you're an organization that is going to do SOA - you should go through a very similar exercise. If you haven't answered the SOA infrastructure questions - or architectural configuration questions and you just start building services you'll end up in a mess. It isn't: "you may end up in a mess" or "you might end up in a mess" - it is "you WILL end up in a mess". Closing your eyes to up-front architecture is a bad idea in web systems, SOA systems or just about any system you can think of.

So, in summary:
YES - "Projects and enterprises should define their Service Architecture FIRST". If you're going to do service based systems - do it within a basic architecture. This doesn't mean go off and architect until you're blue in the face. It means lay down the basic architecture - we did it for web systems - we'll do it for SOA systems.

Tim Bray Supports WS-*

Via Nick Gall.

Tim Bray did an interview and has asked his readers to draw their own conclusion, which I have: Sun customers have had to invent their own distributed computing solutions - on their own dime - and have created proprietary solutions. AToM tried to tell them for years but had little luck.

Questions and Answers with RouteOne
Q: How did you do Routing / Addressing?
A: We went proprietary - because we had to.

Q: How did you do Reliability?
A: We went proprietary - because we had to.

Q: How did you do Orchestration?
A: We went proprietary - we can't wait to get off of it.

Q: Do you put anything standard in the header???
A: Yes, WS-Security

Q: How do you do messaging?
A: Vendor specific; ubiquity via platform API - not protocol

--------------------------------------------
Now it is possible that I misunderstood the point that Tim was trying to make... but if I read it correctly, he has a customer that wants protocol ubiquity. The customer has real problems (transactions, correlated messages, reliability concerns, security concerns, performance concerns). He had to roll his own protocols (boo) and stated that he could have gone faster if he didn't.

I'm so happy that Tim found an enterprise customer that had real issues. Perhaps with the acquisition of SeeBeyond Sun will become a WS-Shop ;-) Gee - having a few protocols implemented sure can change your tune!!!

The funny thing is that my friends at IBM and Microsoft NEVER talk about SOA specs anymore. That's so 2003. Perhaps we should debate JBI (or not).

Service Network Newsletter

The August edition of the Service Network Newsletter is available.


Enjoy.

SOA Boot Camp

After months of work, we're finally kicking off our SOA Boot Camp. MomentumSI consultants will be immersed in 17 days and nights of non-stop SOA education!

You name it - it made it: registries, repositories, governance, mediation, orchestration, composite applications, SO security, SO-BAM, legacy enablement, WS-I & WS-*, Indigo, Axis, SOBA, ESB, SOA reference architectures, SOA reference models, SOA process / methodology, SOA best practices, service analysis & design, XML message design, reuse metrics and more.

We have also signed up many of the leading SOA companies to provide vendor specific training on their best-of-breed products by their local gurus. After we run it internally a couple times we will be taking SOA Boot Camp to our customers.

SOA is a fundamentally different computing model - we've realized that the 2 and 3 day training just doesn't cut it. Let's hope that 17 gets the job done.

MS Composite App Strategy?

Web Services enable composition through lowest-common-denominator ubiquitous protocols.

Composite applications pull together Web services to fulfill some set of requirements. If you compile the services together (bake at 450 degrees), you'll find that the agile-controller isn't so agile.

So, all of this said... what is the Composite Application Strategy for Microsoft? All the work on Indigo was great, but where is my 'service join' facility, or my 'agile controller' - for that matter, where is my 'semantic adapter'?

From one perspective you could say that SDM represents a portion of the strategy - but it only reflects the 'service oriented bill of materials'. BizTalk? Wow - I hope not... any insight?

If you're not familiar with 'composite applications platforms', take a look at:
http://www.cordys.com
and
http://www.aboveallsoftware.com

Hacktivity Level

The 'hacktivity level' is way up. Casual services, dynamic scripting and remixing are clearly the theme for 2005. JSON, POX, REST, RSS, etc. have taken hold - they've intersected with client-side contextual applications like Google Maps to create what some people would have called a composite application - you may prefer the terms remix or mash-up. Even Business Week is taking notice.

SOAP & WS-* have turned into the stiff-collared, corporate world, slow to move technologies. Once again, our passionate technology community is breaking all the rules and creating compelling applications - and I love it.

Hacker - 'I need that service and that other service. I need to combine them. I need a server-side aggregation engine. I need to re-serve the combined service. I wonder who will enrich/transform my service? Who cares. I need a client-side visualization mechanism. It has to have easy API's that front-end the data services. Damn this is cool. Interesting - it looks like a 'web of services'. Clients. I need contextual pivoting - Maps - Austin - Concert Halls - Concerts - Reviews - Tickets - Buy - Receipt - Calendar.'

CIO - "Did you see what some hacker did with concert reservations? Can we provide our core services to the outside world? Wait - can we consume other peoples services and create our own application out of them??" - Why yes.

The World Wide Web is turning into the Service Oriented Web. The technologies will be stiff and flimsy; the artists will be straight-laced and no-laced. The intersection will be magnificent; the Web is reborn. The SOAP/WS "transactional web" will be a portion of the new SOW, perhaps a small portion. Our future will be driven by those who have the passion and desire to create it.

Hey Don Box,

All of us in the SOA blogosphere have appreciated your input and insight. However it has come to my attention that Microsoft isn't practicing what they're preaching. I'm hoping that you can tell me that my information is dead wrong.

I ask one simple favor. Start blogging about how Microsoft is using SOA internally. I've been told that Microsoft spends more money on SOA evangelists than they do on building SOA solutions internally.

I'll extend this challenge to every single Microsoft blogger. Don't tell me about your latest spec or library - tell me how Microsoft uses it internally. The Microsoft I.T. department should become the SOA showcase.

Respectfully Submitted,
Jeff

I, Too, Have Seen the Enemy

Charles Garry comments, "I Have Seen the Enemy and It Is Complexity". He writes,

"Web services is yet another example of potential complexity headed your way." ... "I have perused a number of books on the subject, but the one thing I never see discussed is how one makes a virtual application highly available. What are the implications for backup and recovery of data? How do we manage security permissions and authorizations? Clearly, with the added flexibility, we have also created a new set of obstacles to overcome."

He goes on to state:

I suspect we will continue to see a growing trend towards infrastructure rationalization (fewer kinds of things) and consolidation (fewer numbers of things) as a means to reduce complexity. This trend is both useful and necessary if we have any hope of utilizing some of these new computing paradigms.

Charles believes that Web services increases complexity. By no stretch of the imagination do I consider myself an expert in complexity, chaos, systemics or related concepts. However, I believe that the conclusion Charles reaches is incorrect.

His assumption is that we are adding new elements to a set - and as the set grows, so does the complexity of the set. His answer is to reduce the number of elements (rationalization and consolidation) and this is where we differ.

Enterprise software is going through a 'specialization' and 'externalization' movement. We are taking pieces of code that seemed unique and abstracting them into reusable concepts. We do this with design patterns, architectural patterns, domain specific languages, platforms, libraries and specialized engines. Each time we see these reusable items, we 'write them down' (or externalize them). The complexity in the system already existed; we merely found the pattern, factored out the common substrate and documented it. Externalization is a well-known mechanism for reducing complexity; you take that which was not understood and make it understood.

As we take our mountains of unintelligible 3GL code and move much of it into common infrastructure, we increase the amount of 'known' computing elements and decrease the amount of 'unknown'. This process has an odd effect. As the inventory of known elements grows so does the concern for 'having too much'. This leads to a human fear of 'large-set-complexity'. And the natural reaction is, yes - you guessed it, rationalization and consolidation. This is a human tendency whereby man attempts to diminish complexity by hiding it. We take those First Order computing elements that made our inventory list and reduce them back into 2nd order elements, thus removing them from our list. And somehow, we feel better that we defeated the enemy.

Specialization of vocabularies, models and elements is a natural progression of every science. However, specialization must be followed with abstractions, taxonomies and containment. In essence, we must not hide our new found computational elements, rather we must manage them.

Web services are an additional element to our computing infrastructure. From one vantage you can claim that they are just another element to the set increasing the complexity. However, Web services will help to reduce systems complexity by adding a new abstraction over our computing environment. Although the total number of elements increases, we find ourselves commoditizing those lower in the stack and working with those that are closer to the business value.

The enemy of SOA isn't complexity; it is a lack of understanding. We must not retreat from our sophisticated computing infrastructure. Instead we must use cost effective means to manage the piece-parts, identify the abstractions, create common groupings and externalize the system.

Crush My Head Like an Acorn

Sean McGrath has suggested that the Reed's Proposition may be closer than Metcalfe's Law in describing a theoretical upper limit value function on a Servcie Network.

For those of you who haven't had the pleasure of meeting Sean, his physical presence reminds me of the "boulder thrower" on Braveheart:



I firmly believe that Sean could crush my head like an acorn with his bare hands. Hence, I'll be careful to disagree with him.

Acorns and noggins aside, as usual Sean makes a great point. Metcalfe's law is an over-simplification of a complex formula. In my humble opinion, so is Reed's.

My goal is simple - I want the enterprise to begin looking at their services as a network and determining the value function that works for them. Today, virtually no formulas are applied.

It is true that some services will provide more value than others. And some business processes are fundamentally more important to competitive advantage than others. Services that support key processes will provide greater value. We will also see that 'service gaps' will destroy the value of 'process networks', suggesting that some services only have value when bundled with other complementary services in the same value chain.

Value chains are created around: customers, suppliers, products, employees, etc. We create networks that link these entities using various levels of constraints. Those links which require minimal constraints are usually deployed first and are the most free-flowing and collaborative in nature. From a computing perspective these entities will be joined via well known networks including WWW and RSS. The value of low-constraint, high collaboration networks is well known.

As we increase the level of constraints and diminish the ease of creating self-forming networks due to semantic mismatches, protocol impedance or the mandate to fulfill non-functional requirements, we find ourselves creating services with a lower ROI. This isn't because the "R" (return) has changed; it is because the "I" (investment) has changed.

Enterprise Service Networks will not produce an adequate ROI if they are not able to reduce the 'service drag', lowering the investment. That is, it is the burden of the enterprise to employ 'service oriented infrastructure' that facilitates rapid service creation and composition by removing the aforementioned obstacles. In essence, the value function must look forward in time, forecasting a future value of the network, less the 'drag' related to adding clients and services. This is what I call the AOL Argument; fundamentally the Web created a more fertile environment for adding content to a network than did AOL.

Corporations use valuations that are forward looking vehicles (DCF, EVA, etc.) I would argue that the value of the network is less of a snapshot in time but rather a forward looking forecast based on the potential growth of the network as it directly relates to the value chain of an organization. I view the Metcalfe/Reed concepts as an easy way to introduce I.T. personnel to network valuations. However, due to their inability to capture forward looking results and correlate node-to-value concepts, they both remain inadequate at best.

Please don't crush my head like an acorn ;-)

The Value of the Service Network

Just a quick reminder...



We aren't just building Web services.

We aren't just creating an SOA.

We are building a network of business services.


We must also remember that it takes MORE work to create VALUABLE services.

Sun's Big Year - 2186

Based on historical performance of SeeBeyond (income statement), and including new synergies (sales force, engineering), I estimate that Sun should break even on their $387,000,000 cash investment in the year 2186. I realize that it is tough to wait 181 years for the ROI to kick in - but I'm sure Scott (and his great grandchildren) will be patient - as will Wall Street.

I wish Sun the best of luck getting JBI adopted.

I should mention to the M&A crew at Sun - - I heard a hot rumor! A COBOL company is up for sale!! Time for JSR 38,940 - JCOBOL! You'll make BILLIONS!

The Service Network - July 2005

Our July edition of The Service Network is available.

And yes, the favorite line seems to be:

"SOMA (Service Oriented Modeling and Architecture) –Vaporware or IBM Global Services with a copy of Visio?"

When will SOA take off?

In 135 days.

Seriously - the Web service and SOA era will begin in 135 days.

176 I.T. Jobs Posted in USA

In the last 24 hours, 176 I.T. and software related jobs were posted on Monster.com...

I was curious how many were posted in India however it was difficult to tell. Even when limiting the search engine to only return jobs posted in the last 24 hours, the number posted exceeded the limit. I attempted to further tune the search criteria but was still hitting limits.

My best guess is that in the last 24 hours approximately 3,000 to 4,000 jobs were posted on Monster.com for India.

After sampling the job descriptions, it appears as though approximately 70% of the jobs posted in India were designated to fill U.S. demand. If this math is correct, that means that for every software job requirement that is posted in the United States, fourteen are posted in India to fill the same U.S. demand.

AToM: Sun Needs to Learn From Microsoft

We have a clear winner in the quote of the month category:

Anne Thomas Manes, a Boston-based analyst with Burton Group Inc., echoed those sentiments.

"What bothers me is that I really don't like JAX-WS," Manes said. "The Sun JAX-WS team really needs to learn a few things from Microsoft. They should be building something like Indigo—a common programming model that encompasses JAX-WS/JAXM, JMS, RMI and EJB. But Sun doesn't get that."

Can someone please send me the name of the person at Sun who is responsible for this? (Yes, I'm still interested in seeing prediction #7 of 2004 come true... how sad.)

The WSDL is the Offshore Contract

As more and more organizations move large portions of their I.T. department to offshore facilities they are questioning which activities to keep on-shore and which to push offshore.

For those that have chosen a service oriented path, the decision seems to be clear: "The WSDL is the Offshore Contract". In other words, domestic employees will work on business processes, enterprise architecture, orchestrations (agile controllers) and service design (up to the WSDL). And building the service? You got it - offshore.

It's an interesting line to draw, yet perhaps an obvious one. Using the "software contract" as the "legal contract" invites a clean division of labor and facilitates a simple approach for acceptance testing.

Obviously - the WSDL only defines the external interface leaving other aspects of the service (LIKE THE INTERNALS) to be described by other mechanisms. The algorithms and business logic are described with variations of the Use Case (think 'service case'). The relationship between the server and other elements (the hosting container, other services it calls, etc.) will likely be described by the System Definition Model, or some variation.

The science of describing architectures, interfaces and dependencies has significantly increased in recent years and it appears as though we are entering an era where the adoption level will skyrocket. The ability to describe systems in this fashion is what I've been calling 'precision specifications'. By using digital definition languages like SDM and WSDL, we are able to provide a 'software factory' with clear directions for construction.

Using 'services' as the unit of work appears to be a very manageable approach. Companies that are big into test-driven development will likely find test-driven-services a natural extension. Organizations that have adopted the Agile Manifesto may find this world too rigid. However, the Extreme Programmers may like the ability to lump X number of services into an iteration plan. And project managers will be thrilled to see acceptance tests directly linked back to line items on the Gantt chart.

I.T. organizations must increase their capability to describe systems with precision specifications prior to initiating a 'SOA factory'.

CIO's Aren't Taking SOA Serious

I'm going to make a quick prediction:

More CIO's will lose their job over SOA implementations than lost their job over ERP implementations.

---

I was chatting with a CIO the other day and asked him, "In retrospect, do you think you were involved enough in the early stages of your ERP implementation?"

He answered, "In retrospect - no - I wasn't involved enough. Had I known the size of it I would have definitely gotten more involved."

I followed up with, "Are you aware that an enterprise SOA roll-out will be significantly larger than your ERP implementation?"

He started laughing; he thought I was joking. My face didn't change. He quit laughing. "Jeff, are you serious?"

"Yes, I'm very serious. SOA is a complete overhaul impacting how systems are analyzed, designed, built, integrated and managed. And not just some systems - all systems including packaged applications like ERP."

He responded, "But with ERP, it was an all-or-nothing approach. Doesn't SOA offer an incremental approach - one that can be managed better?"

"Yes and No", I responded - "SOA is first and foremost about a network of services. The value of your service network is directly related to the effort of creating/obtaining services and making them available. All networks require a critical mass of 'valuable services' before they become useful to the consumers. The SOA service network is no exception. What I'm saying is that you can take a slow and steady approach to SOA but be prepared to receive gains that are even slower. Remember - this is really about the 'network effect'.

SOA done properly is a massive undertaking. It requires a rethinking of your infrastructure, development methodology, business impact analysis, budgeting process, organizational design... Don't underestimate the value it provides, the competitive advantage one can assume AND the investment it will take!"

It was clear that I had surprised my friend - perhaps even scared him a bit. And I realized that so many of the people in this space (vendors, analysts, press) are doing a disservice to the CIO by failing to tell the whole story: An Enterprise SOA implementation will be larger than ERP, Y2K and the Web.

Now is the time to get involved.

SOA Training!

For the last couple years we've been training people in SOA and Web service technologies. Recently, we've added a couple courses including "SOA for Architects" and "SOA for Managers" - both excellent.

See:
http://www.momentumsi.com/services/training.html

I'd love to do a course on buidling your SOA Reference Model...

Hiring SOA Consultants...

Momentum is opening up new positions for SOA consultants. All positions are travel intensive and require a strong background in distributed computing and enterprise architecture. Interested candidates should send Melissa a note: careers@momentumsi.com

We're basing the consultants in one of three delivery centers:
- Austin / Dallas, Texas
- Reston / Washington D.C.
- San Francisco / Silicon Valley
------------------------------------------------------

The Service Architect
At Momentum, the Service Architect is responsible for analyzing architectural requirements and translating into service based solutions. The architect will work with various existing architectural styles (monolithic, client/server, 3-tiered) and modify those systems to accommodate a service based approach. The architect will also be responsible for identifying SOA based solutions to the non-functional requirements (security, transactional integrity, reliability, availability, agility, etc.) Lastly, the Service architect will be instrumental in creating Software Architecture Documents, identifying components, estimating time-to-transition, providing guidance to designers and developers and linking architectural investments back to financial returns.


Requirements:

The SOA Architect must be well versed in Enterprise Architecture, Service Oriented Architecture (SOA), J2EE and or .NET Architecture.
Full life-cycle implementation experience (requirements, design, implementation, integration, testing).
Experience designing and implementing n-tiered application architectures.
Fluency in first generation web services standards, technologies and tools (e.g., XML, SOAP, WSDL, UDDI, BPEL, etc.).
A strong distributed computing background is a requirement (CORBA, RPC, DCE, DCOM, RMI).
Familiarity with messaging systems is a plus (Tibco, WebMethods, MQ, SeeBeyond).
Familiarity with commercial Web service products is a plus (Systinet, Blue Titan, etc.).
Knowledgeable on WS-* standards.

Other:

• Strong interpersonal, strategic oriented thinking and excellent written and verbal skills are essential.
  
• Ability to present professionally and persuasively to C-level executives.
  
• A minimum of 10 years experience overall in software development.
  
• A degree in Computer Science is preferred.
  
• Travel is required for this position.
  

About MomentumSI
MomentumSI is a pioneer in service oriented architecture, integration, and application development. Since 1997, the company has built a reputation as a leader in architecting large scale distributed computing solutions for IT organizations. Today, MomentumSI is at the forefront of service oriented architecture and integration for enterprises. We built the first Web service orchestration server for .NET and have been an active member of Web services standards groups, such as OASIS. Momentum continues to invest our SOA practice and is looking for the best of the best to join us.

Entire Software Sector up on Web Services!

Now this is interesting:

NEW YORK (MarketWatch) -- Goldman Sachs raised its rating on the software sector to attractive from neutral on the belief that the emergence of Web services architecture will accelerate industry growth and on expectations that share prices will rise as investors position for an anticipated year-end rally. Analyst Rick Sherlund believes a movement to a new generation of Web services base standards, which will result in more flexible and adaptable systems that can allow for changes in business processes, could reinvigorate growth for information technology vendors. He said the Phase I adoption of infrastructure software from IBM , BEA Systems , Oracle and Microsoft to enable the new standards is well underway, and the Phase II re-architecting of existing applications from SAP , Siebel and others is upcoming. He said Phase III will be integrating the new standard into desktop systems, which should stimulate a replacement cycle. Sherlund recommends positioning in the expected leaders of the next generation systems, even if the benefits aren't expected for a few years, given attractive valuation and the anticipation of a typical year-end rally.

See: http://biz.yahoo.com/cbsmb/050523/390440d059074e4881ce4b5a86c54332.html?.v=1

Increasing Reach

Software reuse has always had issues. In my opinion, the most common types of reuse are:
1. "Copy & Paste Reuse".
2. Off-The-Shelf Reuse - Here, we leverage JVM's, .Net API's, J2EE libraries, open source offerings, etc. that are pre-built and providing immediate value.

The third (and elusive) category would be the homegrown libraries - often domain specific. Let's be clear - the more general the problem, the easier to reuse; conversely, the more specific the problem, the harder to reuse. The easy stuff is packaged in your favorite off-the-shelf library - leaving the more difficult stuff for the I.T. department.

Reuse is difficult - and reuse via language specific code snippets is often easy but fails to scale and provide real value. Reuse via off-the-shelf packages is great - but in many organizations - this one has already been tapped. This brings us to Reuse Through Reference.

Services are a great mechanism for providing reuse through reference - however, many people seem to forget one simple item: REACH.

Reach describes the 'reusability potential'. Imagine having 10 systems 9 of which were in COBOL mainframe apps and the 10th was a .Net application. Let's state that the COBOL applications don't have the ability to access the native services provided by the .Net application, but do have the ability to talk Web services. Interesting - I just stated that it can speak "Web services" - does that mean WS-I basic profile? Which other WS specs does that include?

In an attempt to increase reuse through reference, we (the enterprise architects) are finding the need to increase reach. This means that we must identify protocol ubiquity levels within our organizations and quickly create critical mass.

Generally speaking there are two methods for increasing the reach:
1. Modify the connector capabilities at the endpoints (COBOL apps, etc.) via service enablement (WS-PaintJob)
2. Provide protocol mediation/translation in the network.

WS-PaintJobs are an easy way to black box systems and increase reach. However, the total number of protocol permutations between any given client and any given service grows exponentially with the size of the service network. This leads to the realization that a protocol mediation strategy is not only useful but mandatory.

The Client-to-Service Ratio

It is natural for people to think about their SOA in terms of the number of services that they have created. And as one person joked the other day, every company seem to report having between 200 and 500 services. The criteria for being a service tends to vary significantly - anything from XML over JMS to EJB's to WS-I compliant Web services.

Service Architects love to brag about the number of services - and I let them. Actually, I encourage them to brag. However, I'm quick to challenge these same people with a very simple question:

"200 SERVICES! That's great - but how many clients???"

This simple question usually makes the most pompous architect fall to their knees in shame.

You see, one of the reasons that guys like me go around shouting "SOA, SOA, SOA!" is because we believe that a sharable infrastructure can be used across "applications" to enable a new level of productivity. Yes, we believe in the unicorn called "reuse". The "how many client" question hits at the heart of the share/leverage/reuse debate. I know it - they know it.

Back to my story - I ask the question, "How many clients" - and I stare at their face like a CIA agent looking for twitches. Within seconds, I witness the five phases of panic :-)

Phase 1: "Oh crap. I should know this."
Phase 2: "I better make up a number - I told him 200 services - hence I'll tell him 200 clients!"
Phase 3: "Nope - that won't work - he'd realize that my client-to-service ratio would be 1:1 indicating that I wasn't getting any leverage."
Phase 4: "2:1 seems high at this stage; 1.5:1 seems believable... yea, (1.5 * 200)=300... but I can't tell him 300..."
Phase 5: The Service architect looks me squarely in the eye, head shifted slightly to the left and says "Currently, we have 322 clients."

And I say - "Excellent" followed by a slight grin and a playful stare indicating that I was 100% aware of what had just occurred. I usually then move to relieve the 5 seconds of torture and lies that I just put them through by saying something like, "Well - it is still early; SOA is still evolving and you won't see that number climb until more of the organization adopts it." And my friend gladly take the bait - "Absolutely! Right now, we're just building out the core services - we're confident that the ratio will increase over time."

And I move on to question number 2...

Nokia & Web Services

VNUNet.com reports:

"Nokia has vowed to build support for web services into all its smartphones by the end of the year.

The firm explained that the development push will include support for common software standards including Soap and XML, as well as for more intricate applications.

In the future both the Series 60 smartphone operating system and the Series 80 software used in the Communicator range will offer full web services support.

"By 2006 all Nokia smartphones will be web services enabled," said Timo Skytta, director of web services at Nokia.

Most excellent!

SOA - PHASE III

Several of my buds have been giving me a hard time. They're claiming that I've checked out of the SOA world. And as an emerging tech. guy, I'll agree.

You see, from my vantage point - SOA & Web Services have already crossed the 'value' threshold and are ready for adoption (Gartner is wrong.) Dare I say that WS is no longing 'emerging'. We have now entered phase III.

Phase I was basically 'protocol oriented' - with a focus on the WS-I Basic Profile (SOAP, WSDL, UDDI, XML Schema). Phase II was the WS-* stack (the aspect oriented protocols).

Phase III is about the using the darn things. Whacky, eh? You see, I talk with hundreds of companies about their utilization of web services. In most cases they tell me this:
1. We connect fat .Net clients to Java application servers (and use soap in between)
2. We connect to some ASP (like Amazon, SF.com, etc.)
3. We front ended some legacy system with services.
The same companies will brag about their rich utilization of UDDI, Fabric, ESB, etc.

However, when I ask about SOA from a reference architecture perspective, I mostly get blank stares. Then, I go to the white board, hand them the marker and say, "draw your architecture and show me where you use services." And in almost every occassion they draw an application architecture which is usually a 3+N (the usual 3 tiers plus roughly N external services - pick your favorite - LDAP, EII, orchestration, etc.)

So after seeing the same thing at several different companies you begin to realize that the world is landing on patterns of usage (and yes, 3+N is the most common). But rather than calling these 'service patterns' or something like that, they are really just architectural patterns (that use services).

Full Circle.
It's true - I'm less interested in watching protocol geeks fight in OASIS about non-sensical issues. We're finally here. We're finally at a point where we can use this stuff to create business value. That said - phase III is all about facilitating the adoption. And yes - expect to see pre-canned architectures that use SOA. But beware - the shift is heading back to 'architecture as a whole' and reviewing the place for SOA / WS in our upgraded world. You see SOA Phase III is really about putting cleaning up our distributed architectures and applying SOA appropriately.

Architecture Change Request (ACR)

There are a number of studies that discuss the impact of failing to adequately express system specifications. Oddly, most of the studies focus on the 'functional' side of the equation emphasizing the Use Cases or similar requirements document. As more and more of our work efforts are being moved to assembly line creation (outsourcing, off-shoring, remote development), it is becoming much more important to clearly articulate system specifications and the change requests.

I've witnessed teams go through great pains to version control the least important documents in the system. Source code, JavaDocs or test cases will be meticulously controlled, while the over-arching architectural documents will often go unattended. When they are versioned, they are usually treated as a BLOB, with no ability to perform a "Diff" or to maintain an automated digital log of the changes.

As organizations begin to adopt digital described reference architectures and use those elements to digitally describe their candidate architectures, they will find the ability to perform the 'diffs' and to manage change at an acceptable level.

Suddenly, we begin to see our architectural process looking much more like traditional hard-goods engineering process:


The Architectural Change Request (ACR) enables a software development team to track the changes to one of the most important elements in the SDLC, the architecture. Keep in mind, when architectural changes are made the impact (time / money), is usually significant. I firmly believe that the ACR/ACO process will encourage application architects to think and plan their architecture out more thoroughly at the initial stages leading to less rework after the project is initiated.

ADL, IDL & EDL

There are several different ways to review elements in an architecture.

1. You can look at the "interfaces" to identify the entry and exit points. Obviously, this is a big part of what web services are all about with WSDL as an "Interface Definition Language" or IDL.

2. You can look at the "engine" that provides the base infrastructure: "It's a database! It's a Rules Engine!", etc. Or more precisely, "It's an Oracle Version X database, with clustering". This is the "Architecture Definition Language" or ADL.

3. In addition, you can look at the mechanism that is used to "extend" the "engine" which provides the "interfaces". In an orchestration engine, the extension language might be "BPEL", in a presentation engine, the extension language might be "JSP". Once an EDL is identified, you can begin to look at the capabilities of the extension mechanisms (language features, libraries, etc.). Unfortunately the industry remains very immature in this area. However, I am confident that as the industry begins to understand the basics of a "language to create/describe languages" the art will quickly advance.

The software community is making progress in describing the systems we build. Our meta-descriptions of our building blocks and finished systems is a huge advancement toward the vision of the "software factory", "software IC", or which ever metaphor you prefer.


-------------------------------------------
INSIDE, OUTSIDE, UNDERNEATH & ALL TOGETHER!

Web services & SOA make people think about architecture from the "OUTSIDE" - that is, the external interfaces. The "EDL" allows you to evaluate it from the "INSIDE" - or how you build that which is exposed. The ADL facilitates "UNDERNEATH & ALL TOGETHER".

Although one of the primary benefits of SOA is to abstract the consumer from the architectural elements and extension mechanisms, this doesn’t mean that the architect that is creating the solution can avoid the issue. I firmly believe that the opposite is true. SOA architects must apply equal attention to these concerns as they do to their uniform interfaces, mediation strategies or WS-catalog taxonomy.

----
Note: I'm using the term EDL to describe the capabilities of a DSL or a general purpose programming language.

SDM - it's right.

My father bought one of the first 100 TRS-80 computers ever made and at the age of 11 I was fluent in assembler. By the time I finished high school, I could program in 14 languages. I've found other people who have similar backgrounds - and in almost every case they share a similar trait: the ability to recognize a disruptive technology.

In my career, I've made a few "early calls":
1985 - Windows 1.0
1991 - Visual Basic 1.0
1993 - PowerBuilder 2.0
1995 - Java
1997 - XML
2002 - BPEL

and now,
2005 - SDM

It's that big. It's right - and it will change our profession.

Canned Architecture

Several things in the Agile world has always bothered me. The one I'd like to point out today is the concept known as BDUF. The summary of BDUF is:

"The term BigDesignUpFront is commonly used to describe methods of software development where a "big" design is created before coding and testing takes place. Several ExtremeProgramming (XP) advocates have said that such "big" designs are not necessary, and that most design should occur throughout the development process."

Over the years I've noticed that the closer one is to the code, the less they tend to distinguish between architecture and design. "Code Intensive" people tend to be bottom up guys - they start with code, refactor to a new design - rewrite, refactor, repeat. Call me old fashioned, but I firmly believe in the use of enterprise reference models, template architectures, application blocks, architecture & design patterns and candidate architectures. This clearly puts me in the bucket of "Big Architecture Up Front" - and I'm proud of it.

In the last 8 years at MomentumSI, I've had the chance to see hundreds of projects across our clients. And the one thing that has always amazed me is the reusability of architecture and design elements. Sure, the platforms change - new elements are added - the functional domain is different, but for the most part the basic architecture and design is usually about 80% reusable.

To clarify my position, I'm really a "Canned Architecture Up Front" kind-of-guy. In working with architects, I've noticed an interesting behavioral pattern. Application architects realize that when their architecture is "finished" they're probably going to have to do "design work" or God help them, "coding". For this reason, they tend to re-invent architectures and make them drag on for long periods of time. This is bad.

We are in interesting times. I'm predicting that the "Big Custom Architecture Each Time" guys are going to find themselves in a pinch. Microsoft (and others) are moving us into a world of "drag & drop architecture".

Microsoft's decision to embrace an Architecture Description Language (SDM - the Systems Definition Model) has the potential to overhaul the profession. SDM will likely spark a renewed interest in creating digital descriptions of our architecture, leading to an ADL competition and overall advancement of the discipline.

Digitized architecture also has a greater potential for enabling architectural refactoring, automated vendor sourcing, self-reflecting documentation and automated environment provisioning. But the thing that gets me excited is significant reduction of 'architecture time' by using enterprise-grade, off-the-shelf, canned, distributed architectures.

SOA: Business Impact Strategy

A major shift has been occurring over the last few months in the SOA adoption curve. "Business people" are now being tasked to understand the strategic business impact of SOA/Web Services in their organization. This is a significant deviation from the last set of strategy calls around "strategic SOA implementation".

Many organizations have completed the first stage of maturity. That is, they have a SOA strategy in place, a SOA methodology, a reference model, candidate architectures and core infrastructure components (registry, repository, SOAP routers & firewalls, pipe & filter mediation, service bus, SOA EII, metadata compliance, testing suites, SOAP-to-Legacy adaptors, specialized "service servers", etc.) Typically, a core team has been trained but a larger training plan is available. Again - this is in "leading organizations" - not the majority!

These leading I.T. organizations are now giving the business units the go-forward nod, indicating that they have their act together and passing the baton to the business units to make the next move. Much like we saw in late 90's around the Web, business managers are looking at the capabilities of the technology, studying the disruptions and applying it to their domain.

I'm starting to see SOA patterns of disruption emerge - but it is still early. Very exciting - more to come...

SOAP isn't Dead

The recent garbage being posted about "SOAP being dead" is just that; it is garbage. It is agenda pushing in the hope of creating a self-fulfilling prophecy. And I'm calling bullshit.

Stating that we should use a distributed computing model that fails to handshake the non-functional concerns is sophomoric. Enterprises require a new level of ubiquity for specifying the quality attributes in distributed systems - it's that simple.

I don't care what Google, Ebay or Yahoo are doing for non-critical SOA enablement. They do what they have to do to get Joe-schmoe on board. Following the e-leaders in a time where WS-* remains in a pre-ubiquitous state is just plain silly.

SOAP isn't dead. Let's all beware of the "Blog Oriented Architectures".

Layers in a Service Network

The concept of a layered architecture has deep roots in computer science. It has been applied to protocols, networks, operating systems, containers and applications. In 1996, POSA Volume I clarified the meaning of the layered architecture. The pattern states that the lower layers are 'unaware' of the higher layers. In this manner, layers are used as an abstraction mechanism to facilitate decoupling.

Although there are variations of this pattern, the most 'pure' version is where a layer is ONLY called by the layer above it. Lower layers will attempt to respond to calls from the next higher layer. However if the layer requires assistance, it can only make calles to the layer directly below it.

IBM has recently made a stab at a "partial layered architecture":



In reviewing this diagram, the obvious question is, "what the heck are those two vertical layers?" The equally obvious answer is - "those are the areas where the (horizontal) layered architecture are violated!" I feel quite comfortable saying that IBM's layers 6 and 7 gotta go (as layers). On occasion, an architecture can be modified whereby it is classified as a hybrid, but even then, the author must be VERY CAREFUL to make sure that they didn't just kill the intent of the original architecture.

Let's get to the root of any layered architecture. They all look like this:
- The very top layer is the 'Ask Only'.
- The very bottom layer 'Respond Only'.
- All layers in between are both (they ask and respond).

That said, it is pretty safe to put the human interface as the very top layer :-) To my dismay, humans still don't implement WSDL's. Until then, I'm comfortable keeping IBM layer 5 (the ultimate client).

The bottom layer is quite tricky. One might think that it would be safe to drop things like ERP, CRM and legacy apps there. However, the truth is that those applications will be the MOST LIKELY to be clients of business and technical services! Hence, I am equally comfortable killing Layer 1.

Layer 2, had me stumped. It is hard to dispute that services could sit on top of components. But on that note you could argue that components sit on top of objects, and objects sit on top of virtual machines, etc. At the end of the day, I believe that components shouldn't be depicted in the enterprise view of a layered service network. A more appropriate view of this relationship should be in the 'service view'. If a service is designed with a layered approach, it may be appropriate to show the component-to-service layering. Hence, I am equally comfortable killing layer 2 (from the enterprise view).

Layers 3 & 4 are tricky. Here is the big question: "Is there ever a time when a composite service needs to call a process orchestration?" (Will layer 3 need to know about layer 4?). I'd like to say that there will be a clean separation between process logic and business logic, but I'm just not sure. It seems as though IBM is having a tough time separating functional concerns from mechanism. The aggregation of one or more services using a loosely coupled technique (like BPEL, itinerary based routing, content based routing, etc.) will find it's way into both composite services as well as into process orchestration. And tagging flow of control logic as either 'process', 'workflow' or 'business' is extremely hard and IMHO, impractical from a layered architectural perspective.

The fact is, process services, business services and technical services WILL all leverage intermediaries. This means that they all require direct access to that layer. As much as we (Momentum) would like to break them into separate layers, we can't do it without significantly violating the rules of the layered architecture. The solution that we landed on was 'embedded layers'. Here, we create a service network view which has a pure layered approach. Then the types of services are once again layered. And lastly, the pieces of a single service are layered.

The following diagram is part of the Momentum Web Service & SOA Reference Architecture:


I believe that this diagram depicts the intent of the IBM architecture without violating the constraints of the architectural patterns.

Will AMQ Rock Wall Street?

I believe it was this article which prematurely blew the lid on a new protocol called AMQ.

Even the SOA Group is speculating....

The answer is... wait and see. I've been told by the best of sources that this project will be unveiled shortly...

N-Service Computing

Explaining Web Services, SOA and related concepts isn't always so easy. In my attempts to make these concepts easy to understand by the masses, I've landed on the term "N-Service Computing". And I'm using the following diagram to explain what I mean:



I've found that by showing this simple diagram people usually understand a few key points:
1. Service computing is an evolution of prior efforts
2. Services are the new unit of work
3. Services are part of a network

Obviously, this single slide doesn't explain everything, but I've found it to be a good introduction.

WS-LAMP

Dozens of bloggers have discussed the use of the LAMP stack as a foundation for SOA and service oriented construction. However, the "P" in LAMP traditionally meant using one of several dynamic languages like Perl, Python or PHP. Today, IBM announced their blessing of PHP and is announcing that it will become more Web service friendly. This may be the first of several steps to redefine their core programming strategy, see:

http://news.com.com/Big+Blue+backs+PHP+for+Web+development/2100-7344_3-5589559.html?tag=nefd.top

Clearly, IBM has to be very careful about what they are doing here. The last thing they want is to introduce confusion around their core Java based WebSphere products. However, IBM has excellent disciplines around milking mature products and introducing stars without prematurely killing their cash cow. It will be interesting to watch them slowly commercialize this area. They'll want to be perceived as a leader in the space (by geeks) while simultaneously disavowing the concept to CIO's until the calculus tells them that there is more profit in slaughtering the cow and having the feast - - but this is clearly no time soon.

That said we have a conundrum. Generally speaking, the "open source types" prefer simple mechanisms over complex ones. In the Web services world this means that they like "REST" not "WS-*". Now, IBM has a different view - - they fundamentally understand the value proposition of decoupling functions from their non-functional concerns. IBM realizes that mission critical systems require policy based agreement to the ilities. More importantly, they understand that the compose-ability of services increases with the ubiquitous commoditization of the non-functional concerns... and you're average open-sourcer, well, doesn't even know what that means.

IMHO, we can expect to see IBM fund the research, technology, engineering and marketing around the WS-LAMP space. This might be look like an open source play, but long term, this is an enterprise move.

SOA, Just an Application Server?

BEA's Alfred Chuang was recently quoted:

Are your new products, like Quicksilver (messaging integration software) a way to diversify your products?
It's not to diversify. It's filling the parts. At our size, we can't just sell products anymore. We have to sell both products and vision. We don't have a choice. We can't just keep selling the application server, saying, "But this is SOA. SOA is just an application server." We've got to tool it enough so that people say, "Ah, I understand."

It is good to see that BEA sees SOA as more than just an application server. What is less clear is if they intend to leverage their application server assets as the core to their SOA strategy.

I understand why BEA may decide to position the SOA battle on AppServer-Hill. They own that hill (or at least they used to). Is it possible that the BEA army will be waiting on the wrong hill for a battle that will never occur? Perhaps - or perhaps IBM will bring the battle to that hill knowing that 1. They can 2. They'll easily beat BEA. And of course, Microsoft will state that AppServer-Hill doesn't even exist.

Competitors will take the battle to a new hill, ServiceNetwork-Hill. They'll argue that clumping a bunch of tightly coupled Java API's together under session-clustered JVM's is silly. They will also be quick to note the 'coupling levels' within the J2EE platform and the fact that it was hard coded to support three configurations: one-tier, two-tier and three-tier (not N-Service). The competitors will tell you to read the J2EE blueprints and best practice guides to reinforce their point. The J2EE stack fundamentally wasn't designed to support a distributed service network. It's very name "application server" seems to reinforce it's goal. The complete lack of progress in the JSR arena on creating 'service networking' concepts indicates the strategy, resource allocation and potential adoption timelines.

It is extremely hard to make the transition from one paradigm to the next. When I was employed by a mainframe software company, it was clear to me that the client-server guys were going to kick our butts. It was equally clear that the application service guys were going to win over the PowerBuilder types. And yes, to me, it is once again clear that the service network guys will win over the application server companies.

The door has been left wide open to the startups to create the next big thing in enterprise computing. I wish you the best of luck.

Semantic Web Services

I've had several inquiries about semantic web services; I believe related to the recent predictions made by Graham Glass.

Personally, I'm a big fan. However, I'm thinking that the adoption curve of RDF, OWL and semantic ontologies is pretty far out. The use of first order predicate calculus applied against a structured knowledge base is an excellent idea (in peer to peer mode, when based on 'web of trust'), but again - this is quite a ways out as well. However, with Microsoft moving aggressively with XML in their products and now offering their "Information Bridge Framework", we will begin to see more emphasis on structured information being related by "exact match". By this I mean that the XML schema becomes a 'pivot point'. So, an XML schema that talks about "company X" may have several 'pivots': Company X's Partners, Products, Employees, etc. Thus the user will have the ability to 'navigate structured information'. With the inclusion of semantic ontologies, there will be less emphasis on having the pivot points being exact matches only. Eventually, the pivots will be navigable by synonym, hyponym or hypernym. This is extremely exciting capability, taking office productivity to a new level.

Most organizations tend to think of their web services and SOA efforts as "back office" functionality - a way to connect legacy systems. I am a firm believer that we will see a huge upside when we connect smart services to end user productivity tools. Like most people, I spend most of my day in a handful of tools: e-mail, word processing, spreadsheet, etc. Empowering users through better information is good business.

For those of you who've never queried a semantic ontology, give this a try:
http://www.cogsci.princeton.edu/cgi-bin/webwn
Make sure you look at hypernyms and hyponyms. If you're real interested check out the book "WordNet". I also recommend looking at the Cyc effort.

UPDATE: The Road to the Service Oriented Enterprise

For the last few weeks I've been traveling the nation talking to corporations about their transition to the Service Oriented Enterprise. I've met some great people and listened to their problems.

It has been good for me to get out of the newsgroups, the blog-o-sphere and the intellectual community to go see what Joe Developer is up to. Here are some of my favorite quotes:

"Jeff, it was only a few years ago that we quit writing programs in Assembler and started writing them in COBOL."

"Development methodology? Yea, we have a development methodology - write the code real fast and put it in production."

"We are picking an EAI product to implement our service network."

Here are some other highlights:
- There was a huge gap in knowledge between architects and developers. The architects knew the buzzwords, had read the white papers and most of the developers were completely clueless on next-gen stuff. Developers were soooo busy learning new J-stuff (maven, hybernate, canoe, etc.) that they just didn't have time to worry about the WS-stuff.
- Managers and directors knew the buzzwords but didn't know how to cost justify or create a roadmap.
- People want to implement a "web services catalog" (think UDDI registry - perhaps even slimmed down).
- Most architects had heard of ESB's and were actively shopping for one.
- Many of the customers were explicit about NOT wanting to work with startups on their infrastructure. Their short list was usually: IBM, BEA and Tibco.
- The fundamentals of designing systems for use, reuse, evolve-ability and agility were missing. Developers don't need WS training, they need loose coupling training.
- The role of 'business analyst' or 'requirements analyst' seemed to be missing in many organizations. Use cases were NOT rolled into business cases.
- In large corporations, the majority of the employees had never spoken with their CIO, never seen a strategic I.T. plan (annual or otherwise) and never had the chance to talk with anyone of any influence about the fundamental problems in their organization.
- I.T. budgets are returning.
- Morale is mixed. Some are happy to have jobs others are upset about their career potential.

Conclusion
The potential for solving problems is high, but unfortunately many organizations have a huge gap between leadership and worker-bees. We need a major upgrade in 'business and I.T. alignment' before we move too far down a new paradigm shift like the Service Oriented Enterprise.

WS Questions...

I've posted a couple questions out at:
http://groups.yahoo.com/group/service-orientated-architecture/

Here's a condensed version, with some additions:
1. Should a WSDL be produced for the ultimate sender? If so, should it be stored in the registry? If not, why not?

2. If "Contract First" is considered a best practice, should it applied to ultimate senders in addition to ultimate receivers?

3. Should a WSDL editor create two WSDL's at once (one being the invocation document advertised by the sender, the other being a service-side WSDL)?

4. Imagine that you have two WSDL's: one representing the ultimate sender, the other representing the ultimate receiver. The outbound interface on WSDL #1 matches the inbound interface on WSDL #2 (an operation signature match). Should the 'component-service' composition environment visually enable a 'snap together' programming model?

5. Let's call the thing that components and services snap into the "SCB", a variation of the PCB (printed circuit board). Should the SCB dictate the flow-of-control in a proxy like fashion (like bpel)? Should the SCB act as a message router focusing on endpoint resolution?

These are important questions. As one WS-Luminary told me, "composition is the killer app". Can I be any more blunt? ;-)


-----------------------
Definitions
Ultimate Sender = the caller, the consumer, the client, etc.
Ultimate Reciever = the service, the producer, the recipient of the call, etc.
Contract First = the practice of stubbing out functional interfaces and non-functional policies prior to writing the implementation.

Compliance Oriented Architecture

RedMonk did a nice piece on the intersection of SOA and Compliance Oriented Architecture.

Good going! I agree that SOA is a good technical foundation for meeting compliance needs and that addressing compliance as an architectural requirement is an excellent idea.

Components and Connectors

I liked this statement:

The C2 architectural style is primarily concerned with high-level system composition issues, rather than particular component packaging approaches [3,5]. The building blocks of C2 architectures are components (computational elements) and connectors (interconnection and communication elements). This separation of computation from communication enables the construction of flexible, extensible, and scalable systems that can evolve both before and during runtime. This style places no restrictions on the implementation language or granularity of components and connectors, potentially allowing it to use multiple interoperability technologies for its connectors.

See: http://sunset.usc.edu/~neno/teaching/s99/NR98.pdf

It is also interesting to see C2 and REST presented as alternative architectural styles. I really like the idea of architectural description languages that are constraint based. Section 2.3 of the Web Services Architecture document is wonderful. Oddly, the other portions are out of place.

Service/Relational Mapper

Need a little help here...

I've received a few client requests for what I call, "service/relational mappers". In essence it is the same as object relational mapping but with web services. Here are the typical requirements:
1. Be able to both send and receive WS-I compliant web service messages
2. Ability to visually map an in-coming or out-going web service messages to a relational database schema (insert, update, delete, select)
3. "service oriented cron" - ability to wake up every so often, check the database and turn the new records into SOAP messages for outbound delivery.
4. Maintain a log of the activity
5. Send appropriate exceptions / errors

The key here is simple service to relational database mapping (DBMS vendor neutral).
If you've used a product or products that you are happy with, please send me a note:
jschneider@momentumsi.com

He's got Google-sized balls!

Holy ding-a-lings!

Adam decided to ring-a-ling his jinglers on Christmas day!

Encapsulation, Polymorphism, Inheritance

Greg Vaughn, weighs-in on the "four tenets":

"I mean no offense to those who are trying to nail down the fundamentals of what makes an architecture “service oriented”. For some it is a very handy way to organize their own knowledge. And there’s certainly a need to have laws for any developers on a project who have not reached the required maturity and experience levels, but in my opinion these should be more project specific rather than broad architectural category laws."

Here is the problem. Greg is one of the smartest developers I've ever met. For him, the concepts of SOA are obvious. He's a top 1% guy. The problem is the other 99%.

Remember, "Encapsulation, Polymorphism, Inheritance"? These were the basic ideas that the OO guys were trying to teach. They may not have hit the nail on the head but they did provide guidelines for developers.

Laws or rules aren't childish. They are the nourishment that leads us out of child-like naiveness.

Sun Launches the first pure “service oriented operating system”

Sun Launches the first pure “service oriented operating system”

PRESS RELEASE
On January 1st of 2006, Sun Microsystems is scheduled to release the world’s first service oriented operating system called SunStorm. The operating systems will be released under an open sourced license agreement, free to the public.

SunStorm is based around a new model of computing known as “web services”. Early versions of operating systems were primarily developed in structured programming languages like C and were often exposed to applications using ‘objects’. According to Sun, the internals of the operating system will continue to use these highly efficient techniques for things like disk operations and memory management, but the top layer of the system will now be completely service oriented.

SunStorm also has incorporated most of the capabilities of their last generation computing platform known as “J2EE”. New web services are provided inside the operating system to provide messaging, management, database connectivity and other “enterprise grade” functions. When asked about potential performance concerns, Sun representatives showcased their dynamic web service optimization techniques. Similar in concept to the “HotSpot compiler” release years earlier, SunStorm has the ability to dynamically determine the most performant mechanism to enable services to interact and in certain cases to recompile themselves into a single service which is later cached or deleted.

Sun is also rolling out their new tagline, “The Network is the Application”. This tagline replaces the old line of “The Network is the Computer”. With the launch of SunStorm, Sun is initiating a new era of loosely coupled, network computing for the enterprise.

Interestingly, Sun has not decided to enter into the development market for this space. Having learned from earlier mistakes made in their iPlanet group, Sun is now focusing on building out the operating system and supporting it for their enterprise customers. New to the Sun organization will be an emphasis on professional services.

Apparently, SunStorm has been years in the work only recently emerging from it’s stealth status. As part of this effort, Sun has end-of-lifed both it’s Jini and JXTA technologies. The J2EE technologies have been placed in a ‘mature’ status and will continue to be supported.

================================================================
This press release is 100% false. I completely made it up. However, it is really sad that it is 100% false.

The Four Tenets of SOA

As we near this Christmas holiday, it seems only appropriate to throw the bible into my opponent's face :-)

Consider the following:
'You shall love your neighbor as yourself.' - (can, shall, may, etc.)
'Don't bang your neighbor's wife' - (don't, shall not, etc.)
Subtle difference? I don't think so. Imagine the first "rule" without the second "rule". Talk about a HUGE clarification!

You see, leaving room for interpretation leads to problems. That's why the Four Tenets of SOA are utterly useless (even as philosophies). Don't do X; Don't do Y. Don't do Z. Simple isn't it? Constraints. Ok, hopefully I've established the difference between philosophy and constraints. That said, let's go back and take a look at the four tenets that Don Box provides us peons to guide our professional existence for the next decade:

Boundaries are Explicit
Point: Because each cross-boundary communication is potentially costly, service-orientation is based on a model of explicit message passing rather than implicit method invocation.
Response: This is the "distribute OO was a failure" rule. Rather than saying
A. Distributed OO was a failure because Microsoft screwed the CORBA guys and fragmented the space.
B. Distributed OO was a failure because CORBA was a pain in the ass and it was competing in a rough market (BPR with 2-tier RAD and RAD groupware).
C. Distributed objects was a failure because we didn't adequately factor out the non-functional concerns and network programming was in its infancy.

No, rather than saying A,B or C - Microsoft says "Distributed objects failed because programmers were so stupid that they couldn't tell the difference between local and remote calls. Their solution was simple. Make remote calls a pain in the ass (oops, I mean ... make them non-primitives.) Yes, create some sort of language add-on (an API?) that mandates a mapping layer between the internal type system and the 'generic' type system - that'll teach those stupid programmers to realize when it's local and when it's remote! Oh, but let's also build runtime optimization of local service calls into the Indigo stack just to mess with them!!!
Sweet.

In case you're wondering, the constraint for 'boundaries are explicit' is this:
"language designers and extension builders SHALL NOT create a programming extension or primitive that enables service calls to be or appear to be first order primitives." (This might be the rule, but I can't wait to break it!!!)

The second part of 'boundaries are explicit' is: "The notion that boundaries are explicit applies not only to inter-service communication but also to inter-developer communication...." For the life of me, I can't find a rule, philosophy or anything actionable in this one.

==================================================================

Services are Autonomous
This is a case of bad terminology and description (I believe). If I understand Box correctly, his real intent was this line, "services are almost always deployed atomically". Whew... that was hard. Let's cut to the end and find the constraints:
1. Newly versioned services MUST BE deployed in a manner that DOES NOT negatively affect the prior version.

The second part of 'services are autonomous' states: "Service-orientation encourages a model that increases ubiquity by reducing the complexity of service interactions."
Here, Box takes jumps out of the versioning issue and lands in the 'loosely coupled' issue... although, in perhaps the most ambiguous manner possible. All I can offer is that if you want loose coupling to be a tenet, than you have to make an effort to create an index on loose coupling . Saying that 'leaking intentions is bad' doesn't quite do it for me.

==================================================================
Services share schema and contract, not class

Box states, "Rather, services interact based solely on schemas (for structures) and contracts (for behaviors)." Schema and contract only? Crap, we have to kill SOAP with Attachments and DIME ;-)

I've stated before... I don't have a good answer for this one... the problems with this tenet are clear: sharing binary information, passing predicates, bypassing the schemas with optimizations and sharing instructions other than classes like DSL's and metalogic.
==================================================================

Service compatibility is determined based on policy

Again, I agree.
Rule: Service consumer MUST be able to determine the capabilities of a provider via a publicly accessible, standard policy.
==================================================================

Now, I know that Don Box didn't mean for these to be rules... just some high level philosophies. However, if you watch the MS video, you'll see the MS-band-of-jokers butcher his material.

Back to my original point; we need simple rules or someone might accidentally misunderstand things like 'love your neighbor'. It's all about constraints. :-)

"Schneider... you ignorant slut!"

Like every other morning... I woke up, made coffee and read my emails. One email caught my attention; it was titled, "Schneider... you ignorant slut". Clearly, this one deserved immediate attention :-) Yes, it was another 'anonymous' WS-CTO laughing at me for getting BlogSlapped by Jef Newsome.

Well, I thought I was going to make it all the way through 2004 without a good whipping. Thank God for people like Jef. Block, block, kick, block, punch... here we go...
======================
Jeff said: "All exchanges of data, metadata, logic or other binary asset MUST BE exchanged through a contract. Runtime changes to code dependencies are NOT allowed."

Jef commented: "Boundaries are explicit" is about the fact that you can't see what kind of underpants I am wearing."

Agreed. To me a 'contract' is a combination of the interface and the policies. The contract should black box both functional and non-functional requirements. The failure to block box non-functional requirements was the limiting force of components and the primary driver to move to services. However, neither of us answered the "boundaries are explicit, except for predicates" problem. (Not that I expect us to, but people like MS can't just punt on it either.)
=======================
Jef commented: "I have absolutely no idea what "Runtime changes to code dependencies are NOT allowed" has to do with explicitness of boundaries. In my estimation, nothing. I can think of a number of ways that runtime code dependencies could change in the context of a service oriented system, and everything would be just fine."

It was an example of a constraint - perhaps a bad example, but an example of a constraint. My point was that we need a constraint based description language for the architecture; I wasn't really trying to write the architectural constraint language. The constraints will be probably number in the hundreds, not four or six. That was my real point.
======================
Jef says, "I believe that services *should* be autonomous, that their implementation dependency on other services is an implementation detail (because boundaries are explicit), and part of the responsibility of an autonomous service is to gracefully degrade service when its dependencies have availability or other issues."

It sounds like you believe that a service is autonomous as long as the 'boundaries are explicit' rule is enforced; if that is the case, then this is a duplicate rule and we should remove it. I'm game. I've been game - it sends the wrong message.

I firmly believe that enterprises will eventually be required to migrate to Model 4 services. Yes, their boundaries will be explicit, but their lack of sovereignty or self governance will be more significant. Services will NOT be viewed as single individual entities, but as ecosystems of contracts that need to migrate together.

As we move from noun-first programming to verb-first, we find ourselves dealing with "cross-cutting subjects" (not concerns) that have to be dealt with as a group. What I have found is that the 'verb' holds still while everything else changes (noun, adjective, predicate, context). This isn't to say that you can't have autonomous services; you can. However, I don't believe this approach will work from an economic perspective. I'm probably talking more about the 'development-time' aspect of autonomy than the runtime aspect; however after re-reading the MS description, I have no idea what they were talking about. Their description is largely just a bunch of mumbo-jumbo. They mixed a half dozen concepts under this one tenet that largely don't even relate to each other. If you want to turn "services are autonomous" into a set of constraints, I'd love to see them.

======================
Jef: And I trust he can explain what he means in a way that someone as simple as me can understand. I hope he does, because I don't get it.

My goal isn't to explain it in the simplest fashion, but in the most precise fashion. I want to take the current state of WS-MumboJumbo and bring clarity via a set of constraints. A tenet should be a set of constraints rolled together to satisfy a high-level architectural requirement. Walking into the next paradigm of computing without constraint based design practices is silly, insulting and financially dangerous.

Service Creation Styles

At Momentum SI, we've been working on an EA framework for producing reference architectures and candidate architectures. This work has an emphasis on SOA but also considers many of the other architectural remedies.

One aspect of an architecture is the style that is used to create/execute a service. Although we've seen many variations, here is our high level categorization scheme:













Note that this doesn't cover registries, configuration management, version control or other essential part of the architecture. Those remedies are covered in other areas of our SOA reference architectures.

The Death of Software?

Paul Brown had commented on a note from Eric Newcomer on the "death of software":

"The most significant inventions are over. Some would call this the "death of software." But only as we know it. Software will continue. But we are not likely to have any new languages, or see any significant new inventions. Twenty years ago no one knew what a database was, or middleware, or Java. But now I think innovation like that has stopped because IT doesn't need it any more."

Oh me, oh my. I've been there before - when you just run out of steam and everything looks grey; progress halts and the future dims. The good news is that history tells us that this is dead wrong.

Where to begin? Software remains in a pre-natal stage (not even infancy). I still think is hysterical that I type on a keyboard, look at a fixed monitor, and have to tell the computer what I want it to do. I'm amazed that I am so much smarter than my computer - this is just silly. I hate the fact that the state of artificial imagination is at ground zero, that we don't have digital metaphors and that a computer can't reverse engineer strategy. I think it is funny that we continue to use silicon at the computing substrate, that our programs are explicitly programmed and that they are not self-improving.

Maybe I'm just a kid at heart. Maybe I'm too stupid to know the obstacles. Either way, I am thankful. May I continue to be cursed with creationary optimism.

And may I offer an ounce of optimism to those of you who wore black pants and white shirts today. When your desk is scattered with employee status reports, when your walls speak of posters of UML and Java libraries from 5 years ago, it is time for a change. Not a little change, but a big change. Web services, orchestration, intermediaries, blah, blah, blah - these are the incremental improvements decades in the work. Refill your mind with childlike optimism. Imagine. Invent. Destroy. Laugh. Repeat.

Stuck in the Middle

This is from a slide deck that was put together in 1997:
The [I] box stands for 'intermediary'.


See: http://www.almaden.ibm.com/cs/wbi/Publications.html

Intermediary-based programming is still an art. Everyone wants to talk about services... service oriented this, service oriented that... but no one want to talk about "intermediary oriented". I guess it isn't very sexy. I've challenged the SO group to discuss the NFR's of an intermediary; should be interesting. I've also had some interesting discussions on the categories of intermediaries, including stateless, stateful, 'context oriented', header-only processing, payload processing and more.

I have a feeling that a significant focus of 2005 will be on the intermediary programming model. We'll see great discussions on IOD (design) and IOA (architecture) and refactoring portions of fat services into bumps on the network.

Charcoal Services

If a black boxed service sits in a grey container, what is it?

The Financial CIO

A great quote from a recent Sterling-Hoffman newsletter:

Angel Mehta: Why do you think the climate is so difficult for enterprise software companies, even with the economy having recovered?

Stu Schuster: For as long as I can remember, the process of growing a company has been the Geoffrey Moore, ‘Crossing the Chasm’ approach. When taking technology to market, find the innovators first who buy the technology, and then you go through the early adopters and late adopters, etc. This process, I think, all good high-tech marketers understood.

Throughout my entire career, there were always people who wanted to do something for their organizations that they believed would be a leap forward and would make a name for themselves. Call these the entrepreneurial Chief Information Officer… they were at companies like FedEx or Wal-Mart. They would try to do something with technology that would give the entire company a competitive edge.

After the technology crash, the focus became so oriented around cost-cutting that the entrepreneurial CIO was replaced with a financial CIO. There are so few entrepreneurial CIO’s out there that today everything has to be easy to implement, available by the drink, with very short time to ROI. The innovative buyer, the entrepreneurial CIO has been terminated out of the industry.

Eventually, it’ll cycle again because people will eventually feel that they’ll need a competitive advantage and will look to technology to do that. But today, it’s brutal. It’s amazing how hard it is to find people who are willing to take a chance on a new company or new technology. Fear dominates every IT department. As a result, growing an early stage software company is just harder than ever.

That’s why I place so much more emphasis on the people-side of the equation these days. It doesn’t matter how great the technology is – if the right people aren’t in place, you’ll never convince customers to take a chance.

The Role of the Application Servers in SOA

I've been using the term "service containers" to refer to the sandbox that services live in. However, in J2EE land, there isn't a 1:1 relationship between the 'software server' and the 'container'. You've got the 'servlet container', 'ejb container', etc. The Application Server is really an all-in-one - super-duper server. It slices, it dices, it does transactions, messages, web pages - you name it - it does it.

App servers can host web services. They can also host platform specific services (RMI, EJB). The services provided can be governed via the container and can leverage additional functionality via components and class libraries (most of the J2EE API's).

Just to restate - build services on top of components that are a combination of classes which sit inside of a governed container while making calls to libraries, some of which are runtime generators, perhaps being fed metadata from model-driven tooling. Agreed?

Ok. Now we need to move into "specialized servers", like identity servers, orchestration servers, etc. Many of these servers were built from the ground up as stand-alone servers, while other implementations were built to sit on top of application servers (e.g., a bpel implementation that runs in JBoss). In the J2EE world, specialized servers are really just new API's governed by the JSR process to encourage yet another container or library to be added to the J2EE server. In some ways, J2EE has become a dumping ground for 'bright ideas'. Hence the movement to 'lightweight containers'.

Now, let's ponder...
Question #1: Would you describe the application server as a "Rapid Service Development" (RSD) environment?

Question #2: To what extent should specialized services reside in generic containers?

Question #3: Should the server be 'pre-glued' together (mandate the download all 55MB of libraries) or should it be more loosely coupled and segmented?

Question #4: What are the necessary cross-cutting concerns that should be considered part of the server or container?

Question #5: What functionality is not a cross-cutting concern, but rather a specialized function?

Question #6: Would you consider the app server to be a "Contract First Container"?

The application server has evolved to meet the needs of the masses. It was designed to be an all-purpose server. Can a developer build SOA based applications in it? Sure. It is more work than it needs to be? Absolutely. And the big question... Do I believe that Rapid Service Development to Contract-First Containers is the way of the future? Yes I do.

Application servers will continue to be used when the architect chooses to use a platform specific component-based development model. Here, the focus is on 'building the application' (hence, "application server") versus building the services.

There are some real challenges that app servers will face in the SOA environment. We can be sure that BEA and IBM will be morphing their offerings to make them more acceptable to the SOA crowd. However, I fear that they will leave all of the garbage behind as well.

It's real simple - if your platform fails the loosely coupled test then so will your business applications that sit on top of them.

Remember: "Rapid Service Development to Contract-First Containers".