Open Source ESB - Apache Synapse

It appears as though the Apache ESB project, Synapse, is moving forward. According to InternetNews, Blue Titan, Infravio, WSO2, Sonic and Iona are all supporting it.

For those of you who aren't familiar with ESB's, Microsoft does a pretty good job of describing one in their position paper. They basically say that an ESB consists of:
- Orchestration & Transformation
- Intelligent Routing and Mediation

The heart of the current code base was donated by Infravio. However, it was quickly noted:

Sonic Software itself an ESB vendor (as well as an initial committer to the Synapse project) said that Synapse wasn't a full blown ESB. "This project is related to ESB , but it is not in itself an ESB," Dave Chappell, Sonic Software's VP and Chief Technology Evangelist told internetnews.com,. "What Synapse bring to the table is a mediation framework that allows users to get in the middle between service requesters and providers and perform various tasks – including transformation and routing and that helps to promote loose coupling between services "

So, if I understand Dave correctly - Synapse has the routing and mediation but not the orchestration and transformation. Sounds like a problem! Perhaps I can help?

As of today - I am open sourcing our Orchestration IDE, BPEL engine and Transformation system.



But what good is an open source ESB if you can't get decent support? None! Sounds like a job for MomentumSI.

Done. Stay tuned. More to come.

Mark Baker Hates Architecting First?

Mark, "I won't rest until you REST", Baker kicks a bit of sand on architecture first.

Oh stop it Mark - you architect first, too.

MomentumSI recently finished a large transactional web system for a customer. The customer was a startup - and had NOTHING. Guess what - before writing a bunch of code, we gave them a "Web Architecture"; we talked to them about the ilities (scalability, integrity, availability, maintainability, etc.) We recommended the infrastructure: firewalls, routers, load balancers, operating systems, web servers, web analytic engines, etc. We also recommended some design patterns and some platform choices (J2EE, IoC/PoJo, SOA-IoC/PoJo, .Net, etc.) Then we talked to them about implementations. My point is that these are things that the company didn't have - and needed before a SINGLE LINE OF CODE WAS WRITTEN.

If you're an organization that is going to do SOA - you should go through a very similar exercise. If you haven't answered the SOA infrastructure questions - or architectural configuration questions and you just start building services you'll end up in a mess. It isn't: "you may end up in a mess" or "you might end up in a mess" - it is "you WILL end up in a mess". Closing your eyes to up-front architecture is a bad idea in web systems, SOA systems or just about any system you can think of.

So, in summary:
YES - "Projects and enterprises should define their Service Architecture FIRST". If you're going to do service based systems - do it within a basic architecture. This doesn't mean go off and architect until you're blue in the face. It means lay down the basic architecture - we did it for web systems - we'll do it for SOA systems.

Tim Bray Supports WS-*

Via Nick Gall.

Tim Bray did an interview and has asked his readers to draw their own conclusion, which I have: Sun customers have had to invent their own distributed computing solutions - on their own dime - and have created proprietary solutions. AToM tried to tell them for years but had little luck.

Questions and Answers with RouteOne
Q: How did you do Routing / Addressing?
A: We went proprietary - because we had to.

Q: How did you do Reliability?
A: We went proprietary - because we had to.

Q: How did you do Orchestration?
A: We went proprietary - we can't wait to get off of it.

Q: Do you put anything standard in the header???
A: Yes, WS-Security

Q: How do you do messaging?
A: Vendor specific; ubiquity via platform API - not protocol

--------------------------------------------
Now it is possible that I misunderstood the point that Tim was trying to make... but if I read it correctly, he has a customer that wants protocol ubiquity. The customer has real problems (transactions, correlated messages, reliability concerns, security concerns, performance concerns). He had to roll his own protocols (boo) and stated that he could have gone faster if he didn't.

I'm so happy that Tim found an enterprise customer that had real issues. Perhaps with the acquisition of SeeBeyond Sun will become a WS-Shop ;-) Gee - having a few protocols implemented sure can change your tune!!!

The funny thing is that my friends at IBM and Microsoft NEVER talk about SOA specs anymore. That's so 2003. Perhaps we should debate JBI (or not).

Service Network Newsletter

The August edition of the Service Network Newsletter is available.


Enjoy.

SOA Boot Camp

After months of work, we're finally kicking off our SOA Boot Camp. MomentumSI consultants will be immersed in 17 days and nights of non-stop SOA education!

You name it - it made it: registries, repositories, governance, mediation, orchestration, composite applications, SO security, SO-BAM, legacy enablement, WS-I & WS-*, Indigo, Axis, SOBA, ESB, SOA reference architectures, SOA reference models, SOA process / methodology, SOA best practices, service analysis & design, XML message design, reuse metrics and more.

We have also signed up many of the leading SOA companies to provide vendor specific training on their best-of-breed products by their local gurus. After we run it internally a couple times we will be taking SOA Boot Camp to our customers.

SOA is a fundamentally different computing model - we've realized that the 2 and 3 day training just doesn't cut it. Let's hope that 17 gets the job done.

MS Composite App Strategy?

Web Services enable composition through lowest-common-denominator ubiquitous protocols.

Composite applications pull together Web services to fulfill some set of requirements. If you compile the services together (bake at 450 degrees), you'll find that the agile-controller isn't so agile.

So, all of this said... what is the Composite Application Strategy for Microsoft? All the work on Indigo was great, but where is my 'service join' facility, or my 'agile controller' - for that matter, where is my 'semantic adapter'?

From one perspective you could say that SDM represents a portion of the strategy - but it only reflects the 'service oriented bill of materials'. BizTalk? Wow - I hope not... any insight?

If you're not familiar with 'composite applications platforms', take a look at:
http://www.cordys.com
and
http://www.aboveallsoftware.com

Hacktivity Level

The 'hacktivity level' is way up. Casual services, dynamic scripting and remixing are clearly the theme for 2005. JSON, POX, REST, RSS, etc. have taken hold - they've intersected with client-side contextual applications like Google Maps to create what some people would have called a composite application - you may prefer the terms remix or mash-up. Even Business Week is taking notice.

SOAP & WS-* have turned into the stiff-collared, corporate world, slow to move technologies. Once again, our passionate technology community is breaking all the rules and creating compelling applications - and I love it.

Hacker - 'I need that service and that other service. I need to combine them. I need a server-side aggregation engine. I need to re-serve the combined service. I wonder who will enrich/transform my service? Who cares. I need a client-side visualization mechanism. It has to have easy API's that front-end the data services. Damn this is cool. Interesting - it looks like a 'web of services'. Clients. I need contextual pivoting - Maps - Austin - Concert Halls - Concerts - Reviews - Tickets - Buy - Receipt - Calendar.'

CIO - "Did you see what some hacker did with concert reservations? Can we provide our core services to the outside world? Wait - can we consume other peoples services and create our own application out of them??" - Why yes.

The World Wide Web is turning into the Service Oriented Web. The technologies will be stiff and flimsy; the artists will be straight-laced and no-laced. The intersection will be magnificent; the Web is reborn. The SOAP/WS "transactional web" will be a portion of the new SOW, perhaps a small portion. Our future will be driven by those who have the passion and desire to create it.

Hey Don Box,

All of us in the SOA blogosphere have appreciated your input and insight. However it has come to my attention that Microsoft isn't practicing what they're preaching. I'm hoping that you can tell me that my information is dead wrong.

I ask one simple favor. Start blogging about how Microsoft is using SOA internally. I've been told that Microsoft spends more money on SOA evangelists than they do on building SOA solutions internally.

I'll extend this challenge to every single Microsoft blogger. Don't tell me about your latest spec or library - tell me how Microsoft uses it internally. The Microsoft I.T. department should become the SOA showcase.

Respectfully Submitted,
Jeff

I, Too, Have Seen the Enemy

Charles Garry comments, "I Have Seen the Enemy and It Is Complexity". He writes,

"Web services is yet another example of potential complexity headed your way." ... "I have perused a number of books on the subject, but the one thing I never see discussed is how one makes a virtual application highly available. What are the implications for backup and recovery of data? How do we manage security permissions and authorizations? Clearly, with the added flexibility, we have also created a new set of obstacles to overcome."

He goes on to state:

I suspect we will continue to see a growing trend towards infrastructure rationalization (fewer kinds of things) and consolidation (fewer numbers of things) as a means to reduce complexity. This trend is both useful and necessary if we have any hope of utilizing some of these new computing paradigms.

Charles believes that Web services increases complexity. By no stretch of the imagination do I consider myself an expert in complexity, chaos, systemics or related concepts. However, I believe that the conclusion Charles reaches is incorrect.

His assumption is that we are adding new elements to a set - and as the set grows, so does the complexity of the set. His answer is to reduce the number of elements (rationalization and consolidation) and this is where we differ.

Enterprise software is going through a 'specialization' and 'externalization' movement. We are taking pieces of code that seemed unique and abstracting them into reusable concepts. We do this with design patterns, architectural patterns, domain specific languages, platforms, libraries and specialized engines. Each time we see these reusable items, we 'write them down' (or externalize them). The complexity in the system already existed; we merely found the pattern, factored out the common substrate and documented it. Externalization is a well-known mechanism for reducing complexity; you take that which was not understood and make it understood.

As we take our mountains of unintelligible 3GL code and move much of it into common infrastructure, we increase the amount of 'known' computing elements and decrease the amount of 'unknown'. This process has an odd effect. As the inventory of known elements grows so does the concern for 'having too much'. This leads to a human fear of 'large-set-complexity'. And the natural reaction is, yes - you guessed it, rationalization and consolidation. This is a human tendency whereby man attempts to diminish complexity by hiding it. We take those First Order computing elements that made our inventory list and reduce them back into 2nd order elements, thus removing them from our list. And somehow, we feel better that we defeated the enemy.

Specialization of vocabularies, models and elements is a natural progression of every science. However, specialization must be followed with abstractions, taxonomies and containment. In essence, we must not hide our new found computational elements, rather we must manage them.

Web services are an additional element to our computing infrastructure. From one vantage you can claim that they are just another element to the set increasing the complexity. However, Web services will help to reduce systems complexity by adding a new abstraction over our computing environment. Although the total number of elements increases, we find ourselves commoditizing those lower in the stack and working with those that are closer to the business value.

The enemy of SOA isn't complexity; it is a lack of understanding. We must not retreat from our sophisticated computing infrastructure. Instead we must use cost effective means to manage the piece-parts, identify the abstractions, create common groupings and externalize the system.

Crush My Head Like an Acorn

Sean McGrath has suggested that the Reed's Proposition may be closer than Metcalfe's Law in describing a theoretical upper limit value function on a Servcie Network.

For those of you who haven't had the pleasure of meeting Sean, his physical presence reminds me of the "boulder thrower" on Braveheart:



I firmly believe that Sean could crush my head like an acorn with his bare hands. Hence, I'll be careful to disagree with him.

Acorns and noggins aside, as usual Sean makes a great point. Metcalfe's law is an over-simplification of a complex formula. In my humble opinion, so is Reed's.

My goal is simple - I want the enterprise to begin looking at their services as a network and determining the value function that works for them. Today, virtually no formulas are applied.

It is true that some services will provide more value than others. And some business processes are fundamentally more important to competitive advantage than others. Services that support key processes will provide greater value. We will also see that 'service gaps' will destroy the value of 'process networks', suggesting that some services only have value when bundled with other complementary services in the same value chain.

Value chains are created around: customers, suppliers, products, employees, etc. We create networks that link these entities using various levels of constraints. Those links which require minimal constraints are usually deployed first and are the most free-flowing and collaborative in nature. From a computing perspective these entities will be joined via well known networks including WWW and RSS. The value of low-constraint, high collaboration networks is well known.

As we increase the level of constraints and diminish the ease of creating self-forming networks due to semantic mismatches, protocol impedance or the mandate to fulfill non-functional requirements, we find ourselves creating services with a lower ROI. This isn't because the "R" (return) has changed; it is because the "I" (investment) has changed.

Enterprise Service Networks will not produce an adequate ROI if they are not able to reduce the 'service drag', lowering the investment. That is, it is the burden of the enterprise to employ 'service oriented infrastructure' that facilitates rapid service creation and composition by removing the aforementioned obstacles. In essence, the value function must look forward in time, forecasting a future value of the network, less the 'drag' related to adding clients and services. This is what I call the AOL Argument; fundamentally the Web created a more fertile environment for adding content to a network than did AOL.

Corporations use valuations that are forward looking vehicles (DCF, EVA, etc.) I would argue that the value of the network is less of a snapshot in time but rather a forward looking forecast based on the potential growth of the network as it directly relates to the value chain of an organization. I view the Metcalfe/Reed concepts as an easy way to introduce I.T. personnel to network valuations. However, due to their inability to capture forward looking results and correlate node-to-value concepts, they both remain inadequate at best.

Please don't crush my head like an acorn ;-)

The Value of the Service Network

Just a quick reminder...



We aren't just building Web services.

We aren't just creating an SOA.

We are building a network of business services.


We must also remember that it takes MORE work to create VALUABLE services.

Sun's Big Year - 2186

Based on historical performance of SeeBeyond (income statement), and including new synergies (sales force, engineering), I estimate that Sun should break even on their $387,000,000 cash investment in the year 2186. I realize that it is tough to wait 181 years for the ROI to kick in - but I'm sure Scott (and his great grandchildren) will be patient - as will Wall Street.

I wish Sun the best of luck getting JBI adopted.

I should mention to the M&A crew at Sun - - I heard a hot rumor! A COBOL company is up for sale!! Time for JSR 38,940 - JCOBOL! You'll make BILLIONS!

The Service Network - July 2005

Our July edition of The Service Network is available.

And yes, the favorite line seems to be:

"SOMA (Service Oriented Modeling and Architecture) –Vaporware or IBM Global Services with a copy of Visio?"

When will SOA take off?

In 135 days.

Seriously - the Web service and SOA era will begin in 135 days.

176 I.T. Jobs Posted in USA

In the last 24 hours, 176 I.T. and software related jobs were posted on Monster.com...

I was curious how many were posted in India however it was difficult to tell. Even when limiting the search engine to only return jobs posted in the last 24 hours, the number posted exceeded the limit. I attempted to further tune the search criteria but was still hitting limits.

My best guess is that in the last 24 hours approximately 3,000 to 4,000 jobs were posted on Monster.com for India.

After sampling the job descriptions, it appears as though approximately 70% of the jobs posted in India were designated to fill U.S. demand. If this math is correct, that means that for every software job requirement that is posted in the United States, fourteen are posted in India to fill the same U.S. demand.

AToM: Sun Needs to Learn From Microsoft

We have a clear winner in the quote of the month category:

Anne Thomas Manes, a Boston-based analyst with Burton Group Inc., echoed those sentiments.

"What bothers me is that I really don't like JAX-WS," Manes said. "The Sun JAX-WS team really needs to learn a few things from Microsoft. They should be building something like Indigo—a common programming model that encompasses JAX-WS/JAXM, JMS, RMI and EJB. But Sun doesn't get that."

Can someone please send me the name of the person at Sun who is responsible for this? (Yes, I'm still interested in seeing prediction #7 of 2004 come true... how sad.)

The WSDL is the Offshore Contract

As more and more organizations move large portions of their I.T. department to offshore facilities they are questioning which activities to keep on-shore and which to push offshore.

For those that have chosen a service oriented path, the decision seems to be clear: "The WSDL is the Offshore Contract". In other words, domestic employees will work on business processes, enterprise architecture, orchestrations (agile controllers) and service design (up to the WSDL). And building the service? You got it - offshore.

It's an interesting line to draw, yet perhaps an obvious one. Using the "software contract" as the "legal contract" invites a clean division of labor and facilitates a simple approach for acceptance testing.

Obviously - the WSDL only defines the external interface leaving other aspects of the service (LIKE THE INTERNALS) to be described by other mechanisms. The algorithms and business logic are described with variations of the Use Case (think 'service case'). The relationship between the server and other elements (the hosting container, other services it calls, etc.) will likely be described by the System Definition Model, or some variation.

The science of describing architectures, interfaces and dependencies has significantly increased in recent years and it appears as though we are entering an era where the adoption level will skyrocket. The ability to describe systems in this fashion is what I've been calling 'precision specifications'. By using digital definition languages like SDM and WSDL, we are able to provide a 'software factory' with clear directions for construction.

Using 'services' as the unit of work appears to be a very manageable approach. Companies that are big into test-driven development will likely find test-driven-services a natural extension. Organizations that have adopted the Agile Manifesto may find this world too rigid. However, the Extreme Programmers may like the ability to lump X number of services into an iteration plan. And project managers will be thrilled to see acceptance tests directly linked back to line items on the Gantt chart.

I.T. organizations must increase their capability to describe systems with precision specifications prior to initiating a 'SOA factory'.

CIO's Aren't Taking SOA Serious

I'm going to make a quick prediction:

More CIO's will lose their job over SOA implementations than lost their job over ERP implementations.

---

I was chatting with a CIO the other day and asked him, "In retrospect, do you think you were involved enough in the early stages of your ERP implementation?"

He answered, "In retrospect - no - I wasn't involved enough. Had I known the size of it I would have definitely gotten more involved."

I followed up with, "Are you aware that an enterprise SOA roll-out will be significantly larger than your ERP implementation?"

He started laughing; he thought I was joking. My face didn't change. He quit laughing. "Jeff, are you serious?"

"Yes, I'm very serious. SOA is a complete overhaul impacting how systems are analyzed, designed, built, integrated and managed. And not just some systems - all systems including packaged applications like ERP."

He responded, "But with ERP, it was an all-or-nothing approach. Doesn't SOA offer an incremental approach - one that can be managed better?"

"Yes and No", I responded - "SOA is first and foremost about a network of services. The value of your service network is directly related to the effort of creating/obtaining services and making them available. All networks require a critical mass of 'valuable services' before they become useful to the consumers. The SOA service network is no exception. What I'm saying is that you can take a slow and steady approach to SOA but be prepared to receive gains that are even slower. Remember - this is really about the 'network effect'.

SOA done properly is a massive undertaking. It requires a rethinking of your infrastructure, development methodology, business impact analysis, budgeting process, organizational design... Don't underestimate the value it provides, the competitive advantage one can assume AND the investment it will take!"

It was clear that I had surprised my friend - perhaps even scared him a bit. And I realized that so many of the people in this space (vendors, analysts, press) are doing a disservice to the CIO by failing to tell the whole story: An Enterprise SOA implementation will be larger than ERP, Y2K and the Web.

Now is the time to get involved.

SOA Training!

For the last couple years we've been training people in SOA and Web service technologies. Recently, we've added a couple courses including "SOA for Architects" and "SOA for Managers" - both excellent.

See:
http://www.momentumsi.com/services/training.html

I'd love to do a course on buidling your SOA Reference Model...

Hiring SOA Consultants...

Momentum is opening up new positions for SOA consultants. All positions are travel intensive and require a strong background in distributed computing and enterprise architecture. Interested candidates should send Melissa a note: careers@momentumsi.com

We're basing the consultants in one of three delivery centers:
- Austin / Dallas, Texas
- Reston / Washington D.C.
- San Francisco / Silicon Valley
------------------------------------------------------

The Service Architect
At Momentum, the Service Architect is responsible for analyzing architectural requirements and translating into service based solutions. The architect will work with various existing architectural styles (monolithic, client/server, 3-tiered) and modify those systems to accommodate a service based approach. The architect will also be responsible for identifying SOA based solutions to the non-functional requirements (security, transactional integrity, reliability, availability, agility, etc.) Lastly, the Service architect will be instrumental in creating Software Architecture Documents, identifying components, estimating time-to-transition, providing guidance to designers and developers and linking architectural investments back to financial returns.


Requirements:

The SOA Architect must be well versed in Enterprise Architecture, Service Oriented Architecture (SOA), J2EE and or .NET Architecture.
Full life-cycle implementation experience (requirements, design, implementation, integration, testing).
Experience designing and implementing n-tiered application architectures.
Fluency in first generation web services standards, technologies and tools (e.g., XML, SOAP, WSDL, UDDI, BPEL, etc.).
A strong distributed computing background is a requirement (CORBA, RPC, DCE, DCOM, RMI).
Familiarity with messaging systems is a plus (Tibco, WebMethods, MQ, SeeBeyond).
Familiarity with commercial Web service products is a plus (Systinet, Blue Titan, etc.).
Knowledgeable on WS-* standards.

Other:

• Strong interpersonal, strategic oriented thinking and excellent written and verbal skills are essential.
  
• Ability to present professionally and persuasively to C-level executives.
  
• A minimum of 10 years experience overall in software development.
  
• A degree in Computer Science is preferred.
  
• Travel is required for this position.
  

About MomentumSI
MomentumSI is a pioneer in service oriented architecture, integration, and application development. Since 1997, the company has built a reputation as a leader in architecting large scale distributed computing solutions for IT organizations. Today, MomentumSI is at the forefront of service oriented architecture and integration for enterprises. We built the first Web service orchestration server for .NET and have been an active member of Web services standards groups, such as OASIS. Momentum continues to invest our SOA practice and is looking for the best of the best to join us.