Thursday, January 02, 2003

Decoupling the Enterprise Data Model

ToDo:
service access to rdbms data
surrogate keys
GUIDS

Wednesday, January 01, 2003

Reverse Engineering Coupling Levels

An obvious extension to having a coupling rating system is having tools to automate the rating. Some of the newer web service management tools allow one to create an inventory of the services on the network. These tools are designed to help manage what you currently have rather than optimize your go-forward design. Design and architectural issues in web service infrastructure will first surface at the macro level.

In our recent architectures, 'ilities' emphasized responsiveness, availability, scalability and reliability. The 'ility' (non-functional requirement) of choice in the SOA will be Agility. The materials and the structure that we use to build our systems will have the opportunity to be nimble. Agility is not automatically granted in an SOA; it must be part of the plan.

Monday, December 30, 2002

Update the Coupling Rating System

When you ask a software person about coupling, they will usually tell you that it comes in two flavors: loose and tight. Other developers will tell you that it comes in 5 flavors (Data, Stamp, Control, Common and Content). Doug Kaye, will likely tell you that coupling is a continuum based on attributes. I believe they are all right - and that is part of my problem.

The art of coupling will likely rise to be the single most important design factor in the Service Oriented Enterprise. Unfortunately, the art of coupling (and cohesion) remains rather primitive. I think that we all would agree that Doug's recent effort to identify categories of coupling attributes was a great start. And by cruising other blogs, it is apparent that others have improved on some of his concepts.

As we approach a new year, it seems like an appropriate challenge to Doug (and friends) to create a coupling rating system. I'm aware that some of this exists in the academic settings (mostly on cohesion), but it is time to bring it to the masses. I'll gladly lend a hand and I'm sure that our Loosely Coupled community will chip in as well. And all good things take time - I propose that we iterate through it a few times, let it linger, demolish, rebuild and on January 1st of 2004 we publish a 1.0 of a new coupling rating system.

Well?

Saturday, December 28, 2002

The SOAP Firewall - Level 3

It would appear as though many of the functions of a "transport firewall" would serve the purposes of a service oriented architecture, however an additional layer will likely be necessary. Let's assume that a transport firewall has two primary levels of protection:
1. Packet Filerting (TCP/UDP) Level (IP Source, Port, etc.)
2. Protocol Intrinsic Command Level (FTP-PUT, HTTP-GET)

First, we can assume that most firewalls will be able to identify SOAP messages (Protocol Intrinsic Commands), thus allowing those messages to pass through.
In addition, filtering based on SOAP contents will be needed. The SOAP firewall will be required to look at the ws-routing information and determine if the SOAP routing information is acceptable (valid source, valid destination, valid intermediary). The firewall may (or may not) use additional information (beyond IP address) to identify a valid trading partner. In this setting, the SOAP firewall is acting as a first line of defense (bastion).

Note: The SOAP Router will act as the bastion when the message contains bogus routing information.

Note: A level 4 SOAP firewall may determine if the service / operation is acceptable for the given trading partner.

Note: Scoped out of the SOAP firewall are individual service security devices; these will be placed close to the service (hash, authenticate, access control). In this setting, we end up with distributed security - macro at the firewall, micro at the service.

Guaranteed Hops, Guaranteed Routes

Guaranteed Hops
While travelling down a route, a SOAP message may use several different underlying "Guaranteed Hop Protocols", that is, protocols that either succeed or fail in moving a message from one node to another via a single hop. Examples of Guaranteed Hop Protocols include HTTP and SMTP. Now, neither of these protocols are considered reliable in that that they don't keep re-trying until the message is successfully transferred. Rather, the guarantee isn't to succeed, the guarantee is to to tell you which one happened (success or failure). Also note that the emphasis is on moving the message forward a single hop rather than the entire route. In Web Services, WS-Routing is used to faciliate moving the message the entire route, while Hop Protocols (usually guaranteed) are used to move from one node to another.

Guaranteed Routes
If I have a multi-hop route and all hops have the ability to provide a guaranteed hop, can the requesting client be granted a guaranteed route?
Answer: No. A SOAP router is a stateless device that does not predefine a message path or create a virtual circuit. Thus, the initiator is not aware of the level of guarantees or reliability of the SOAP routers in the path. Thus, the fact that all the routers in the path have the ability to provide guarantees does not give the initiator any guarantee. In addition, the WS-Routing specification clearly states that the return of a fault is optional in section 5.2:
If there is a reverse message path present in the faulty WS-Routing message then the WS-Routing fault SHOULD be returned to the initial WS-Routing sender. If the message path is not bidirectional then the fault message SHOULD be discarded.

The key here is the use of the IETF "SHOULD" - meaning, "No, the route is not guaranteed. However, if you and all the routers along the path implement faulting AND you specify a return path, a guarantee can be expected."

Side Note: I can't figure out why they didn't upgrade the SHOULD to a MUST... one reason might be on the lingering questions around non-guaranteed hop protocol bindings - as stated in section 7.2, "Editor Note The UDP binding described in this section is preliminary and is likely to change before requesting a well-known port from IANA."

Cross-Domain SOAP Routing

SOAP routing is essential to a service network. That is, a service engine should be able to drop a SOAP message on the grid and it should magically get to its destination. If a destination IP address is specified, why wouldn't it reach its destination? One reason is that the IP address is, "non-reachable". This is to say that something prevents the direct addressing of a given node on a network. Typically, this is the result of a firewall, proxy or NAT. In some cases, large internal networks may have been broken down into smaller domains to make them more managable or to control traffic. In other cases, devices like firewalls or present to secure networks. Either way, the problem of non-reachable nodes surfaces.

This is where a domain-based SOAP router enters the picture. The SOAP router takes on the responsibility of finding non-reachable nodes - enabling a SOAP message to reach its final destination even when the sender didn't know the actual address. In virtually every business to business web service interaction, we can expect that the participants will NOT know the physical address of where the other guys services are located. And why should they? A company should be able to move its servers around and not worry about having to update the trading partners on the new URI's or IP addresses. A cross-domain SOAP router enables a corporation to maintain an internal routing map for services, thus abstracting the 'service consumers' from the physical network.

Friday, December 27, 2002

The Differences Between Local and Remote Calls

Mr. Jini and friends knocked out a nice white paper in 1994 about the differences between local and remote invocations:
http://research.sun.com/techrep/1994/smli_tr-94-29.pdf

Saturday, December 21, 2002

Web Service Presentations

I just ran across a good powerpoint:
http://www.almaden.ibm.com/u/mohan/WebServices_TES2002_Slides.pdf



WS-Duopoly

WS-Duopoly is an alias for the stronghold that IBM and Microsoft have on Web Service specifications.

Many view this "arrangement" as a good thing; Microsoft and IBM have been fast tracking web service standards in order to reinvigorate I.T. spending. Others view it as a duopoly trying to crush second tier players.

SOAP Router Homogeneity

After reading the ws-routing specification, it appears to me that people will interpret the composition of a SOAP Router in very different ways. The specification requires the implementations to conform to a minimal amount but also encourages alternative uses. Additional functionality that may be included in a SOAP Router that are beyond the ws-routing scope include: message filtering, message transformations, content based routing, load balancing, response-message caching and others. Actually, the inverse is more likely to happen: participants in an SOA will incorporate routing functionality (e.g. SOAP Load Balancers will incorporate routing functionality, etc.)

The 'good side' of this is that people will create new servers (like content based routers) that are single-purpose utilities and leverage the basic infrastructure of the ws-routing specification. The ability to drop routing functionality into the server facilitates the pipeline service pattern. This, in my humble opinion, is where the magic of a service oriented architecture is created.

There are a few potential downsides to not having an actual specification for a SOAP Router. One is that many of the functions that a router can perform are very dissimilar (domain routing vs. content routing). By keeping a server homogenous, you are able to more accurately predict the response to changes in the router settings (e.g. routing logic based on complicated algorithms, etc.) Also, from a marketplace perspective, it is hard to comparison shop for a 'SOAP Router' when the functionality can swing so drastically - although, one could argue that this is what slow down product commoditization.

Either way, when vendors begin releasing SOAP Routers, it may be worth asking, "So, what exactly does your router do?" And perhaps one day, the WS-I or another organization (the ws-duopoly) will create a SOAP Router Profile.

UDDI and the SOAP Router

WS-Routing describes a way where you can send a message from point A to point B with *minimal* concern for the path that was taken.

It has occurred to me that the entries in a public or private UDDI server *should* point to the SOAP router. Thus, UDDI is responsible for all the usual stuff (aggregating service descriptions, providing 'publish' and 'find' syntax, etc.) but it is not responsible for identify the actual address of the service provider, rather it is responsible for providing the address for a SOAP Router that is aware of one or more applicable service providers.

*minimal* - You still specify a 'from', 'to' and potentially a 'via' in the route description - thus, address based routing.

Friday, December 20, 2002

Message Correlation

So, I have been trying to figure out how asynchronous, reliable messaging in web services will occur at the wire level (not api). This seems like an easy enough task, however my small brain has been in overdrive trying to crack the nut.

Part of the problem seems to be the ambiguous use of the term 'message correlation' throughout the various specs. I'm now of the opinion that at least three levels of message correlation exist:

Correlation at the Business Process / Web Service Orchestration Level
Imagine a WSO engine is running a single orchestration script, however there are 50 instances of this 'process' in play. A WSO engine must be able to take an in-bound message and correlate it to the proper instance of the process. An example of this is described in the BPEL4WS script titled, "Message Correlation".

Correlation at the SOAP Router Level
In a service network, documents are passed across enterprises. Company A shouldn't care about the physical location of where Service X is running inside of Company B. This abstraction takes place in the SOAP router. It magically gets a SOAP document to the right place by using internal routing tables and acts as a proxy between domains. So, a service network will send a document from A to B and potentially use multiple go-betweens (aka, intermediaries, SOAP routers, etc.) as the transportation vehicle. Along the way (or at the end) a fault may occur. If this happens, the service network must be able to backtrack and send the fault code back to the originator. This means that it must tell each Router that, "I am message #968, please pass on that I failed." In this context, message correlation is used by an intermediary to *dynamically or statically* create a reverse path for the purpose of fault notification.

Correlation at the SOAP Server Level
If SOAP Server (A) sends SOAP Server (B) a message (M1) in an asynchronous fashion, it may want a response (M2) back (e.g. an answer to a request). When (B) sends (M2) (the response) to (A), (A) must be able to determine that (M2) correlates (is the response) to (M1). It appears as thought this type of message correlation also is taken care of by ws-routing (although I'm not sure). Unlike many network protocols, a service network must be able to send asynchronous, bi-directional & correlated, reliable messages without utilizing a virtual circuit. The downside of this is that it means that any initiator or receiver must be able to sniff the routing path to determine where to send the message back. This seems to violate the distincition between SOAP routing and SOAP serving, but it appears as though it is a necessary evil. Thus, the reverse message path in ws-routing not only sends fault information but also provides the necessary information for a SOAP server to send an 'application-level reply' back to the initiator.

* Note that SOAP 1.2 clearly states that 'message correlation' is out of scope for SOAP.

** One other item of interest is that "retransmission" information is NOT sent in the ws-routing information. In theory, you would need this to increase reliability. One could guess that this was intentionally left out and in the ws-* fashion, reliability will become its own spec (think ws-reliability or ws-sla).

WS-* Roadmaps

Overall, I have been impressed with the detail of the ws-* specifications. In general, they are detailed, consistent in terminology and provide some level of examples. However, the overarching document seems to be missing. This would be the specification that puts all of the specs into a single context. Now W3C has an architecture document that is coming together nicely, however it addresses the architecture at more of a conceptual level. The IBM/MS duo are in need of a ws-* roadmap. It would show how the pieces connect to each other, define the common glossary, spell out what is done today, identify what is out of scope, etc. Now, I know that 25 different people have all made best-effort stabs at these maps - but we need an official one from the source.

The ws-security team actually did a nice job of putting together their local roadmap.

Wednesday, December 18, 2002

What's in a SOAP Router?


SOAP Router =
Message correlation service +
Message forwarding (proxy, alternate path routing) +
Transport protocol swapping (tunneling) +
Router configuration (think telnet kind of stuff) +
Durable queues for store & forward +
Invoke service (when there is no "to" element) +
Activity logging +
Regular SOAP server stuff including security






An Asynchronous Landing...

It's late... I've read too much on asynchronous reliable messaging with SOAP. So, here is where I've landed:
1. People are going to use HTTP for everything on the outside of the firewall
2. You can break 1 synchronous call into 2 asynchronous call as long as you have a message correlator.
3. It looks like WS-Routing will provide wire syntax for correlation - thank you MS for an early impl, too.

Tuesday, December 17, 2002

Terminology: "Orchestration" and "Choreography"

I found an interesting email thread on some terminology that I thought I'd share.


From: "Sanjiva Weerawarana"
To: ,
Date: Wed, 14 Aug 2002 14:18:35 +0600
Subject: Re: "Orchestration" and "Choreography"


I'm not sure whether there's such a clear distinction between
the terms. During the lifetime of the BPEL4WS document, it was
at one point called WS-Orchestration, then WS-Choreography,
then WS-Business Process and and eventually BPEL4WS. I can tell
you that the name changes had nothing whatsoever to do with
technical distinctions between the words .. it was all marketing
and, um sometimes, politics ;-).

To me orchestration, choreography, business process, workflow
are all ways of indicating how to take a bunch of things and
put them together to do something meaningful. We also use the
term composition for the same thing .. BPEL4WS is a language
for composing a set of Web services into another service. The
thing that makes it a workflow language is that the composition
primitives chosen are those that are well-known in the workflow
domain.

Edwin's distinction seems to be the difference between what
WSFL called global models vs. flow models. I think that distinction
is definitely valid (BPEL4WS doesn't yet handle global models,
for example), but I don't think the terms choreography and
orchestration distinguish between those two.

Coordination is different IMO; that's really distributed
synchronization (rendezvous).

Hope this helps,

Sanjiva

----

Later, I found this email which goes into some working definitions that the w3-arch group considered.

Monday, December 16, 2002

JMS as an Asynchronous, Reliable, Message Passing Vehicle

I was reading where some people were wanting to use JMS as a vehicle for asynchronous web service communication. This made me go back and re-read the spec. It was as I thought, JMS is a predefined API, but leaves the wire protocol up to the vendor. Thus, the app dev person should be able to swap out JMS providers (vendors), but in all likelyhood, no two vendors calls will work with each other - thus, it isn't an option for B2B reliable messaging (nor was it intended).

APP 1 --> JMS API -->
[[Proprietary wire protocol]] -->
[[Server]] -->
[[Proprietary wire protocol]] -->
JMS API --> App 2

Nigel Thomas reminds us, "JMS implementations from different vendors are not required to interoperate - details of wire format and transport protocol are left to the provider's discretion. So JMS usage is limited to 'single provider' situations - where the developer controls both producers and consumers.

JAXM is designed to support inter-business messaging. Based on Simple Object Access Protocol (SOAP) 1.1 with Attachments, message formats, payloads and transport are standardized so that the other business will be able to receive and understand them. Unlike JMS, JAXM only supports the point-to-point messaging domain - not least because B2B messaging is usually one to one. The high performance, low latency and programmatic flexibility of JMS are traded for simplicity, reliability and interoperability. "

Hmm... better go back and read the JAXM spec. From what I remember it was a rather wordy API that didn't actually spell out a SOAP binding.

Asynchronous, Guaranteed Web Service Protocols

Most people seem to agree that web service orchestration will primarily occur through an asynchronous, document based architecture. One should also assume that some level of guarantee is required (the package arrived in whole, the package arrived intact, the content of the package makes sense, etc.)

I'm hoping that someone has done a comparsion of potential wire protocols for delivering asynchronous, SOAP messages in an open format. If so, please drop me a note.

Compare / contrast the following wire protocols as potential vehichles for delivering asynch. SOAP messages:
- BEEP
- HTTPR
- AWSP
- SMTP
(or others that I may be missing, for now I'm not interested in vendor specific or platform specific stuff)

SOAP 1.1 binding to BEEP:
http://www.ietf.org/rfc/rfc3288.txt

SOAP 1.2 binding to SMTP:
http://www.w3.org/TR/soap12-email
Note that the SMTP binding has a field for message correlation.

SOAP 1.2 binding to SMTP Illustration:
http://www.w3.org/2000/xp/Group/2/02/emailbinding.html

AWSP to SOAP (not really a binding...) :
http://www.trans-enterprise.com/technology/aws/awsp.html

Other:
Generic SOAP binding framework (proposal):
http://www.w3.org/2000/xp/Group/1/10/12/Binding_Framework_Proposal

SOAP binding to HTTP:
http://www.w3.org/2000/xp/Group/1/10/11/2001-10-11_Framework_HTTP_Binding



Friday, December 13, 2002

Another PhD shows off his stuff!

Hard to believe that the person that posted this message goes by the handle, " anotherphd " :-)

"SOA breaks the OOP model
With all the talk of the so-called benefits that SOA brings to the table, I am shocked to see how everyone is missing the glaring fact that SOA breaks the pure OOP model by removing the concept of objects altogether.

It is my opinion that this will be the very thing that prevents SOA from succeeding.

From: anotherphd Date: 10/22/02 "


It is clear that some people are still having a hard time understanding what a Service Oriented Architecture is....

SOA Papers

A number of people have written whitepapers on Service Oriented Architectures.

Here are some of the ones I've run across:

- Service Oriented Architecture: A Primer, by Michael Pallos
- Service Oriented Architecture - Introduction, by Michael Stevens
- The Benefits of a Service Oriented Architecture, by Michael Stevens
- Web Services: Pathway to a Service Oriented Architecture?, by Gregor Hohpe
- The Evolution of Web Applictions into Service Oriented Components with Web Services, by Steve Burbeck

Sunday, December 08, 2002

To a carpenter, it looks like a hammer...

I seem to be bitching a lot lately - mostly about items written about web services by really smart people. As I look back, in each case I see that the author is bringing their baggage with them - OO baggage, business improvement baggage and most recently database baggage.

Doug Barry is a smart guy. He recently published a white paper on, "Getting Ready for a Service Oriented Enterprise Architecture". I got excited by the title - it was the first time I saw someone other than me use the term SOE. I dove into the paper! Right off the bat I see Barry telling a story of a message oriented world - I'm loving it! Then, it starts to go down hill - he moves into using a "database of record" - hmmm... he's talking storage. But then he takes it up a beat and starts talking about EAI, but refuses to use the term EAI and instead refers to it as a "Data Router"??? Now I'm suspicious - then, he moves into how object databases are better and the one from Versant kicks ass. UGGGGG.... I felt like the kid Ralphie on, "A Christmas Story" who just found out that the message for the secret decoder ring was really just a commercial for Ovaltine. Doug, if you want to create yet another commercial for Versant or the object DBMS, just title your paper, "Why I am the King of Persisted Objects!" or something like that.

Doug obviously realizes that he's doing this and comments:
"For those readers who know of my background in object DBMS's, you are probably saying something about how I am beating the same old drum concerning object DBMS's. Although this may be true, I am trying to point out a specific use that should make sense to almost any organization that has a need for middle-tier persistence." Enough said.