Composite Application: Def #1

Phil at Loosely Coupled was kind enough to send me a definition of "composite application":
 
================================
Here’s Loosely Coupled’s definition:
 
An application built by combining multiple services. A composite application consists of functionality drawn from several different sources within a service oriented architecture (SOA). The components may be individual web services, selected functions from within other applications, or entire systems whose outputs have been packaged as web services (often legacy systems).
http://looselycoupled.com/glossary/composite%20application
 
It’s a favourite term of Gartner’s Massimo Pessini, the man who also gave us ‘ESB’
================================
 
I had great conversations on this topic recently with some really smart people. Unfortunately we were all rather drunk and I don't remember what we concluded - although I think it had something to do with molecules. Ok, this in only partially true. We basically agreed that the definitions of composite apps was still too loose (SOI, SODA, integration at the glass, how many of the NFR's need to be addressed by services, how much of the app needs to be SO enabled, etc.) Bottom line is that we need to move to a 'definition by constraints' view.
 

ToDo: Create Agility Index

On my list of things to do is to create an agility index (and a fragility index). Before I go too far down the road I thought I'd post to see if anyone already has a favorite agility/fragility index in use. Lemme know.
 
Oh yea... so it isn't about being service oriented. Service networks must be evaluated from an 'ility' perspective. I'll start with agility.

Nice paper

Google led me to this nice paper by ACM Fellow, Carlo Ghezzi:
http://www.elet.polimi.it/upload/ghezzi/_PRIVATE/paper.pdf

Composite Apps?

I just caught this from one of the Oracle guys :-)

"The focus of this new blog will continue to be around BPEL and the design of composite applications." (Collaxa)

Has anyone defined "composite application" or determined the criteria for an application to be considered "composite"?

If so, please send me a note - or just post it.

The economics of integration has already changed.

The economics of integrations have already changed.

NEW YORK, July 2 (Reuters) - WebMethods Inc. (NasdaqNM:WEBM - News), whose software links different applications, expects a wider quarterly loss than previously forecast, citing an "unusual number" of large deals that did not close by the end of the period, it said on Friday.

Its stock tumbled 33 percent, or $2.74, to $5.60 on Nasdaq, its lowest level in more than 18 months.

Consider the effects of service oriented integration. The game has already changed. I'll repeat this - the game has already changed.

Fun Facts

Today was a slow day in India.

Only 163 new jobs requiring Java development experience were created on MonsterIndia.com - today (of the well over 5000 current openings).

Most of the positions were paying around $12,000 USD annually.

Today was a slow day in India. Perhaps things will pick up tomorrow.

How many languages do you use?

I love asking Java developers the following question, "How many languages do you use when developing a J2EE application?" They almost always look at me with this pitiful look. It's the, "Jeff is so old that he doesn't even know that J2EE uses Java" look. I'll usually string them along while they try to describe what J2EE is. The answers are great.

After they finish 'educating' me on Java, I follow up with the following set of questions:
1. Is HTML a language and do you ever use it on your J2EE applications?
2. Do you use a 'batch' or 'make' language on your apps?
3. Do you ever use the 'structured query language' (SQL)?
4. Do you ever use JavaScript?
5. Do you ever use JSP?

Usually I stop there - by this time they understand my point.

J2EE is a combination of languages and libraries. We use Domain Specific Languages (DSL) all the time. In the Java world we tend to wrap them with 3-letter acronyms that start with the letter 'J'. We often glue the our languages and libraries together in the form of an application by using an object oriented 3GL (Java).

In many cases, our libraries (JMS, JDBC, JNDI, etc.) merely act as a standardized API to a server (or set of services). These services often need to be customized. We have several ways to customize services: upload metadata, pass in parameters, etc.

J2EE is our container of DSL's and libraries. It acts as the vehicle to pass information from one DSL to another DSL, from one library to another library, from a DSL to a library or from a library to a DSL. The Java language is used to type the data, transform it, perform Boolean logic and pass the data on.

The questions that I've been asking myself are:
What should a library/DSL integration language look like?
Is Java (or any OO-3GL) the best fit for a library/DSL integration language?
What common functionality of the DSL should refactored into a generic 'DSL engine'?
How does a platform architecture provide more consistent extensibility mechanisms across the suite?
To what extent should the components of the platform be mandated to have a symbiotic relationship with the other members?

Back to the original question, "How many languages do you use?" The answer is usually 5-10, and the number is growing. Writing J2EE applications is becoming less about programming in Java and more about mastering domain specific languages, libraries and integrating them across contextually bound Use-Cases.

J2EE has grown organically over the last 7 years. It has not had the opportunity to be massively overhauled. In my humble opinion, the enterprise software community is ready for a massive refactoring of J2EE; one that isn't dragged down by backward compatibility issues. I believe that we are ready to incorporate our lessons learned to design the next-gen software development platform.

Beyond Services

I've been actively writing about services (and mostly web services) for three years now. We've moved from the base specs, to the WS-* functionality, to BPEL, Service Network design, Service Oriented Integration and Service Oriented Development of Applications.
However, the one thing that has been clear to me for the last year is that services will play an important role in enterprise architecture, but will only be successful when combined with other architectural concepts.

In the past, I've thrown out terms like, "Model Driven Services" or "Model Driven Metadata" and "Metadata Driven Services". The fact is, most "web service" companies have been so busy trying to knock out the ws-* spec implementations that they have not had time to think about the converging programming model. But they must!

After the WS-dust settles, we will have solved our LCD problem and hopefully even our GCF problem. What is less clear is how service based development will interact with many of the popular programming concepts (eventing, aspects, code generators, process engines, policy systems, rules engines, etc.) It is relatively easy to create a new specialized brand of computing (like services). It is much harder to weave it with the concepts that are already in place. Failure to do so results in a massively disjointed architecture. The symbiotic relationship between our architectural components must grow.

Dynamic Coupling

Loose coupling has more up front costs than tight coupling. The assumption is, over the life of a component (or service) that the loose-coupling device will pay for itself.

A common belief is that software engineers should design services to be 'loosely coupled'. That is, they should verify that the service can speak the 'least common denominator' and do it in a manner that mitigates the impact of location: attempt to minimize the impacts associated with distributed computing (location transparency).

In the physical world, we use devices to decouple entities. Instead of soldering two components together, we connect each component to a plug and then use the plugs to connect the items together. This is necessary in the physical world.

In the logical world, we have the opportunity to attach a 'plug' to the end of a component or a service. And just like in the physical world, it costs more money to buy and attach the plug. But, the plug should make the component more reusable. That is the investment.

Imagine an electrical component that is attached to a printed circuit board. The component has nice plastic plug coming off of it enabling other components to easily connect to it. In addition, the wire lead on the component is exposed and it is mounted in such a way that additional wires can be soldered directly to it. In essence, the component was designed with loose coupling in mind (the plug) but the instantiation (or the use) of the component allowed the engineer to bypass the plug by directly connecting to the wire lead.

Here, the component has both 'component design-time' and 'assembly design-time' properties. The 'component design-time' properties describe the features that the component has; in our case, we are interested in the plug at the end. The 'assembly design-time' deals with the decisions that the engineer makes at design time when they assemble multiple components together (e.g., does the engineer use the plug or solder directly to the wire?)

When we discuss loose coupling or tight coupling, we must remember that there are different issues to consider:
1. The level of investment that was placed on enabling a component to be loosely coupled (e.g., it comes with a plug attached). [POTENTIAL COUPLING]
2. The decisions that the component assembler made when combining the piece components (e.g., they used the plugs everywhere vs. soldered them together everywhere vs. some combination) [ASSEMBLY COUPLING]
3. The option of delaying the assembly interface mechanism decisions to runtime. [DYNAMIC COUPLING]

A significant amount of literature is available on making components/services reusable (creating high potential decoupled devices); thus I won't revisit. Less is written on Assembly Coupling, but I'm still happy with my, "Service Coupling Index" as a foundation. However, very little information is available on the concept of Dynamic Coupling.

In many ways, Dynamic Coupling contradicts the 'loosely coupled' manifest. Rather than stating that something must be implemented in a certain way (asynchronous, document centric, XML based, etc.), it states that the component need only have Decoupling Potential AND that the component be aware of any Dynamic Decoupling Device.

I like to think of the Dynamic Decoupling Device as a "magic printed circuit board". Here, the assembler inserts components into the board. But rather than soldering the components together or using plugs to connect them, the assembler will never execute the actual assembly mechanism. Instead, the assembler tells the printed circuit board which components need to be connected to each other. The "magic printed circuit board" is now responsible for querying the components and determining the best mechanism to connect the piece parts together. We have moved from explicitly stating the connection mechanism to only stating the intent.

In the physical world, creating the "magical printed circuit board" is... well, challenging. In the logical world it is quite easy. At the core, it means:
- We use policies to describe capabilities and interfaces.
- We publish the policies in a consistent manner.
- We perform protocol negotiation at runtime.
- We build components with high decoupling potential.
- We build components that were designed to connect to Dynamic Decoupling Devices.
- We quit hardcoding our assembly mechanisms.
- We mandate the use of Dynamic Decoupling in assembly.

If an architecture is indeed defined by the constraints rather than the features, than the aforementioned may serve as an architectural style that facilitates reuse without giving up performance.

The Suckline

The other day I bumped into an old friend (Tim) that specializes in Java and database development. He asked what me about my interests and I told him that I have been working with web services. He said, "Oh, aren't those slow?"

Sure, I had 101 smart-ass answers loaded in my arsenal, but this individual was asking a serious question that deserved a real answer.

I answer him, "Well, I guess it depends. Generally speaking, anytime you use distributed computing techniques you will incur extra overhead - and that overhead could lead to poorer performance or response times." This answer clearly annoyed him. He responded, "Yea, I know that you use 'distibuted computing' to distribute computing and the effects that this has... my concern is that web services leverage a bunch of abstract protocols that just weren't designed with performance in mind."

"Ah... I get your point. The protocols suck?"
"Yes - the protocols suck - if you have any concern for performance."

Ok - now we were getting somewhere.
"Tim, the reason they suck is because the big-boys (IBM/MS) chose to roll out 'Least Common Denominator' (LCD) solutions to distributed computing first. When you need to create an ad-hoc computing platform and you don't know the capabilities of all of the participants you must choose only those capabilities that are guaranteed to be available across all nodes. You have to create an LCD baseline (which I call the suckline). The suckline is the decision to use the lowest common denominator capabilities across participating nodes in order to guaranteed communications."

Tim nodded in agreement and commented, "But what you implying is that there is something else out there that doesn't suck - - like a Greatest Common Factor (GCF) solution?"

"EXACTLY - - but there isn't! And that my friend, is the beauty of it...(laugh)..." He looked at me like I was insane - which is a look that I thrive on. "Web services are about protocol negotiation. Currently, we only have one solution for just about any facet of web service computing - encoding, encryption, reliability, transactions, discovery - you name it - we have a version of it that more or less sucks! And you know what - that's ok! IBM and Microsoft have had the patience to work on an LCD platform for as long as they have - and in the face of 'performance critics' - I tip my hat to them. If the protocol negotiation is done correctly you should be able to use a metadata described invocation and interface description mechanism that is linked to a protocol policy for runtime resolution."

Tim asked, "So you believe that it really is just an issue of 'premature optimization'"?

"Absolutely - two communicating java applications will be more efficient at sending serialized java objects then they will be at sending and receiving XML documents. If the applications are on the executing on the same vm, it would be more efficient if they used some kind of shared memory. The GCF capabilities of the participants MUST be determined dynamically (or cached). Should the two applications be prepared to go down to the suckline? Sure - the suckline represents the ubiquity baseline of modern computing."

"Jeff, you just crossed from distributed document oriented to distributed object-oriented computing. I thought that web services was all about document oriented, XML based, loosely coupled, asynchronous..."

"No! - and yes..." - I injected. "This is where I disagree with most of the world. If you ask most people that live and breathe web services... they would agree with your comments. However, I'm not one of them. To me, web services describe a programming model that is evolve-able. The SINGLE most important feature for me is the ability to use protocol negotiation to determine the instance of the programming features and functions that you deem necessary to create a dynamic virtual computing platform. The document oriented problem is really a fatty-chatty problem - and there are solutions to this as well."

Tim commented, "That's cool. But we still have the problem that today web services are slow and we are stuck with the suckline."

"Yep - you are correct. Microsoft has already moved Indigo towards a GCF programming model. IBM is doing the same. What is unclear is if a generic J2EE type solution will become available (in time) to compete against the MS solution. Otherwise, you will see a splintering in the next-gen Java SOA marketplace with only 1 or 2 vendors agreeing to agree on the GCF solution. And yes, it is very possible that the GCF solutions will have patents hung all over them."

"Tim, I can't predict the future, however - I bet on smart people and funded projects. Right now MS, IBM and others are putting their best people on web services and throwing lots of cash at it. Trust me - we will move way beyond the suckline - there is too much money on the table."

The Dallas Duo

I just realized that the Dallas Duo are blogging:

Glenn Vanderberg and Greg Vaughn

It seems as though Greg is hanging out in the ladies restroom, while Glenn is off discussing complicated topics :-)

No blogging from Billy, Patrick or Tom? Bummer.

The J2EE Verbs

Here are the verbs of the J2EE API:

acknowledge, activate, add, after, are, cancel, clear, clone, close, commit, compare, connect, contains, convert, create, delete, destroy, do, does, encode, execute, exists, find, flush, forward, get, handle, include, init, initialize, insert, invalidate, is, load, log, next, on, passivate, pop, prepare, previous, print, publish, push, put, read, read, receive, recover, refresh, register, release, remove, request, reset, rollback, run, send, service, set, start, store, truncate, unset, unsubscribe, update, validate, was, write

Here are my 'kernel verbs':
Level 1: do
Level 2: get, set, insert, delete, allocate, deallocate, validate, resource.peform, run, locate and receive
[warning - this is first pass]

I'm still working on level 3. It will probably look like a cleaned up version of the J2EE verbs. I'll remove the synonyms and change some of the state oriented verbs into nouns. All level 3 verbs are just combinations of level 2 verbs. Creating level 3 will likely shake out my level 2.

I'll take a look at the .Net verbs as well. I already did the assembly language verbs - I thought it would be interesting but I failed to gain much insight; odd eh?

Process Visibility as a Service

Should 'Process Visibility' be a service?

It has been my experience that most people who look at BPEL solutions want a combination of the following:
1. A way to combine multiple web services into a single service (composition).
2. A method to explicitly define the flow of control between activities (or steps) in a business process.
3. The ability for management to know what step or state a process is in.

Number 3 is the Use Case for 'Process Visibility'. So, how do you do this as a service? Well, you still have to identify all of the activities to the degree in which you are interested in visibility. You also have to identify combinations of 'interesting state'. For example, if an order for a part comes in and no inventory is available for that part, well - that is interesting. This becomes a declarative rule applied towards a handful of variable to define a single state of interest. Also, anyone that is participating in providing visibility must be able to identify which instance of a process they are talking about, thus the equivalent of a correlation token must be used.

I'm thinking something like this:
Key=(ProcessID, InstanceID, CorrelationTokens)
updateProcessData( Key, SomeData)

It would be easy enough to turn this into an aspect (AOP) in a BPEL engine - but I'm not so sure that you want to bottleneck these calls. I do think it should be an aspect but invoked at the event source.

Now that you have a service that is collecting data across a distributed environment, it must tell others about the 'interesting states' that it has encountered. That sounds like a pub/sub problem to me.

I don't know... maybe something like this:
subscribeProcess(ProcessId, StateID, StateOfInterest)
(This implies that it is interested in all process instances that achieve a certain state.) Obviously, the BAM system would end up being a subscriber to this.

Hmmm... interesting. Now we have a state machine firing when certain states of interest are achieved.

Next up: 'Flow Control as Metadata', followed by 'Service Oriented Metadata'

The Systinet Gateway

Just some quick highlights on the Systinet Gateway product:

Systinet Gateway is able to extend and bridge the following products: IBM WebSphereMQ (formerly MQSeries), TIBCO Rendezvous, and any JMS-based middleware solution. Gateway can extend these MOMs and bridge between combinations of these products.

Systinet Gateway provides full support for the latest standards, including SOAP 1.1, SOAP 1.2,WSDL 1.1, WS-I Basic Profile, WS-Security, WS-ReliableMessaging, WS-Addressing and WS-Eventing.

-------------
In essence, the Systinet Gateway modernizes some of the last-gen Queue/Bus products by bridging the gap between proprietary and open WS-* standards. This is nice if you already have an existing product (Tibco/IBM-MQ/JMS-*) in house and want to move down the ServiceNet path.

Systinet chose not roll yet another Queue/Bus but instead has placed a wrapper around the best-of-breed that exist today. One thing I'm not sure about is the support for WS-Policy (is that implied?) Also, I'm not sure but it sounds like they do some sort of translation between a WS Endpoint and a Topic (I'm guessing).

Is the Gateway a big deal? Yep. The Queue/Bus is commoditized, but the WS-Bus is still emerging. The Gateway must still prove to be performant (minimal overhead) and consistent with the architecture of the products that it front-ends (e.g., how well does it work with clustered versions, does it gracefully start-up/shut-down with the Bus, does it have similar versioning mechanisms, etc.)

What does this mean for Systinet? First, it is one of their higher end products ($$$) and will require a more sophisticated sale then what they are used to with Wasp. Second, they will have to convince Tibco/IBM/Sonic/Spirit that they aren't competition, but complementary. Lastly, they will have to fight off the other 'legacy-enablers' like Iona and the adapter players. This product (along with UDDI and their service enablement line) now give companies the tools to turn 'last-gen' into 'next-gen'.

So what can we expect next from Systinet?

Iona to Reduce Number of Press Releases!

A representative from Iona has stated that they intend to reduce their number of press releases... or something like that. Actually, I believe what he said was:

"Gateways are simplistic -- anyone can write one" - (meaning that even Iona can write one)

and...

"I'd personally be embarrassed to issue a press release touting a mere gateway."

Interesting.

Let's take a look at the Iona press releases which are so damn important:
May17 IONA Makes Standards Compliance Easier for Telecommunications Equipment Manufacturers
May12 Chinese Premier Visits IONA Technologies
May10 IONA Joins Telemanagement Forum
May04 IONA Puts Developers on the SOA Fast Track

Hmmm... they joined an industry group, had a diplomat visit and wrote a white paper. Iona - you kick ass!! Don't bother with those silly press releases about new products - those... well, those are just embarrassing.

BEA opens up some slots, too

From Monster.com:
======================================
Major Duties and Responsibilities:
You will join the development team for state-of-the-art BPM application development framework aimed at enabling corporate-level developers to build and deploy scalable, highly-available, web-based enterprise applications involving web services and other B2X e-business applications.
Specifically we are seeking a person in our runtime team to help build the Business Process Engineer.

Qualifications/Necessary Skills:

A related degree and 5-7+ years relevant experience in java programming at the systems level. Must have in-depth expertise with J2EE technologies, enterprise level server development.
A solid understanding with BPEL required.
===========================================

What no BPELJ required???

Job Openings - Momentum

Momentum Software has recently opened several new positions:

(2) Java / J2EE Architects
(4) Senior Java Developers
(1) Junior Java Developer
(1) Web Services / SOA Consultant
(3) Senior .Net Developers (C#)
(1) Junior .Net Developer

All of the aforementioned positions are based out of Texas and require some travel. Both full time and contract positions are available.

Interested parties should contact the recruiting department: careers@momentumsoftware.com

Legos and Jigsaw Puzzles

Toys.

An infant can pick up building blocks and with no training build something - just put one block on top of the other.

A toddler can use legos. Line up the "in's" and the "out's" and push the pieces together.

Jigsaw puzzles are trickier. Puzzles are packaged by the number of pieces they contain. Why? Because in a puzzle each piece has only 1 potential place in the puzzle (1 'in' matches 1 'out') and as the number of pieces increases so does the complexity.

Imagine a set of legos where the notches (or interfaces) were different on every block. Each lego only worked with one other lego in the set.

In software we see a similar problem. As the number of pieces (and interfaces) increase, so does the complexity of assembling the components. Some items plug in nicely while others require significant transformations. Scanning for the right component takes time -like scanning for a single puzzle piece in a 1000 piece puzzle.

Personally, I love the interface for the database. I believe whole-heartedly that the interface is what made it so damn popular. do(DDL) and do(DML) are absolutely beautiful. Something to create, destroy, administer and manipulate - and all done on a solid mathematical foundation.

Call me silly for not liking an operation called, "getLastStockPrice". Call me dumb for not being able to figure out the REST equivalent of the operation. For some reason the magic of the WWW feels like it comes more from HTML than HTTP. Is the real magic in the browser: do(HTML)? Is chasing the HTTP verbs a red herring? or just an obvious fact that when realizing a domain specific language you shouldn't screw it up with a complex API.

The web... simple addressing, simple verbs acting as manilla envelope; and a domain specific language that can describe everything from pornography to the weather, all working together to create the worlds largest distributed computing platform. Simple addressing, simple envelopes and DSL's.

WS-REST

Mark Nottingham entered the Matrix in search of the spoon.

First, I tip my hat to Mark for tackling a complex issue.

Mark points a handful of things out:
1. A single http request is indeed synchronous
2. The http request doesn't have to block for the end result, instead it can block for an ACK or a fault code
3. Two http requests can be combined to create an asynchronous message exchange pattern
4. Pub/sub can be simulate via polling.
So far, so good.

He continues, to say:
5. One can hold open a TCP connection and use event looping to minimize the resource consumption issue.
Ok, this is an interesting one. In my opinion, there are some problems here:
1. It assumes that not only the end server utilizes a specific mechanism (event looping), but also every intermediary between the client and the server.
2. Connection oriented protocols don't work well when the server crashes (all connections must be reestablished; kiss your performance gain bye-bye).
3. Clustering connection oriented protocols (IMHO) is more cumbersome than connection-less protocols.

Then he introduces the peer-to-peer model, where the client has a well-known address and continues on to acknowledge the firewall/NAT issue. Very good.

Ultimately Mark points out that many people hear 'http' and think synchronous - which is right and wrong. Yes under the covers the application blocks, but not necessarily for the 'answer'. He also states that people should consider the level of abstractions that they need (describe the message exchange versus describe the implementation). I agree. He ends his post with noting that modern distributed computing practices require the ability to describe the interface and the message exchange pattern, then he goes on to mock one up.

In my opinion, Mark has blasted past the issue of 'WS-I' style versus 'RESTifarian' style and is well on his way to finding the best-of-breed. This will be one to watch.

Emerging Technologists

Every solution creates a new problem. As soon as a solution is introduced emerging technologists begin thinking about the new problems and the means to solve it. They jump right over the 'current' problem and begin attacking the issues that haven't occurred to the majority.

Over the years, I've created a few broad categories that I use to describe the 'emerging' people:

Spotters - a spotter is someone that scans the available technical literature looking for the next big thing (a spot).

Weavers - a weaver is a person that combine multiple 'spots' into a candidate solution (a weave).

Appliers - an applier is a person that takes a well known problem and applies a new candidate solution towards the problem (an application).

Inventors - an inventor is a person that illuminates a new alternative solution to a well known problem (an invention).

Evangelist - an evangelist is a person that makes persuasive arguments in support of a new spot, weave, application or invention.

Referee - a referee is a person that attempts to mediate the 'emerging' people and their solutions.


What are you?

RESTifarian Evaluation

Per my earlier note, I have evaluated my feelings on the units of REST:

1. An architecture is defined by its constraints - Strongly Support

2. Separate client from server - Strongly Support

3. The client / server relationship must remain stateless - Support

4. The client must be able to cache data - Support

5. Uniform Interface - Support

5.1 Identification of Resources - Strongly Support

5.2 Manipulation of Resources through Representations - Strongly Support

5.3 Self Descriptive Messages - Strongly Support

5.4 Hypermedia as the engine of application state - Support

6. Layered System - Support

7. Code on Demand - Don't Support

REST is a single candidate architecture. And an architecture must be evaluated on the whole, rather than the individual parts. Fielding does a great job of stating that every decision in the candidate architecture has trade-offs. He further identifies the applicability of the architecture (distributed systems, system 'feels' stateless, desire to scale, desire to evolve, etc.) Like all candidate architectures, one can only evaluate the architecture against a problem that you are trying to solve. REST by itself can not be evaluated, only evaluated as a candidate solution to a pre-identified problem.

What was interesting to me was the lack of association between REST and HTTP, or more precisely the 'verbs' of HTTP. Instead, Fielding places significant attention on the need to use Resources (think URI), Self Descriptive Messages (think XML), but does imply that a generic interface is required. Somehow... I find myself agreeing with Fielding but disagreeing with many RESTifarians (I'm not sure how this is possible). Fielding suggests a "a uniform interface between components ", however he never mandates what that interface should be - only stating the characteristics. However, I find significant 'folklore' stating that the interface is, "...defined completely and solely by the specified semantics of HTTP, i.e. GET / PUT / POST, etc." (Jeff Bone) He continues to argue, "*there are no applications you can think of which cannot be made to fit into the GET / PUT / POST / resources / representations model of the world!*" The only issue I have is his term "made to fit" - meaning, what was the price of 'making something fit'?

So, with all of this said, where am I landing?
1. REST is a great example of a candidate architecture and should be evaluated based on the problem at hand.
2. REST doesn't dictate HTTP or the HTTP verbs and this is a good thing.
3. REST does state that the interface should be uniform, and this too is a good thing.
4. The base HTTP application semantics can be 'forced' to solve any problem, they just aren't always a good fit.
5. Additional exercises should be performed to document how many RPC style verbs would be translated into a more REST-stlye. Verbs that don't translate shouldn't be forced to fit, rather new verbs should be introduces as part of the first-order vocabulary.

Tim Bray on things

Tim is now at Sun :-) and has new things to say.

1. He doesn't like composable message architectures. He doesn't like describing the resolution to non-functional requirements of distributed computing via specifications because it is hard. In my opinion, you either specify your separation of concerns with specifications or make them proprietary. I lean towards specifications. Also, the solution will utilize a building block approach or a big-bang approach. Your call. I prefer building block.

2. He doesn't like automated service discovery (UDDI). I think he misunderstood what UDDI was all about. That's ok, many people do - it is a design time service, not a run time. Throw away the "automated" part and you're a lot closer.

3. He doesn't like declarative application building. He mentions BPEL and ws-chor as examples of this - unfortunately neither of them are valid examples. Actually, I kind of wish they were! Declarative approaches for state based decision making and distributed invocation has a real place in software development.

4. He doesn't like leaky abstractions. Me either, but unfortunately he doesn't give an example.

5. He's concerned about the standards process. Perhaps the new Sun employee should take a look at the JSR process :-)

6. He's concerned that we aren't making it simple enough. Here he has a decent point - it is complicated. Could it be simplified? Sure, throw away functionality and we can simplify it. Keep in things like reliable messaging, self-contained enveloping, trust, authentication, multiple levels of transactions, etc... yea, you can make it as simple as you'd like!

Welcome to web services :-)

RESTifarian?

Mark Baker declared that I was on the way to enlightenment (AKA, seeing things his way).

You know, I'm not sure if I am a RESTifarian.

When I first saw this:
http://www.extremeprogramming.org/rules.html I used it as a tool to determine what components of XP I believed in. It is tough to say, "Yea, I am an XP guy or Yea, I'm a RESTifarian". I've looked at some of the REST wiki's out there and I haven't found the simple list of rules and guidelines.
What I'd like to be able to do is say, "I support RESTifarian rules/practices 3, 5, 9, 11 and 14."

Perhaps if one of the REST supporters would give me the discrete rules for REST, I could more clearly state my opinion.

-------
Side note:
The XP guys did a great job. For the record, I'm not an XP'er. I've worked too many huge projects. Here is my stance:

Planning
User stories are written. - Support
Release planning creates the schedule.- Unsure
Make frequent small releases. - Strongly Support
The Project Velocity is measured. - Unsure
The project is divided into iterations. - Strongly Support
Iteration planning starts each iteration. - Strongly Support
Move people around. - Support
A stand-up meeting starts each day. - Partially Support
Fix XP when it breaks. - Strongly Support

Design
Simplicity. - Don't Support
Choose a system metaphor. - Don't Support
Use CRC cards for design sessions. - Support for OO systems
Create spike solutions to reduce risk. - Strongly Support
No functionality is added early. - Don't Support
Refactor whenever and wherever possible. - Partially Support

Code
The customer is always available. - Unrealistic - Don't Support
Code must be written to agreed standards. - Support
Code the unit test first. - Partially Support
All production code is pair programmed. - Don't Support
Only one pair integrates code at a time. - Don't Support
Integrate often. - Fully Support
Use collective code ownership. - Partially Support
Leave optimization till last. - Don't Support
No overtime. - Don't Support

Testing
All code must have unit tests. - Partially Support
All code must pass all unit tests before it can be released. - Support
When a bug is found tests are created. - Support
Acceptance tests are run often and the score is published. - Fully Support

No, I'm not an XP'er. I think they had some great ideas - but no, I didn't just close my eyes and say all of them were good ideas and jump on board. Am I a RESTifarian? Give me the tools to evaluate and I'll let you know.

Inserting Nouns into the Service Network

For the last 6 months I've been investigating the separation of nouns and adjectives from the verbs and adverbs. For those that don't follow my blog on a regular basis, here is a quick summary:
Nouns are people, places and things - in the software world they are our domain entities (User, Invoice, etc.) - and adjectives are the properties that describe the nouns. In the J-world we used Java classes to describe them - in the W-world we use XML schemas.

Verbs are the actions. Actions have (at least) two forms: technical (insert, update, delete, publish, etc.) and domain or business specific (ship, pack, etc.).

Technical verbs often act the same way on any given noun. Why is this important? Imagine walking into an enterprise with a set of nouns on a floppy disk. The company already has a service network in place. This ServiceNet is composed of 12 servers (portal, database, security, etc.) and runs 46 services. Each server has the capability of 'listening for new nouns' and a 'registration service' has the responsibility of: 1. Notifying the servers of newly registered nouns and 2. Notifying the servers of newly registered servers.

Ok, we just moved into a world where the network is aware of the network. No magic, just simple registration. What gets interesting is when nouns have 'default implementations' of a verb on a server. And surprisingly, you will find that this is possible and beneficial. If I add a new noun to the ServiceNet will it likely have security permissions? Will it need to be stored? Will it be viewed? Should I role all of this code by hand? Again, the goal isn't to perform magic - it is to create a network programming model where we think more about how to create smart verbs and dumb nouns. The servers are slowly given the ability to interrogate each other and eventually a common set of functions is factored out of the servers and dropped into the network.

I'm convinced that service oriented programming has the potential to get absolutely unmanageable. I talk with customers every day - they almost always start the conversation by bragging about the number of services that they have. Every time I hear this I think "Oh my God, what a maintenance nightmare! And they've only begun." In Schneider-land, the goal is to go for the smallest number of services, with the highest amount of reuse. Figuring out how to do this is a bitch.

Going from Services to the ServiceNet
The first step that I recommend is to take a look at all of the operations on all of your wsdls. Categorize what you already have (VerbOnly, VerbNoun, VerbMechanism, other). Then, determine if you could have made more re-usable services. What would they look like? Create a spreadsheet with the verbs as rows and the nouns as columns. The cells (or intersection points) are the realization of the VerbNoun. Beware of verb synonyms, or made up verbs (verbs that are really nouns). What is the smallest number of verbs you require to fulfill your needs?

Custom Metadata

I was just talking with one of my CTO buddies. He was bragging that implementing his software product at a client site requires no custom code. I paused for a moment and asked, "Does it require custom metadata?" To which he answered, "Yes, and it has a nice front-end to enter it in."

My first thought was that he was just pushing the problem from one location to another. But for some reason, it does feel better pushing the problem from code to metadata, but I'm not entirely sure why. I like the idea of not compiling everything (a benefit of metadata). I also like the idea of having domain constraints on the data being entered (versus 3rd gen languages). I guess that a system that embraces MOF/MDA/metadata concepts requires less in-depth expertise. In essence, the expertise is built into the framework, enabling a 'paint-by-number' approach to recurring pattern based problem solving (resources require less training, thus less expensive).

I'd love to hear what you think the advantages to a metadata driven approach are. Blog it. Link to this. Click through once. I'll repost referers.

Is Java an Interchange Format?

So, Microsoft believes that BPEL is an interchange format. It occurred to me that they likely perceive Java as an interchange format as well. If one really, really, really wanted to... you could take C# code, convert it to Java, then convert it into VB.Net. Yes, yes - Java is an interchange format!! I don't know why that didn't occur to me earlier. And all this time I've thought of Java as a programming language.

I'm such a fool. Officially, an 'interchange format' is anything that can be used as a temporary place holder while being 'upgraded' to the Microsoft format (X/Langs, C#, etc.)

Top Ten Interchange Formats (Microsoft perspective):
1. Java
2. BPEL
3. C
4. Perl
5. Python
6. Perl
7. C++
8. JavaScript
9. Haskell
10. Ruby

;-)

BPEL4XLang, BPEL4Java BPEL4*

It has been an interesting week in the land of BPEL.

For starters, Microsoft has been in a bit of a predicament. In August of 2002, Microsoft came out saying that they were going to deprecate their proprietary process language (X/Lang) in favor of a new open process language called BPEL. Well, Microsoft missed a couple of ship dates on their BizTalk product and eventually shipped their BizTalk 2004 with X/Lang. In order to save face, they created a mechanism to import and export portions of their X/Lang scripts in BPEL format. Now, Microsoft is saying that oh... the Business Process Execution Language isn't really an execution language - get this - it's an interchange format! Brilliant.

Now, depending on who you talk to at IBM you may find that BPELJ is a good thing. Although, some of the product groups were quite surprised to see the research guys published a white paper on the topic. Interesting. Well, it is worse. In addition to WSFL, BPEL, BPELJ, and the new JSR207, it appears as though IBM is considering additional process languages related to the Rational line of products that are more closely tied to UML. Brilliant.

Well, BEA not wanting to be 'out-dumbed' by MS or IBM has decided to throw more effort at BPELJ, and whine about the lack of progress on JSR207. As one of the BEA insider told me, "BEA is a Java company, not a web services company." What this implies is that BEA is having a hard time dealing with a 'service oriented language' like BPEL in their object-oriented, java-based platform.

So, what can we expect? Here are my predictions:
1. BPEL continues to move forward and serves as the primary foundational technology for executable business process descriptions.
2. A variant of BPELJ becomes JSR207.
3. Microsoft never budges off of X/Lang as their base language
4. The OMG guys create an enhanced Activity Diagram that is 'close enough' to stubbing out BPEL.

Next question: Is this good?
Yes. It is expected that MS will do things their own way - Microsoft will create whatever technology that they need to make things easy for their customers - sure, it will largely lock them in, but most Microsoft shops already know that they're locked in - otherwise they'd be Java shops.

And it is expected that IBM/BEA will bypass the JSR process to expedite a new Java-friendly implementation that Sun will end up incorporating into J2EE. Sun will grumble, eventually adopt it and then find a way to lose money on it.

Last question: What will OpenStorm do?
[coming soon...]

MS Posts Infopath to BizTalk Examples

Scott Woodgate has posted some examples of Infopath using orchestrations. However, I busted open the orchestration file and I'm having a hard time understand this one line of code....



Hmmm... XLang/S.... BPEL export = False? What does that line do?

More on UDDI

Jeff,

I just read your blog entry on this, and it's totally in harmony with my own thoughts on the matter. Until a few weeks ago I was in charge of directing the UDDI pilot->implementation programme for [a major investment bank], and we've pulled the plug on it because the whole t-Model thing is just far too complicated to be used in any meaningful way by the constituency that needs it.

We're now planning on building a service registry around an XML metadata store, and will probably go for an XQuery interface as the primary service discovery mechanism. We may keep what I've been terming a 'naive UDDI'
interface for compatibility with existing tools etc., but the long term hope is that we can throw this back over the wall to then vendors (and standards bodies) as something will need to replace UDDI in the web services unholy trinity

--
Chris Swan


Now Stefan states, "I have since come to the conclusion that all this taxonomy and categorization stuff is actually pretty ingenious." Stefan, I agree in a computer science kind-of-way it is ingenious. However, most people just want to quickly store, index or retrieve "Service Oriented Metadata". People don't like taxonomies or t-models.

UDDI was largely designed for external use (the UBR); the use cases that is supports are so far beyond the use cases that are required for intra-company needs that is quickly becomes too complicated.

I'd love to see someone try to explain to me why UDDI isn't a complete piece of shit. Justify it - I'm listening.

The UDDI Failure

SOA is about a triangular relationship - producer, consumer and directory. It seems simple. The UDDI represents the directory leg of the relationship, and in my opinion (and I'm not alone) has been a failure. When most people see UDDI for the first time, they usually ask questions like, "couldn't I just stick this information in a database or in LDAP??"

I recently had a discussion with an architect at a leading online brokerage house. He commented that they had looked at UDDI but thought it was overly complicated while simultaneously delivering a lack of functionality. Wow. Useless and a pain in the ass - that is hard to do. Well, these guys punted on UDDI and spent a day writing a small directory using RESTish terms with easy http access. They are very happy.

I'd like to highlight some alternatives to UDDI - if anyone wants to shoot me a note telling me how you're solving the problem or other alternatives, I'd love to hear from you.

BPEL versus Vaporware

The majority of Radovan's concerns seem to be around the structure that BPEL enforces. Sometimes this is called a "structured process". These are processes where the actors are given rules and are told not to break the rules. Structured processes exist in every corporation that I've been in.

"Semi-structured processes" are those processes that have 'wiggle room'. That is, they enforce a base structure or flow, but at key points run-time decisions can be made.

"Unstructured processes" are really more about achieving goals via whatever means. Here, the participants, the activities and the order will all change at run-time in order to achieve the end goal. The down side is that this often borders on chaos and has limited repeatability - which is a key driver behind process.

What we are finding is that BPEL can handle virtually every case for 'structured' and 'semi-structured' process descriptions and execution. Unstructured processes usually don't lend themselves well to any kind of 'ordered activity machine'. Rather, these processes are more likely to be executed via 'process liberation' tools like Groove, where the focus is on communication and inter-team task visibility.

I don't fully understand the critiques of BPEL. It really is a powerful language - although I can see if you don't work in it where you might be confused. But... we have a team working on this all day, every day... as does Collaxa, FiveSight, SeeBeyond and a host of other companies.

Agree and Disagree

Read this. Negate virtually every sentence and you will have my view.

Read this. Here, I agree on virtually everything. One thing I'd add is that the DSL is a virtual language - it consists of your base language (like bpel AND all of the services (nouns and verbs) in your enterprise vocabulary. In a manufacturing company, your verbs and nouns will relate to purchasing, picking, packing, shipping, etc. Late binding, loose coupling, interoperable schemas, etc. all create a new meaning for the domain specific language. After you pick a first order DSL, you will spend the next decade creating your second order DSL (the enterprise vocabulary).

Eric Newcomer from Iona recently stated:
I can't see that the drawing approach will really work. I don't know of any graphical software development tool that has yet to address the entire lifecycle; or that generates code of sufficient quality.

I think it's just a problem that isn't meant to be solved.

Since this statement Eric and Stefan have found some common ground:
-Models can have common ui notation and be graphically driven.
-Models can drive metadata.
-The graphical notation and the resulting metadata are two completely different concepts and should be managed that way.
-XML is a great way to drive a metadata approach.

Here are some additional random thoughts:
-Models and action semantics can create traditional 3GL code, but I'm not sure you want them to.
-Metadata can be interpreted by engines and executed.
-Engines are at the heart of reusable services.
-Nouns translate nicely into metadata.

OpenStorm Orchestrator Update

Just a quick note - next week we will be making available the 2.1 version of the suite. Updates include:

Studio Update
- Improved wsdl editing
- 'Helper' for specifying duration and deadlines
- 'Helper' for graphically creating correlations
- 'Helper' for rapid message definitions on in-bound and out-bound web service calls
- Full XPath manipulation tool
- 'Helper' for boolean expression building (switch, while)
- Ability to create rapid services (drop either Java or C# code directly into an invoke node and it compiles and deploys the code as a remote web service, then replaces the code with the wsdl that front ends the code).
- New 'Web Service Invoker' tool for calling/testing services
- Canvas supports zoom-in/zoom-out, also has a new 'thumb-nail view'

Java Server Update
- Increased support for variations of the 'assign' tag
- New console with drill down (view by bpel, by instance, by correlation, etc.)
- Reliability upgrade with full state machine persistence (test = pull the power plug, plug it back in)

.Net Server Update
- Reworking web service layer to support latest Microsoft WSE upgrades
- Creating long term architecture for Indigo support

documentation
- New BPEL Programmers Manual
- Packaging WSDOX web service documentation with software

In addition, we've added a couple people to the engineering team so we should be able to move a bit quicker on the 2.2 release.

The Promise

From the forward of, "Business Engineering with Object Technology" by David Taylor:
We stand on the threshold of a new era in business engineering. For the first time in the history of computers, it is now possible to build information systems that directly reflect and expand the way we think about business processes. The critical enabler for this transformation is object technology.

Hmm... sound familiar? Well, once again, we find ourselves on the same ole quest.... to achieve a COmmon Business Oriented Language. Yes, I'm a fan of service oriented-bpm. I believe that we need a new vocabulary... for the last several years we have seen some consistent patterns in enterprise computing:
1. A gradual movement towards specialized verbs (servers/services).
2. A desire to separate the verbs from the adverbs (factor out non-functional requirements).
3. Separating nouns and adjectives from verb implementations (XML Schema vs. classes).
4. Standardize the prepositional phrase (predicate).
5. Create a sentence grammar (composition languages).

The verbs and the nouns that we choose to program our computers is central to productivity. The separation of our terms is essential. Breaking the bad habits of the object oriented verb-noun coupling will be tough. Creating our new vocabulary will be even harder. Finding a single Business Oriented Language may be impossible. Yet, we define our programming model by the means in which we separate our concepts. Moving beyond the standards gook is essential.

Oh, and just for fun - here are the verbs of COBOL: Accept, Add, Alter, Call, Cancel, Close, Compute, Delete, Disable, Display, Divide, Enable, Enter, Exit, Generate, GoTo, If, Initiate, Inspect, Merge, Move, Multiply, Open, Perform, Read, Receive, Replacing, Return, Rewrite, Search, Send, Set, Sort, Start, Stop, String, Subtract, Suppress, Terminate, Unstring, Use_After, Use_For_Debugging, Write

If only COBOL had the verbs 'publish', 'subscribe' and 'transform' ... then it we could have called it the 'COBOL-ESB' ;-)

Tech question...

I've run across a number of web service operations that look like 'foreign-key' relationships.

As an example, look at the following web service operations:
getTeams() // returns a list of all teams, including all the team id's
getPlayers(teamID) // returns a list of players, for a specific team id

Now, this seems intuitive for a human to figure out - call getTeams(), and then use the key to make a call to getPlayers(theKey) for the team you want. However, this isn't intuitive for a computer to figure out. When you are in an interactive setting (InfoPath, Excel, etc.), and you make a call to getPlayers(..), you want the software to give you a drop-down list of all of the teams. But in order for this to happen, the software (Excel or whatever), needs to know about the relationship between the two operations (getTeams and getPlayers). This information is usually capture at the DB level and is also captured at the object level (UML), but I don't see a mechanism to publish this type of relationship at the service level.

So, here is my question: How do I define and publish computer-readable operation-to-operation relationships using a standardized metadata description language (like WSDL, or other)? Email me: jschneider at momentumsoftware dot com

WS-Metadata Exchange

WS-Mex is now published at the Microsoft site.

The spec is 975 lines long, but I can summarize it for you.

One verb: [get] and three nouns: [schema, wsdl, policy]
Naturally, this took 6 type definitions, 11 messages and 3 operations.

Suddenly: ?WSDL, ?POLICY, ?SCHEMA doesn't look so bad...

They hardcoded in three types of metadata into the spec - isn't there something IRONIC about that???

Oh, the editors are:
Francisco Curbera (Editor), IBM
Jeffrey Schlimmer (Editor), Microsoft

Verb Only (when the verb acts the same on all subjects - e.g., save (x), save(y)

Verb Noun (when the verb acts differently (different logic) on each subject processPurchaseOrder(), processSalesCommission()

Qualified Verb (Bidirectionally Link(x,y)) versus link(x,y); used to qualify the intent of the verb (changes functional requirement, unlike most adverbs which only apply to NFR)

Verb Mechanism (when the logic is changed more by the mechanism than by the subject)

Verb Mechanism

Many of the web services that are created take on the form of "VerbMechanism"; that is, they specify a verb and the mechanism in which to accomplish the task, then they usually require the 'who' and the 'what' to be passed in:
sendEmail ( 'to bob', 'the status report')
sendFax( 'to bob', 'the status report')
sendInstantMessage( 'to bob', 'the status report')

Here, the mechanism is 'Email', 'Fax' and 'InstantMessage".

Another way of writing this is by using 'via':
"send the status report to bob via email"

When specifying a mechanism, we will likely have preconditions on the mechanism. "via email" requires four pieces of information (email server, email credentials, email destination, email payload); The data related preconditions can be expressed in a declarative manner on each mechanism. Currently, we push these data nuggets into the signature of the VerbMechanism. But usually there is an overlap between the verb and the mechanism (it was 'send' that required two of the pieces of information (destination, payload); the mechanism required the other two pieces (server , credentials).

A mechanism is a way to overload an operation. The verb I'm overloading is 'send' and the mechanism is 'Email". When we overload in object oriented systems, we usually just grow the signature: send(destination, payload); send(server, credentials, destination, payload); I'm not a fan of this... I prefer the declarative OCL like approach but with an 'implied signature' based on the overloading with a mechanism.

Another issue to consider is the 'invisible mechanism' or 'implicit mechanism'. Consider:
savePurchaseOrder(...);
Here, the designer intentionally hides the implementation mechanism from the user of the operation.
Save Purchase Order via Relational Database
Save Purchase Order via Flat File

Now, I agree that it is a good practice to not burden the user of a service with mechanism. However, I'm not sure that it is always a good idea to hide it from them (burden them versus inform them).
Again we see that each mechanism will require a set of preconditions:
"via Relational Database" needs (a JDBC Driver, a connection string, credentials, etc.)

I'll save "Reliably Save the Purchase Order via Relational Database" for another morning.

The Service Network

Today, there are largely three camps when it comes to web service architectures: The SOA guys, The Bus guys and The Protocol Network guys.

The SOA guys
The SOA guys were the original 'web service' people. They viewed the new paradigm as a ubiquitous service based system that would utilize the architectural pattern known as SOA (Producer, Consumer and Directory). The SOA guys are the people that build UDDI servers and SOAP platforms. They have taken the best of the CORBA world, learned some lessons and reapplied it to a new set of protocols that are accepted by MS and IBM. The SOA guys rarely mention a 'network topology', but their implicit topology is point-to-point (my consumer directly calls your producer).

The Bus guys
The Bus guys are the people who believe in messaging. Ultimately, messages and services go hand in hand. Some people put more emphasis on the service (SOA guys), while other put more emphasis on the message (Bus guys). The Bus guys also love asynchronous communication - thus, they loves queues or any other store and forward mechanism. Unfortunately, most of the Bus guys come from the JMS world and largely their stuff doesn't interoperate (think ESB). The Bus people usually think that the web services network topology is hub and spoke.

The Protocol Network guys
The Protocol Network guys are the people who treat web services more like networking protocols. These guys put emphasis on two things: wire protocols and policies. They see everything as a policy on a protocol. These guys love specs like ws-addressing, ws-discovery and ws-policy. The Service Network guys see the network topology as being adaptive based on the state of the network. They rely on routers, load balancers and other devices that have knowledge about the running services to make informed decisions on the fly.

The reality of is that most large companies will need a hybrid of all three. They will embrace a standard SOA triangular pattern, but letting a Protocol Network make routing decisions at run-time, with the messages often ending up in queue. I continue to consult to my clients about the convergence of the three paradigms. This convergence is what I call the Service Network.

The Service Network is a message-based, service-based and protocol-based computing model.
- It leverages the SOA model to decouple producers and consumers and to provide lookup capabilities for self-describing services.
- It leverages the Bus model to provide asynchronous communications for long-running processes.
- It leverages the Protocol Network to provide runtime decision making about locating and executing a service on the network based on the service network conditions.

Another way of looking at these models is:
- SOA decouples software units (consumer and producer)
- Bus decouples software in time (synchronous = time-coupling)
- Protocol Network decouples software from hardware (run a service on some machine)

The goal of the service network is to provide all three forms of decoupling.

WhiteHorse and Virtualization (take 2)

Alex Torone the Lead Program Manager for the Microsoft Visual Studio Enterprise Tools Team gave me a gentle kick in the balls regarding my inaccurate posting around WhiteHorse. Here are his clarifications:

What the diagram really represents:
The LSAD (Logical Systems Architecture Diagram) represents "logical run time hosting environments" (hence the name). Each box represents a "logical server type". Specifically, the large blue boxes represent a configuration of IIS, whereas the endpoints on the large blue boxes represents web sites. We model the entire IIS meta base (in this example). So a user could either supply "desired configuration" in the tool, or they could simply point to a "canonical server" that has the "desired configuration" and harvest those settings. Once the settings have been defined in the LSAD model, the user can then define constraints against the application environment. For example: Suppose my datacenter policy for "front end web servers" require web apps to use forms authentication and impersonation. These constraints will be validated against the application designer (we model all of system.web for example) and can be expressed in this logical design. There are also two additional layers in the SDM model (part of DSI refer to links below) which represents the network layer, and device layer that are more in line with your comments and are slated for a much later tools and platform release.

About the Physical DataCenter:
Data Centers host many types of applications. Network infrastructure diagrams (we've all seen them) have physical machines, IP address, Vlans, switches, routers, etc. The LSAD is meant to represent abstractions over the physical data center. We want to represent types of server not physical servers. One box on the LSAD does not necessarily equate to a physical server in the data center. In fact, you can create multiple Logical web servers and place then on one physical server with SQL as an example. When we get to actual deployment releases post Whidbey (please refer to the Dynamic Systems Imitative (DSI) links below), we will then provide a logical to physical mapping. This action will populate all of the deployment parameters with they physical URL's of the web server etc.

In conclusion, the LSAD is about conveying that information which is important to the developer (such as what kinds of services are available to me, what communications pathways are open, what configuration must I adhere to, what are the boundary conditions that I must be aware of, etc) such that we will increase the probability that their design will actually work when it is physically deployed.

The more I dig into the DSI, the more impressed I am. If they can pull it off with design-time integration it will be one heck of a story. Here are some links Alex provided me to reduce my ignorance:
http://www.microsoft.com/windowsserversystem/dsi/default.mspx
http://msdn.microsoft.com/vstudio/productinfo/enterprise/enterpriseroadmap/default.aspx?pull=/library/en-us/dnvsent/html/vsent_soadover.asp
http://msdn.microsoft.com/vstudio/productinfo/enterprise/enterpriseroadmap/whitehorsefaq.aspx
http://microsoft.sitestream.com/PDC2003/TLS/TLS345_files/Default.htm
http://msdn.microsoft.com/msdntv/

Book Recommendations

It is tough to find good books on subjects like web services. The people that write on the subjects are usually writing about the current state (which is outdated by the time you buy the book) or a future state (which is usually wrong). Generally speaking, I don't buy books on web services - however, I do buy books on areas of convergence. That said, here are my recommendations:

1. Policy Based Network Management - A book by John Strassner, a Fellow at Cisco and thought leader in DEN, writes on the topic of the declarative network. The book covers basic policy models and then explores the DEN-ng policy model as an example. The book will never mention SOAP, web services or anything at the application level. It is up to the reader to draw analogies between Network Policies and Application Policies. For those that are familiar with the WS-Policy specs, you will find significant overlap between WS-Policy and PBNM (Policy Based Network Management). This book should help you understand the importance of declarative policies and their uses in application-level service networks.

2. Grid Computing - A book by Joshy Joseph and Craig Fellenstein, this book is part of the IBM 'On Demand Series'. The book covers the basics of grids, the merging of grids and web services, OGSA, OGSI and the programming model for the Globus GT3 Toolkit. This is a good book for anyone who needs a crash course in grid services. It is a high-level read - not a reference manual.

Why the Outsourcing Flap Makes Cents

Rich Miller, who runs my favorite blog, made a posting near to my heart titled, "Why the Outsourcing Flap Makes No Sense" - where, Rich asks what the flap is all about.

The flap is about...well, pissed off people that lost their jobs.
I spoke with two different people today who are old friends. They both worked (past tense) at Sabre. The first was a VP of engineering who was laid off and his job was moved to Poland. This guy was a real "do-er" - the kind of guy that made things happen - he'd cut through the BS and get things done. Does he cost more than someone in Poland? Yes. Sucks to be him. He's now remodeling his kitchen and sending resumes out on Monster.com. He gets it - lower wages for Sabre lead to a more competitive product - higher share prices for share holders and lower prices for consumers. He completely gets it - but he's still pissed off.

The second person I talked with (a project manager) told me that she was told that she had to let her whole team go because they were to be replaced with people in India. She told her manager that she didn't like the idea but went along. Eventually she was told that her job was moving offshore as well but they gave her the option of being manager of 'offshore procurement'. She did it. Then when the procurement was done she was let go. Now she is reading books on how to day-trade. And yes, she is pissed off.

The flap is about people that are great at their job who get let go based purely on cost. They weren't given the opportunity to accept a lower salary - after all, their employer has a mandate to move 25% of product engineering jobs offshore. The flap is about humans who felt disgraced by long-time employers.

Maybe the U.S. installed a minimum wage system too early. Maybe we drove up our own cost of living and that drove up our salaries. It is our own fault. I know - yelling about free markets won't change anything. But if my friends at Sabre and all the other companies that have cut U.S. workers want to bitch - I understand. Flap all you want - because that is all you are going to get.

Objects, Services and Verbs

A few years ago, I asked my mentor what the difference was between objects and services. He told me a handful of things, but the one that I really remember was, "objects use a 'noun.verb' notation and services use a 'verb.noun' notation." I asked him why this was a big deal and he told me that in most systems there are less verbs than nouns. His point was that you would end up with less first class citizens in a service oriented world than you would in an object oriented world. Very interesting. He later pointed out that the service citizen would likely treat data as meta-data, making it much more manageable than an object that tries to use polymorphic behavior to apply verbs functionality on nouns.

I've become fascinated with service / operation naming. As an example, the WSDL spec uses, "GetLastTradePrice". Immediately I break it down, "Get" "Price" ... what kind of price? "Trade"... which one? "Last"... or perhaps it is: "Trade.Price"?... Hmmm... How about :
Trade.Price.get().last();

Object, attribute, getter, ordered set operation - an interesting way to break it down. Now, why did the service people run the whole thing together (GetLastTradePrice)? Good question. Don't I end up with a ton of operations if I run them all together? What if... I didn't run combinations together, but instead identified the command components: (getter/setter) x (an objects enumerated attributes) x (potential set operations)? Should an operation name be one big concatenated string where all the potential combinations are combined at design time? SQL sure didn't do it this way - - they went command language for base manipulations - and then went stored procs with fixed names for one-offs (oh, and declarative rules *triggers* for eventing - here the name didn't matter).

Anyway... all I really wanted to do was share some of my favorite verbs:
insert, add, update, set, delete, remove, erase, get, select, fetch, subscribe, publish, receive, listen, send, notify, call, invoke, create, destroy, deallocate, dispose, show, view, hide, close, open, drop, restore, resume, suspend, pause, clear, filter, cache, run, start, execute, stop, allocate, new, advance, go, post, do, find, locate, evaluate, jump, visit, goto, exit, break, spawn, join, split, lock, unlock, process, print, transfer, throw, push and pop.

It's a pretty good list of verbs. There are some people that REST'd after only finding two or three verbs. But they just like wrapping verbs inside of other verbs :-) That's ok - I don't think they hate verbs. They just like a couple verbs a whole bunch!

I like the idea of having an enumerated set of verbs. This isn't the list, though - too much redundancy. Also, some verbs are really just combinations of other verbs. Hmmm... first order verbs. Second order verbs. How many first order verbs does a system need to have a semantic foundation for 'doing' things?

And no, I'm not a verb bigot. I love adverbs too! Nouns suck - although I admire those that have the patience to play in the noun space. Adjectives are cool - only in that they are simple ...

It is my belief that we are *slowly* moving towards a semantic and service oriented world. Operation names that are concatenated strings of verbs, nouns, adjectives and adverbs worry me. There is a better way.

OpenStorm Blog

Ryan, Dave and the crew have kicked off the OpenStorm blog. It will focus more on using the Service Orchestrator and general questions related to BPEL and service composition.

See:
http://www.openstorm.org

I don't see an RSS feed, but I'll ask them to put one up.

AT&T and GrandCentral Partner

See:
http://www.internetnews.com/ent-news/article.php/3324191

"AT&T WebService Connect allows different applications from different sources to communicate without time-consuming custom coding. And because it is XML-based (define), it's not tied to any one operating system or programming language.

Developed over the last year with partner Grand Central Communications, WebService Connect plays into AT&T broader strategy of evolving from a long-distance phone company to a provider of enterprise network services. "

...

"The service will be rolled out gradually in the coming months. It starts at about $34,000 per month, although prices could run higher depending on usage. In terms of its telecom competitors, AT&T believes it is farthest along in offering Web services (define) to its customers. "

Congratulations to GrandCentral!

Jim Waldo clarifies position

Jim does a great job clarifying his position on standards:

http://www.artima.com/forums/flat.jsp?forum=106&thread=4892

Jim states:
Point one: Just because something is called a standard doesn't make it open; and something that isn't a standard is not, because of that, proprietary.
Point two: A standards body is often a lousy place in which to invent a technology.
Point three: The previous posting was not a veiled (thinly or otherwise) attack on any particular standards group or collection of standards groups.
Point four: If there are multiple groups competing to write a standard for the same thing, it is probably a safe bet that the technology being standardized isn't ready for standardization.

Well done. I think Jim fully understands it. Sucks doesn't it? Oh well.

Now, I'd like to see Jim (the brain behind Jini) take some of his vast knowledge and write a couple new standards... for starters, I think he'd have quite a bit to add to ws-discovery: (ws-leasing, etc.)

So, here is the WS-* formula:
1. Find a concern (think separation of concerns, they usually end in "ility")
2. Find a remedy to the concern.
3. Take the name of the concern and put the letters "WS-" in front of it.
4. Use as much protocol (with XML) to describe the remedy, use wsdl and the other ws-specs to weave a full story.
5. Publish your spec.
6. Wait for either MS or IBM to "expand" the idea, change the name and republish it with a higher degree of separation of concerns and a name that has a striking resemblance to the name you gave it.
7. Bicker to the press about it.
8. Wait approximately 6 months. Feel free to knock out your reference implementation during this period.
9. Watch the MS-IBM version become popular.
10. Terminate your version and publicly support the MS-IBM version. Be happy that a spec exists.

It really is a very simple, straightforward process. Best of luck.

Supply Chain Orchestration: RFID meets BPEL

Last week I made a visit to Boston to meet the people at Connecterra. These guys are in the "RFID middleware" space. This category is often called a 'savant' - although the industry seems to be moving beyond this term.

It was absolutely fascinating to see RFID signals get picked up by the readers, be sent to specialized RFID middleware where the signals were aggregated, filtered and eventually turned into web service (soap) calls. These calls could then could be consumed by a BPEL engine for processing. Immediately, the opportunities for "supply chain orchestration" are illuminated.

I've been working on a white paper called, "Supply Chain Orchestration" with Bob Betts for the last couple of weeks. The topic is huge - the impact is significant. We should have the paper done in a couple of weeks - I'll post a note when it is available.

Adobe launches beta of XML/PDF Form Design Software

See:
Adobe Launches Public Beta of New XML/PDF Form Design Software




"Developers can easily integrate form data with core enterprise systems via XML, OLEDB and web services. Additionally, Adobe Designer allows users to design forms that can be used with digital signature technologies for facilitating secure electronic transactions."

Very exciting!

WS-Discovery and Jini

News.com and Ron Schmelzer have once again teamed up to inform the public on web service specifications:



Ws-disc and Jini have a discovery component and Jini is a 'java' only thing. A couple other things worth noting:
1. Sun made a huge mistake by not bundling Jini with the J2EE stack early - this killed Jini - it was considered a 'device only' api.
2. Jini is API not a protocol - they later rewrote this functionality as a protocol for Jxta
3. Jini bundled remedies to concerns (leasing, discovery, proxy, service matching and tuple space) in a single spec.

My belief is that ws-discovery will catch on as long as people don't say, "The idea is very much the same as Jini". As cool as Jini was, it was doomed by Sun. If you want to describe ws-discovery, say "it's a multicast framework for dynamically finding resources on a net (wan, lan, scatternet, piconet, etc.) without knowing the address of any resources ahead of time."

WS-Discovery isn't bundling a ton of things together - it is a lightweight protocol for finding stuff across a variety of networks. It isn't based on a single language and has good support. It can make it - as long as we don't accidentally kill it in the cradle.

The Web Service Dial Tone

Web Services are failing and I know why.

Most people I talk to think the lack of adoption is for one of two reasons:
1. The huge WS-* stack is too complicated and it reminds them of CORBA.
2. We haven't found a killer app for ws.

These are both interesting - but in my opinion, they are not the real reason why ws are failing. I firmly believe the answer is very simple; we haven't created a web services dial tone. When I plug my laptop into a network, it immediately sends a broadcast message to the network. Certain devices like routers, firewalls, gateways, etc. respond to the inquiry and pass on some information about their capability and service offerings. This transparent conversation creates a 'network dialtone' - it enables a device to plug into a network, discover the services and converse with them. This capability is at the heart of our TCP networks, but has not been realized in our 'web service networks'.

In order to create a ws-dialtone we need only a handful of capabilities.
1. Consumer-side applications need the ability to send a broadcast message to a network (UDP for web services).
2. Producer-side applications need to be able to listen to the broadcast and respond. They also need the capability to broadcast their availability.
3. UDP style broadcasts are limited to a sub-net. Thus, sub-net routing (via a soap router) is critical.

Much of this functionality is available via the WS-Discovery specification. However, a large number of people in the ws community view the aforementioned protocol as a tool to be used strictly in ad-hoc networks or for consumer hardware devices. And although this is a subset of the target audience, it has broader applicability in the enterprise environment.

As we continue our movement towards contract based development we will begin to see more emphasis on standardizing the contract. Today, our standardized contracts come in the form a platform like J2EE (technical contracts) or from industry working groups (business contracts). The combination of the standardized contract, the discoverable implementation and late binding will introduce a computing model that enables a service oriented system to automatically find producers for a predetermined piece of functionality and to bind to it. Imagine installing a workflow system and immediately after installation, the server found all of the other dependent services (LDAP, single sign-on, etc.) and registered the bindings to those services. This is the vision of a service oriented enterprise - it focuses on the simplified integration of systems by using ubiquitous protocols, standardized contracts and late binding on a dial tone network.

Whitehorse & Virtualization

Microsoft has been touting a next generation designer for creating services and then facilitating the deployment of the services. As Microsoft puts it:
"When creating mission-critical software, application architects often find themselves communicating with their counterparts who manage data center operations. In the process of delivering a final solution, the application's logical design is often found to be at odds with the actual capabilities of the deployment environment. Typically, this communication breakdown results in lost productivity as architects and operations managers reconcile an application's capabilities with a data center's realities. In Visual Studio Whidbey, Microsoft will mitigate these differences by offering a logical infrastructure designer (Figure 19) that will enable operations managers to specify their logical infrastructure and architects to verify that their application will work within the specified deployment constraints."

The environment allows you to drag a service description to a physical node and drop it on the node to signify deployment.


At first this seemed like a great idea, but after spending some time with the IBM grid team they quickly reminded me that "services belong to the network, not a predefined physical node". Thus, hardcoding service locations to a physical node kills the benefit of virtualization. Well - I agree - you want to drag your service to a "service network" and there should be various sub-nets that are partitioned based on resources and capability.

But, I must admit... the Whitehorse demo sure looks cool... It is very "Microsoft".

Mach3 Turbo ESB

This is a must read:
http://www.theonion.com/opinion.php?i=1&o=1

Come on... shouldn't someone do the Turbo ESB???

You're ESB only does queues, routing and transformation! Mine does all that plus [fill in blank]. Think I'm joking... just wait.

WebMethods to Close Office, Lay Off Workers

I almost missed this:

"WebMethods lost $11.1 million in the quarter ended Dec. 31. Since it went public in 2000, WebMethods has only reported a profit in one quarter, $157,000 for the three months ended March 31, 2003.

"Their inability to show profitability has been a very sore spot for investors," said David Hilal, an analyst at Friedman, Billings, Ramsey & Co. "They are doing what they can to appease investors."
See:
http://www.washingtonpost.com/wp-dyn/articles/A20530-2004Feb6.html

When is a service not an orchestration?

When is a service not an orchestration? This is the question that Christoph asks?

His point is that all bpel orchestrations 'are services' and if one wanted you could use the orchestration as an agility mechanism. For example, you know that you need a service 'foo' - why not make an orchestration 'foo' where the 'foo' orchestrations calls the 'foo' service. In essence, you treat the orchestration like a proxy to the end service. Christoph goes on to point out that like any proxy-calls, you incur extra overhead. By proxying, you don't tie the 'foo' service directly to the non functional requirements (logging, security checks, business activity monitoring, etc. ) you could add additional steps into the orchestration to perform these for you.

Overall, I'd have to agree with where Christoph landed - if the orchestration is time sensitive or doesn't look like it will need any 'decorators' - then just make it a service. If it is part of a long running business process and isn't necessarily time sensitive than consider front-ending the service with an orchestration and add the decorators.

In the long run this may kind of 'orchestration-proxy for NFR's' might go away. I'm still a big believer (as is Christoph) in using declarative aspects on services. In this manner, you would declare the decorators on the service and when the service was called and it would trigger the decorator calls (think Aspect Oriented Services). Either way... I'm not sure which one is easier to read / debug. Although it does raise a good point - we probably need a metatag to state if a service call is part of a NFR resolution or if it is a legitimate Functional step in the orchestration...

OpenStorm Launches

Today we are officially launching OpenStorm Software. See: http://www.openstorm.com We unofficially launched the company in August of 2002 - but went back into a semi-stealth mode to significantly enhance the product.

Our primary goal was simple: create a product that had the simplicity of a Microsoft tool and the scalability and reliability of an IBM tool. We think we did it. First, we took our original .Net orchestration engine and rewrote it in Java for the J2EE platform. The design leverages a message oriented architecture and facilitates logical and physical tiering between the 'communication server' (http, etc.), the message broker (JMS, MSMQ) and the orchestration server (native BPEL, exposed as a service).

Then, we rewrote the entire user interface for the developer tooling. We chose the .Net platform for the UI for a couple of reasons: 1. We believe that we can build better interfaces in a shorter period of time than on just about any other platform. 2. We intend on facilitating "Office Orchestration" - enough said.

The only issue we are having from a timing perspective is that the .Net Web Service Enhancements (WSE) remain in beta. Thus, we will be waiting on some enhancements before we make the .Net BPEL server available. We also decided that we wouldn't make the first release downloadable. Instead, we are inviting partners and qualified prospects to install the system and work with an assigned OpenStorm team to ensure success.

I think the product turned out great. It will serve as the foundation for service oriented integration (A2A and B2B). It will also serve as the underlying engine for the next generation of BPM (Service Oriented, Process Driven Architectures).

Lastly, I want to congratulate the team - you kicked ass. They are now busy working on the next release. Stay tuned.

BEA on Web Services

I largely disagree with the BEA position on web services:



Now, of course, an intelligent app server can detect if the two services are local and bypass the marshalling. I think what BEA is saying is "our web services implementation isn't smart yet..." - and that's Ok.

If you take BEA's advice - you'll likely end up with a tightly coupled, Java based application. Thus, you only integrated at the edges. I'm not a fan of this approach. Don't get me wrong - not everything should be a service - but saying, "you should place business logic in custom Java controls instead of web services" is definitely not my way of thinking! In my humble opinion, this is one of the faults with the WebLogic Integrator. It leads to 'inlining java code directly into the process code' - - which quickly turns into a mess.

The Case for Orchestration Tooling

As the Stencil Group points out, moving to a modular system has great benefits:




Now, if you were to look at the back side of the 'modular system' you would notice that there are wires running in every direction, hooking all of the modules (or services) together. If you've ever had to wire together the components in your stereo system, you know what kind of nightmare this can be.



The neat thing about monolithic systems is that they don't expose the ugly wires to the outside world. Of course, there is a huge downside in this approach as well - you're stuck with the whole elephant. The goal of orchestration is to get the best of both worlds: the flexibility of a modular system with the simplicity of a monolithic system. In most enterprises, you won't be dealing with a small handful of components, it will be hundreds, if not thousands of services that need to come together. Imagine trying to wire a stereo system with a thousand components...

Service Types?

I'm looking for some common vocabulary to describe the various nomenclatures found in service operations. Here is what I've been using:

Pass-All Service – This is a service that passes the verb and noun into a generic service entry point. An example would be: “do( Add, Employee, EmployeeInfo)” , or do("insert into Employee ...")

Verb-Only Service – This is a service that states the kind of action that will be requested in the operation name, but passes in the nouns. An example is “Add(Employee, EmployeeInfo)”

Verb-Noun Service – This is a type of service that states both the action and the nouns (the target of the actions) in the operation name. An example would be AddEmployee( EmployeeInfo ).

Is there already terminology for these different interface styles? If so, please send me a note.

BPEL Data Mapping

One of the early lessons that we learned was that data mapping between xml documents was a tough thing to do manually. BPEL leverages XPath extensively - thus to do it right, you need an integrated XML data mapping and XPath manipulation tool to facilitate the mapping between service calls. See:



In the OpenStorm product this tool gen's the code for the 'Assign' tag based on drag & drop interfaces. We are finding that it significantly reduces the potential for syntax errors. It has also reduced the time to create complex orchestrations by about 40%.

Web Service Documentation

It is great to see that products are coming out to help document web services. This is another step in the maturity process. Now... we need people to learn how to write good documentation for their wsdl's and bpel's.

Take a look at this:
http://www.wsdox.com and the samples: http://www.wsdox.com/samples/index.html

I'm very excited to see the BPEL documentation as well. The team as Wsdox are also planning on authoring guidelines about writing good documentation for web services. In addition to tooling, we need best practices.

In my inquiry, they also mentioned some interesting notes about their roadmap including: documenting the WS-* aspects of services, making the tool a web service :-), documenting some of the other artifacts (XSL, XQuery, etc.) -- very good stuff. I'm also glad to see that they are OEM'ing the software - so many products should be using this kind of stuff...

The Twenty-Twelve Stack

Patni issue oversubscribed 22 times

Offshore software development continues to be hot for the investment community:

The initial public offer (IPO) of Patni Computer Systems has been oversubscribed 22 times. The company is raising $95 million, which is the largest thus far by a software company in the last four years. In 1999, HCL Technologies had come out with $182 million issue.

Amit Chandra, member of the board, DSP Merrill Lynch, said, “The overwhelming response to the Patni book-built issue during the extremely choppy secondary market environment shows the strong investor appetite for high quality Indian paper.”
...
GE has a 10 per cent stake in Patni and relies on the US company for about 45 percent of its revenues.



This is the company that my brother in GE Atlanta lost his job to. Way to go GE! Cut costs by replacing American workers with Indians, invest in Indian companies and help them go public! GE - you're one class act. Maybe they could take their proceeds and try to get that pesky law about freed slaves overturned.

From: http://www.business-standard.com/today/story.asp?Menu=1&story=33841