More on Pass-by-Reference

Stefan Tilkov made an interesting point, " I believe anytime you make a copy of some data - which is unavoidable if caller and callee don't reside in the same address space - you can't do pass by reference."

Here are some things that I've been thinking about:

----
Pass by Value implies the creation of multiple copies of the data and any changes made to one copy do not propagate to the other copies.

Pass by Reference implies a single 'master' instance of the data; thus all changes made to one copy apply to all references.
----
Pass by Value implies that the data 'value' is created at the time determined by the sender.

Pass by Reference implies the data 'value' is populated at the time determined by the receiver.
----
Pass by Value implies that the copies of the data were a snapshot in time. Thus the data may become stale between accesses.

Pass by Reference implies that the data is 'current' - however, it may not reflect the state of the data as intended by the sender.

Both Pass by Reference and Pass by Value can use a 'leasing' mechanism to deal with stale information or stale resources.
----
Pass by Value implies that the contents of the message is data. (Out of scope: mobile agents and Jini style proxies)

Pass by Reference implies that the contents of the message is either a pointer to data, a pointer to functionality or both.
----
Pass by Value implies that the contents of the message are fully contained within the message, thus you pass *everything*

Pass by Reference implies that the contents of the message are not contained within the message and the recipient can choose at runtime which piece of the data they are interested in.
----
Pass by Value can be implemented in a single call.

Pass by Reference requires at least 2 calls; the first one sends the pointer, the second call retrieves the data (by value)
----
Pass by Value implies that the message contains the exact information desired by the sender (maintains data integrity)

Pass by Reference does not guarantee that the data being passed is that which was desired by the sender. This is due to the potential modifications of the data between 'send time' and 'retrieve time'. This problem can be overcome by using some sort of locking or checksum mechanism.
----

- I like the idea of not passing around 'fat messages' (extra information that the ultimate receive may not be interested in)
- In some cases, I like the idea of passing pointers to data.
- In some cases, I like the idea of passing XQuery expressions that point at virtual endpoints where the predicate is populated by the ultimate receiver and a lease verifies that the data/resource isn't stale. Here, I'm not sending the data - I'm sending a partial evaluation expression for the data.

However, you'll notice that this style of programming is starting to sound like what we did in the client/server days of stuffing all of our messages in our relational database and then passing around SQL statements that referenced the data. Issues here too.

Stefan is right in that remotely referenced information eventually becomes a *value*. But the Pass by Reference model has additional attributes that should be considered at design time. It is my opinion that PbV and PbR will both exist in the web services world and should be part of the toolbelt of the service designer. The aforementioned differences should be weighed when deciding between them.

I'm worried

I lived the dot-com life. My consulting customer base was every bad idea from Garden.com to BubbaJunk.com. I saw venture investors who were normally smart people throw their money away. I saw hard working employees work nights and weekends, dreaming for the next big IPO.

Well, in Austin it is now apparent that things are beginning to turn around. The first 'major' sign happened last night - the Austin Business Happy Hour returned. And unlike the last 3.5 years - the event was packed. People were optimistic - startups were emerging. Pretty much everyone had jobs and those that didn't have jobs had left on their own. The event was reminiscent of the dot-com days (minus the recruiters).

So, why am I worried? I'm worried for two reasons:
1. The startups that I'm seeing emerge look just as stupid as the ones I saw emerge in 1998.
2. The startups aren't going to get funded by VC's. Entrepreneurs may be quick to forget, but the VC's aren't.

I don't mean to be a stick in the mud, but business is still tough. Sure, things are picking up - everyone is doing better, but it is no cake-walk. Sometimes I get the feeling that the startups that are forming right now are based purely on the fact that the smart people that went into hiding (took a job with a big company), just can't stand working in the cube from 9 to 5 anymore and feel the need for speed.

Well, I hope I'm wrong. I enjoyed partying like it was 1999...

Pass by Reference : Answer #2

As previously posted, I've been looking at options for passing by reference. It has been interesting to see how people interpret this question. The first divide occurs in regard to 'what' is being referenced in the pass: pointers to data or pointers to services (or pointers to service that front-end data).

One thing that everyone seems to be agreeing on is that the 'pointer' is the URI. Last night I had a brief discussion with Dave Langworthy of Microsoft. Dave confirmed a couple of things. 1. There is no WS-EndpointReference specification, the portions of that spec that were relevant were rolled into the WS-Addressing specification. 2. I can accomplish what I want by using the endpoint reference functionality.

This threw me for a loop, mostly due to my lack of knowledge on this spec. Dave went on to explain that there are two primary pieces of the WS-Addressing specification:
1. The Message Header - This gives the from/to info for the header of the SOAP envelope
2. Endpoint References - This points to resources; these resources could be data, processors or other. Also, the Endpoint Reference is a 'type' which can be used in the body of a soap message. This means that you would define your WSDL 'message part' using an Endpoint Reference as the 'type' of a 'part'.

Within the 'EndpointReference' structure, there are a couple items of interest:
1. 'Address' - This is the URI of the data that you want to reference (or the actual endpoint referencing a service that gives you access to the data)
2. 'Reference Properties' - This is the bucket where you add qualifiers to zero-in on the information you are looking for. In my case, it will likely be an XPath expression that points to a specific element in an XML schema.

Summary - you can use an endpoint reference as standard type to act as a pointer to a resource. This standardized type can be leveraged inside of your own messages.

Sidenotes:
- The Address could specify a 'virtual' address that performs further resolution
- The Address could point to the URI of 'localhost' enabling pass-by-reference in local mode
- I'm not sure if there is a standardized way to drop in an XPath expression in a ReferenceProperty (anything special?)
- If the Address is referencing an XML document don't expect locking / synchronization, etc.

Web Services and Grid

Here's a nice presentation on web services and grid technologies (WSRF):

http://www.globus.org/wsrf/sabbah_wsrf.ppt

'Pass by Reference' in web services

I've been looking into my options for providing 'pass by reference' capability in web services. I know, I know - PBR is the root of all evil in loose coupling. But it is also the performance enabler for many applications. Thus, the goal is to identify the best mechanism that enables loose coupling while still providing performance.

My initial quest took me down a road that I'm calling "Pass By RESTference". I ran the concept by the smart guys on the Yahoo SOA group and it didn't get nuked (which is good in that group). The next issue is, how do you do it? Well, the closest example I've seen is probably this:
http://wiki.astrogrid.org/bin/view/Astrogrid/RestStyleWebServices

I'm going to chase down a couple other items... it has been rumored that WS-EndpointReference has something to say about this problem. In addition, I'm hearing that the OGSA guys have an alternative answer. More to report later.

Analysis of Orchestration Space

I know it isn't traditional to post your competitive analysis - but I'm not very traditional. Here is a copy of my internal analysis of the orchestration space (from an OpenStorm perspective).

=============================================
Don,
You were asking about competition. Here's a breakdown:

High-end EAI: (SeeBeyond, Tibco, Vitria, WebMethods.)
Big 3 (IBM, BEA, MS)
BPEL Pure Plays (Collaxa, FiveSight, Vergil, Choreology)
Web Service Networks with BPEL (GrandCentral)
Workflow (Dralasoft, Reactor)
Pure play BPM (Lombardi, Fuego, Savvion, Intalio)
JMS and ESB providers (SpiritSoft, Sonic, Fiorrano)
Web Service platform providers (Systinet, CapeClear)

The fact is... orchestration is a sweet spot and people are looking at it from many different angles. Here's my take on how it might play out:

High-end EAI - These guys are having a hard time competing on cost and putting together a message that doesn't bastardize their current high-end revenue stream. They will likely take a 'wait and see' attitude towards orchestration, putting out an offering, but not really marketing it. It is likely that they will spread some FUD about orchestration, saying it is slow or lacks functionality and that their proprietary offerings are more 'seasoned'. Ultimately they will have to play in the game or will become outdated.

The Big 3 (IBM, MS, BEA) have a different story to tell. Their story is "Orchestration is great, but it is only a small piece of the puzzle". They won't try to have the best orchestration offering, just one that integrates into their stack the best. IBM shops will want IBM, MS shops will want MS... These guys will get a significant piece of the pie; they always do - people buy here for peace of mind. BEA may have a bigger challenge; their offering lacks substance and BEA doesn't have the clout that MS or IBM have.

BPEL pure plays - perhaps a more interesting category. Collaxa came out with an early lead but then went on to put out 14 or so betas and still haven't knocked it out of the park. Vergil, Choreology and FiveSight are all still a bit early - mostly in stealth. Thus too early to tell. Expect to see these guys either partner, strategic partner or drop into a niche market.

Web Service Networks - A couple of months ago Grand Central announced that their web services network would support BPEL. Although I'm not sure, it appears as though they rolled their own implementation of BPEL - which would be odd for an infrastructure provider. The implementation had a significant number of non-standardized extensions which I believe they needed to more fully support their existing network. Maintaining a BPEL engine is a bitch - in the long run, I would expect GC to OEM an engine from somewhere else - unless they are getting out of the ASP market all together.

Workflow - Workflow and Web Service Orchestration may seem like a similar problem, but in reality they are very, very different. Some of the vendors that claim features similar (but different) will likely confuse uneducated buyers -but the customers that we want will know the difference.

Pure Play BPM - These guys have had every opportunity to break up their monolithic architectures into a component-ized (or service oriented) architecture. As far as I know, none of them have made significant progress down this path. Intalio seems to get it but doesn't seem to be delivering the message. If these guys start delivering a "SO-BPM" story (Service Oriented BPM), watch out - they could be real contenders (of course, they will have to have product to back it up).

JMS / ESB - Here there are two categories 1. Sonic, 2. Everyone else - The fact is, Sonic is a marketing machine. I tip my hat to them. These guys not only could sell ice to Eskimos - they do! In the short term, I think they will be the single biggest threat. However, they have some baggage they bring with them (JMS is commoditized, ESB will eventually be a joke) All of the other ESB vendors will have to compete based on product features/functionality/price (think bake-offs against Sonic).

Web Service Platforms - Oddly, there are really only 2 platform providers left (Cape Clear and Systinet). Both of the players will need to get an orchestration offering. Their issue will be to knock orchestration out of the park - otherwise they will have to lead with their less expensive and commoditized SOAP stacks - and then try to upsell to a more expensive offering.
===============================

If you believe in the 'web services programming model', 'service oriented integration' or 'pervasive integration' then it is only a matter of time before you realize that orchestration is the flagship of this rather complex offering. Protocol offerings should be built into the operating system (and will be). Queues are commoditized... and making 'web service enabled queues' takes about a day of engineering effort. The 80-20 rule on transformation makes out of the box, pipelined XSL offerings attractive. However, debugging a distributed, service oriented, message based, declarative policy based, multi-platform, asynchronous, concurrent, cross-enterprise process is.... "hard". The issue around orchestration will quickly turn to 'total cost of ownership' - the vendor that wins will be the one that allows the enterprise to reduce the cost of maintaining production instances (keeping the service network up, debugging issues, resolving performance problems, maintaining version, etc.) The offerings will have to go beyond orchestration and into 'composite distributed applications'. The expectations that will be placed on our new service oriented virtual machine (the service network) will be significant. As the new programming model emerges, people will be wanting a vendor that isn't a 'spot' technology but rather an enabler of the new model - that's where we come in.

Microsoft on RFID

From the RFID Journal, see:
http://www.rfidjournal.com/article/view/779/1/1/

Process Contracts

The concept of 'design by contract' was made popular in the object oriented world - or perhaps more precisely, in the component-oriented world. Much of the work that went into this study was centered around making a single module adhere to a contract and adhere in a consistent manner. When the Java language was introduced many people bumped into the 'interface' for the first time. This was the vehicle that guaranteed 'what' an object could do, but not 'how' it would be done.

This concept was extended with WSDL. The extensions included: defining the interface in a language neutral manner, allowing for multiple bindings between interface and implementation, allowing for a more message-centric design of parameters / arguments, strong support for remote calls and declaration of quality attributes as declarative policies (reliability, integrity, etc.)

In the process world, we continue to leverage the 'design by contract' mentality. Languages like BPEL expose themselves to the outside world through WSDL. This means that every process IS A service with a contract. In addition, BPEL makes all external calls VIA web services. Thus, all calls are first described via the contract.

As we continue to find best practices in designing BPEL code, we have tested a variety of means to get from concept to delivery. The one that seems to be getting the most ground is what we are calling 'process contracts'. Here, the analyst takes on a 'RAD' view of the process. The goal isn't to get all the detail right on the first pass, but rather to quickly identify all of the service calls that need to be made to compose the process. You can think of this almost like the old CRC exercises (Classes, Responsiblity, Collaboration) - except replace 'classes' with 'service calls'.

One thing that is becoming apparent is that people want to use BPEL in different ways. I usually put the usage scenarios into one of three different buckets: B2B, A2A and composite applications. In the B2B and the A2A scenario, it is common for the endpoints to already exist, but the contract to the service (type, message, operations) still needs to be created. In the 'composite apps' space, it is common for neither an interface nor an implementation to exist. In any scenario, the designer finds that they quickly need to begin creating the interfaces or contracts.

We are finding that most people are comfortable building out a complete process by 'stubbing' out the process. This involves identifying all of the major steps (service calls) and the flow/logic between the steps. At this point, the designer isn't going into any detail about the step, just putting boundaries around 'what it does'. The designer then goes back and starts to add detail around the contract of each call. This involves detailing out the WSDL interface (not binding and service). Here the designer begins to look at using common messages and types for consistent service calls. As the designer prepares the calls for invocation, he/she will usually begin to identify if the scope of the call is correct. This is a byproduct of identifying the 'fattiness' of the message, the 'chattiness' of the calls, etc.

All of this is happening up front - meaning it is happening before we begin to 'program in the small' via your favorite language to program services: Java, C#, etc. Upon completion of the process contract, you will have identified:
1. The interface to the process itself
2. The division of labor between the services
3. The interface for each service called
4. Verification that the data needed to call each service is available to the service (var scopes)

It is at this point where you can begin working with your service implementer (java guy, etc.). Here is how the conversation goes:

Jeff: "Hey Bob, I just finished my process contract for the new Fulfill Order process."
Bob: "Cool. How many service calls?"
Jeff: "Well, there were a total of 8 calls. 3 of them were canned and I will be sucking them out of UDDI, but 5 of them are new."
Bob: "Did you stub out the 5 new ones?"
Jeff: "Yea.. I did my best. I'll email you the BPEL and WSDL's"
Bob: "Better yet.. just check your process into version control and I'll pick it up there"
Jeff: "Good thinking. Will do. How long do you think it will take for you to turn around the implementations?"
Bob: "Jeff, I have no idea - I haven't even seen the WSDL's yet! Let me take a look and I'll get back to you..."
Jeff: "Understood. I'm going to move on to the Replenish Inventory process."
Bob: "Sounds good - I should have estimates in a few hours."

This conversation - and this style probably already feel familiar. The one thing that is improved is that in many cases, the process contract will become a new deliverable in the development cycle. It will force a more structured deliverable to come out of the late analysis or early deisign stage and may likely result in a more timely delivery of the end product.

Structured Process Cases

I've been playing with variations of the "Use Case" that are more process-centric and contract oriented. Overall, I think there is a good match between capturing process descriptions from an analysis perspective and tying that information back to the BPEL.

Although this is a very incomplete example, consider the following. A Process Analyst sits in front of their favorite tool (Microsoft Word) and writes down the name of the process, a small description and a handful of steps:



Now, the analyst adds a schema (or two) to the Word 2003 document by pulling up the "Tools: Templates and Add-Ins" window. In the example I'm using, they are adding in the BPEL schema but it might likely be a higher-level (more abstract) grammar that they choose.


Now, the analyst goes back to the document and "paints structure" into the content. This assumes that the analyst is given some extra training in such a process (think UML like training, "Process Case").



After painting the structure around the Process Case, the user is able 'hide xml' so that it looks like a normal requirements document. The analyst then takes the 'requirements' and sends it off to development. But, since it was 'painted' with a well known schema, the developer merely uploads the requirements document directly into their orchestration tool which stubs out the entire process for them. Now, of course there will be mismatches in syntax and scope, but the developer will make the changes and forward back the structured document for approval to the analyst.

At the same time, the developer will be creating a vocabulary of services. These will be marked up in WSDL (service:operation). Imagine terms like "checkInventory", "fillOrder", etc. These terms will then be made available to the analyst to drop into their future requirements documentation. By repeating this process, the links between analysis and design will continue to grow in strength.

This is still an early concept but our early tests indicate that we can significantly reduce development / integration costs by using the aforementioned closed-loop mechanism.

A first peek...

In the last few months, I've had hundreds of inquiries about the next version of the OpenStorm orchestration product. And although we are still in limited beta of the 2.0 release, I thought I'd share a single slide from the new deck.

The new product has a great feel to it, with some really interesting features. This is a shot of the main orchestration canvas, where you paint your process flow.



I'll post a bit more later on the product. Thanks to all of the people who have guided us through the endeavor and worked with us on the feature / function tradeoffs!

Does the Service Fabric make the SOA Obsolete?

I had an interesting question posed to me recently: Does the service fabric make the SOA pattern obsolete?

First, be clear. I have a very clear definition for the SOA:
1. It is an architectural pattern
2. It uses 3 actors (directory, consumer and producer)
3. Whereby, the consumer looks up an interface to a producer in the directory
4. The consumer creates a binding to the producer and calls it

The SOA pattern allows for dynamic lookup and binding, which means that it is a vehicle for finding the *right* implementation of a service at runtime. The service fabric often plays a similar role, but it often does it using routers. In this case, the fabric is aware of the implementations of a given service and when a message is sent to it, it can dynamically send the message to the *right* implementation. So, both techniques allow for a message to be dynamically delivered to a destination. But, does one make the other obsolete?

In my opinion, the answer is, "no". I am of the opinion that people will often mix the techniques. In essence, the service call will still be designed to do a UDDI style lookup, but the destination may likely point at a router. Alternatively, developers will continue to write services and populate the descriptions in a directory, then make their router aware of the directory. In this case, the directory is still being used, just by the router instead of the programmer.

Let's Merge!

Did anyone else catch this?



What? I say what?

Can you imagine the CEO of J.P. Morgan Chase and the CEO of Bank One Corp. talking before their merger about web services?
"WHAT??? Bank One isn't using web services - THE DEAL IS OFF!!!"

SeeBeyond does rush job?

I found this in the new literature for the SeeBeyond ESB/BPEL implementation:



For some reason, the phrase "a rapidly implemented, limited edition version" doesn't make me feel all warm about the product....

Here is how I read through the lines:
1. Product Marketing didn't give Engineering time to do it right
2. Engineering is likely pissed, and let marketing know it
3. Marketing is OK with it, because they still can't figure out how to position the ESB against their more profitable lines
4. A 'limited edition version' will give marketing more time to think about what to do about the 'creative disruption' of SOI

Features of a Service Oriented Language

Service Oriented, Protocol Connected, Message Based
Here are some casual thoughts on what I would like to see in a service oriented language. This is my first attempt at this... I'm sure I'll get some feedback ;-) and will update it.

Services
- The service is a first order concept, both sending and receiving
- Support for the strong interface (think WSDL)
- Mandatory support for Long Running Transactions
- Faults and Compensation are first order
- Protocols for transport and fulfillment of NFR are intentionally left out of scope

Messages
- Message is a first order concept
- Message is defined using a platform independent markup
- Support for message correlation properties; helpers for distributed state mgmt.
- Universal addressing scheme (assumes router)

Types / Vars
- Typing & vars are consistent with message system
- Remote variables (think REST)

Functional Containment
- The 'service' is a container (and managed)
- The 'object' continues to live. Objects are contained in services.

Invocation and Service Hosting
- WSDL's can be imported directly into the runtime. Operations and messages become first order citizens.
- Access & manipulation of binding / listener is first class.
- Language assumes that all systems are peer (both client and server)

Schema Manipulation
- DDL: Import / export / create / modify (consistent with type system)

Data Manipulation
- DML: transform (consistent with type system)

Flow Control
- Usual branching & looping
- Parallel execution; parallel joins (first order)
- Forced sequential processing

Event Based
- Events are a first order concept
- Time & activity based events

Metadata Ready
- Declarative metadata becomes a first order item (see jdk1.5)
- Service oriented loading / unloading of metadata / models / gen’d code at runtime
- Reflective knowledge of declarative non-functional polices (think ws-policy access)

Base Service & Extensions
- Concept of an extendable base service
- Service may have multiple interfaces
- Aspects may easily be applied to service
- Language is extended via ‘more services’ not ‘more syntax’

Service Network Awareness
- Service is aware of the network that it lives in (topology, routers, etc.)
- Service is aware of service-enabled remedies to non functional requirements (Virtualization, etc.)
- Service has the ability to modify the service network at runtime

OK. So a good question is, "which of these are part of the 'language', which are 'service libraries' or other?" I'm in favor of the *least* amount of required syntax possible. But, I like the idea of having *mandatory* service library extensions.

Clarification on Orchestration

In a recent ZapThink article, which has an amazingly similar name as my blog.... ;-)

I found this:


Ok. Pretty close... but I just want to clarify a bit.

First generation orchestrations are typically used as a process-integration or a system-to-system integration mechanism. This means that the orchestrations are VERY message oriented and the granularity between the operations are almost always coarse-grained.

In addition, there is a logical difference between "service orchestration" and "process orchestration". Typically, service integration is lower level. It involves the many calls to "technical services" and attempts to hide higher layer calls from the ugly technical details and are often exposed as a single business service. "Process orchestrations" are orchestrations that occur at a higher level. In these cases, virtually all of the calls are to other 'processes' and tie together a digital business process.

Now - in the future (2005+), I expect to see more fine-grained web services being used in 'composition' tools (a close cousin to orchestration). In this setting, more emphasis will be placed on in-lined services and performant service compositions. But for now, this is outside of the scope of what we typically classify as 'orchestration' or 'choreography'.

[BlogService]

[BlogService]
public class HelloWorldService
{
public String HelloWorld(String data)
{
return "Hello World! You sent the string '" + data + "'.";
}
}

=====================
Here is what I want to do:
1. Post source code (java, c#, etc.) on a new kind of blogging engine
2. Have the blogging engine compile my code and turn it into a web service
3. Host the service for execution (with wsdl retrieval)
Done.
Think Apache Axis (with .jws features) meets a blogging engine.

Automated Contextual and Conceptual Engineering

I just ran across something I wrote a few years ago... always interesting to look back at old notes...

Automated Contextual and Conceptual Engineering
I am attempting to convey a difficult concept to readers - right now. As I write this, my Microsoft Office is checking the spelling and grammar. It is putting my words and sentences into a context and using pre-defined rules to suggest areas of syntactic improvement. Dare I say it is using simple artificial intelligence (heuristics and a knowledge base) to improve my writing.

Visionaries have been promoting the concept of the Semantic Web for some time. By putting my words, sentences and paragraphs into context, the author is able to work with the software in a more advanced manner. If my word processor knew that I was writing a research paper on 'Automated Contextual and Conceptual Engineering', it could begin acting like an automated research assistant, scanning the web for applicable articles or illustrations, followed by suggestions on document structure, content, automated bibliographies, and footnotes. The more my software knows about what I am trying to write the more help it can offer.

Pushing context engineering to the next level takes us to conceptual engineering. Here, software not only understands the context of what I write, but understands the base concepts via conceptual ontologies. Once the software understands a concept it is able to look at the attributes of the concept and begin substituting alternative values suggesting related concepts. We would probably refer to this process as the elicitation of cross-domain metaphors or analogies.

Have you ever met someone that was good at connecting the dots in a business, scientific or personal problem? Typically these people are good at applying metaphors to problems. The goal of automated contextual and conceptual engineering is to create better content in less time while educating the author as he or she develops the content.

The Web has made it easy for anyone, anywhere to publish information. Browsers, cell phones and web pads are making it easy for anyone to read published information anywhere. I believe that the progress that we have made in mass-authoring content, cross-site syndication and ubiquitous rendering has made the world better. I now believe that the time has come to begin making the content better.

This is still an interesting concept. With the advances in web services, MS Office using XML and gains in the semantic web, this kind of stuff may be closer than I originally thought.

Things I'm Reading...

I'm in the middle of reading a few things. Here are the ones that seem interesting:

Short reads:
Steve Cook and Stuart Kent's OOPSLA report: The Tool Factory


Long reads:
Joe Armstrong's thesis: "Making reliable distributed systems in the presence of software errors"
Peter Van Roy's yet-to-be published book: "Concepts, Techniques, and Models of Computer Programming"

Blogging Terms

I'm going to break my number one rule about blogging: NEVER TALK ABOUT BLOGGING!

But, I read blogs - and there are a couple of things I've noticed...

BlogNosing
A variant of brown-nosing, this is the practice of always saying nice things about other people on your blog. Example: Bill did a great job of blah blah blah.. Bill always is right... blah, blah, blah. Don't get me wrong, often people will do good work and it should be acknowledged, but you know when you're BlogNosing...

Tightly Coupled Blogs
This is the practice of assuming that everyone reads your blog (and all of your friends blogs) every day and that in effect they have tuned into your mini-soap opera. I believe that Tightly Coupled Blogs were invented by Microsoft employees. Example: Don farted, then Gudge laughed, but that was cool because Tim and Dave from the Sicily project were walking by (in building 42) and it really didn't smell that bad... A person should be able to read a single entry in your blog - and it should be able to stand on its own. Long Running Conversations that carry Session and Identity between posts is a bad practice... don't worry, I won't do a Blog Coupling Index.

BlogSlapping
Although I didn't invent this... I do feel that I've mastered it ;-) This is the practice of calling someone out on something stupid that they've said/written. Quite frankly, I don't think there is enough BlogSlapping in the world today. In the near future, I'm considering having a special "BlogSlap Schneider Day", (one free pop shot) just to give everyone a little practice. :-) Damn, That will be FUN!

And no, these aren't 'blogging patterns'

LinkedIn

I'm now "LinkedIn" - feel free to create a 'connection'.
https://www.linkedin.com/

I don't know how to post a link to myself, so for now, use my name and email from the main search page.

Bob Martin Demonstrates His Knowledge on Web Services

Bob Martin, from Object Mentor decided to slam web services.

The reason I bring this up is because of how poorly he did it. Sure, there are plenty of ways to cut up web services, but unfortunately, many people still don't even have the basic concepts down. Even seasoned people like Bob Martin have such little knowledge on the subject that tend to confuse people, rather than actually making a valid point.

Bob says stuff like:
- it is rpc
- it is attached to http and we use it to get past firewalls
- it has a negative effect on coupling :-)
- it uses xml, which is "big, ugly and slow"

Ok, for those of you who are new to web services. It isn't rpc, it isn't attached to http, it has a positive effect on coupling and yes it uses xml as a typing system, which trades some performance for interoperability.

You know, I love when people have valid complaints against web services. There are plenty of them too. I'm reminded of the movie Roxanne with Steve Martin.



Do you remember the scene where he was in the bar and some big drunk called him, "Big Nose". And Steve Martin came back asking, "Big Nose? Is that the best you can do?" Then, Steve Martin went on to find 20 names for his big nose...

Hmm. Interesting. Maybe I should post "20 valid complaints about web services"... but, as always, I'll need your help! (valid complaints only, please... )

You and Your Research

An excerpt from, "You and Your Research" by Dr. Richard Hamming:

Now, how did I come to do this study? At Los Alamos I was brought in to run the computing machines which other people had got going, so those scientists and physicists could get back to business. I saw I was a stooge. I saw that although physically I was the same, they were different. And to put the thing bluntly, I was envious. I wanted to know why they were so different from me. I saw Feynman up close. I saw Fermi and Teller. I saw Oppenheimer. I saw Hans Bethe: he was my boss. I saw quite a few very capable people. I became very interested in the difference between those who do and those who might have done...

The Microsoft Drug

I just ran across a great presentation from Todd Proebsting of Microsoft. The following statement seems to sum up the MS philosophy:

2004 Hot Technologies List

It is time once again for me to make my predictions about the hot new technologies for 2004. Dominating this years list are items related to web services and alternative programming styles. Perhaps this is a prediction list - or maybe it is just my personal wish list...

1. Programming Model Convergence - The convergence and interlacing of the various programming models will likely surface to the top spot in 2004. As software vendors and enterprise customers consider their service oriented architectures, object oriented systems, aspects, model-driven architectures, integrated development environments and the other programmer facing technologies, they will find an inconsistent mess of technologies. 2004 will be a year of cleaning up the mess, both for ISV's as well as for the enterprise architects.

2. RFID - Already a hot topic, RFID is quickly becoming the "Y2K" of 2004. With the US-DOD and Walmart mandating the use of the technology, we will see the price of the tags and equipment tumble, opening up opportunities for new cost-sensitive applications.

3. Service Fabric - 2003 saw the introduction and early adoption of this enterprise enabling technology. In 2004 we can expect to see the infrastructure of web service networks continue to unfold. Look for less emphasis on "web service management" (reactive software) and more emphasis on intelligent service fabrics that proactively resolve quality related issues. Also look for the ESB to continue to gain ground, but eventually to be rolled into a small handful of services that the fabric handles. Lastly, it is likely that the protocol vendors and the fabric-via-service vendors collide, with the winner being the group that manages to pull protocols and services into a single product line.

4. SIP-based Enterprise Messaging - Many advanced organizations currently use instant messaging as a core communication vehicle. However, mainstream business has not yet adopted the technology. I believe that 2004 will be a chasm-crossing year for IM in the enterprise. Corporations will likely bring IM servers in house for security reasons - eventually, it will be granted a similar role as email.

5. WS-* Rosetta Intermediary - As web services continue to be adopted, a new breed of protocol translation service will emerge. This service will act as an intermediary that resolves differences between protocols introduced by vendor one-offs, competing standards and versioning. This technology will have a similar role that the 'gateway' or 'bridge' had in early LAN environments, only it will focus on the web service protocols. (Note: this topic is not related to Rosetta-Net)

6. IP telephony - Although this is far from being a new technology, I am predicting that 2004 is an adoption year. The number of vendors offering the service has increased as well as the functionality of the implementations. We are also starting to see a market emerge for VOIP add-on products.

7. Independent Invocation Models - Most invocation models are platform / language specific. 2004 will be a breakthrough year where the invocation model is viewed as a platform independent artifact. Just as WSDL created a platform independent entity for describing the server side interface, we will see new entities created for describing the client side invocation scheme. Concepts from WSIF will be leveraged, but the mechanisms will be ages ahead of what are currently available.

8. Presentation Offerings - For the last several years, the browser has dominated as the primary presentation (UI) vehicle for applications. In 2003, a handful of startups and established players built early versions of alternative user interfaces. Look to 2004 to see a real fight for adoption of these next-gen user interfaces.

9. Advanced SOAP Foundations - Much of the work that has been accomplished in the SOAP space has laid a foundation for the normal use cases. 2004 will usher in more advanced uses of SOAP including multicast SOAP, in-proc soap, async soap, etc. In addition, we will see the ws-* specifications enter into the mainstream. In many cases, SOAP will start to be viewed as a semi-static, standalone document with editable fields that can be routed to a destination specified by its header.

10. Disconnected PM's - The Internet and the web has most programmers thinking in a 'connected-only' manner. However, with the release of SDO many Java programmers (in addition to their .Net ADO counterparts), will begin designing systems with disconnected programming models. This will eventually lead to XML encoded formats that leave a time based change history that will be leveraged by both the .Net and the J2EE platforms. This technology will re-introduce batch style off-loading of non-time sensitive data and force synchronization vendors to become compatible with the newer technologies, thus creating some level of interoperability in data synchronization.

One underlying theme that may be noted is that many of the 'hot technologies' are still at the conceptual level and many of them are buried deep in the technology stack. As packaged applications like SAP have matured, we are finding that completely new paradigms are needed to bring a new level of functionality. The technologies that were nurtured over the last couple of years and are slated for release in 2004 will offer new programming shifts and ultimately will lead to a whole new generation of applications.

Is the JCP obsolete?

Richard Monson-Haefel, a man who has seen Java from the beginning responds to an interesting question, "Is the JCP (Java Community Process) obsolete?

Richard defends it.

Perhaps it is no surprise that I think that Richard is dead wrong and that the JCP is largely obsolete. The Java game used to be about creating enough mass for wide spread adoption of a platform (J2SE, J2EE, J2ME, etc.) Well, this has already happened. And more importantly, a couple of vendors (IBM and BEA) have come out on top.

Richard states, "The truth is, IBM and BEA need the JCP. Neither of these companies could go it alone, outside the JCP. " IMHO, this is one of the most naive statements I've heard in a while. IBM and BEA realize that they the competition isn't Sun, Oracle or Geronimo - it is Microsoft. And as long as they have the JCP dragging their innovation cycles they will be competing in an uphill battle.

So, is the JCP obsolete? My answer is - partially. It is obsolete when the IBM / BEA need it to be. There will continue to be many technologies that aren't critical path that can be run through long laborious debates (aka, jcp). But, for those technologies that are core to competing against Microsoft, I anticipate (and hope) that they take the quickest path with the least drag on the innovation cycle.

The Perils of Predictions

I'm writing my 2004 hot technology list at the same time that I'm outlining a paper on converging programming models. I'm quickly reminded of the perils of predictions...



Hmmm... perhaps the "Constraint Based Enterprise" ? ;-)

Top 10 Technologies of 2003

Last year at about this time, I posted my top 10 technologies for 2003. Overall, I'm pretty happy with my calls.

1. Business Process Execution Language (BPEL)
Yep, still a good call. In 2003, it became apparent that BPEL won the digital process language war. Look for adoption in 2004.

2. Web Services II / GXA
This referred to the ws-* stack. Good progress was made on standards - not much adoption.

3. Microsoft .Net
This was an easy call. MS .Net kicks ass - however their long development cycles on the tooling isn't helping.

4. Flash MX
OK, this one didn't really happen to the extent that I was predicting. Instead, I believe that we are seeing a general trend towards richer functionality user interfaces, some leveraging xml or web services.

5. Industry XML Standards
This one did OK, but could have done much better. I think that many of the standards bodies advanced their specifications and cleaned up their schemas.

6. Business Process Management (BPM)
OK, this was a bad call. BPM has sucked wind due to the 'architecture in a box' syndrome. BPEL with MDA will help.

7. Portlets
Portlets did get off the ground - but no where near what I'd hoped. Perhaps next year?

8. Total Value of Opportunity (TVO)
Bad call. Does Gartner even promote this? It sure seemed like a good idea...

9. Software Asset Reuse
Terrible call. Software asset reuse requires a major change in the programming model.

10. Extensible Business Reporting Language (XBRL)
OK, this one actually got a bit of traction. However, I'm not sure if it was hype or actual usage.

HELP - I'm running out of time! I still have to write my top 10 enterprise software technologies for 2004. If you have ideas, please send them to me: jschneider AT momentumsoftware DOT com.

What is a programming model?

My recent entry on programming models generated a couple of responses. Shane Claussen of IBM commented that the Service Network is the model, or perhaps the SoA is the programming model. Now, I fully agree that a service oriented architecture presents a programming model, but I am not sure if it should be considered the 'first-order' model.

Per my request, Shane went on a took a couple of stabs on the definition of a 'programming model':
1. A PM is the tools and methods for building a service or an application AND
2. The PM is the visible components of an architecture that a programmer codes to/with.

Hmmm. I like these definitions. I've scanned around the internet a bit... and haven't found much better.

But Frank Martinez, CTO of a leading service fabric company (BlueTitan) posed a new question:
What's more important (i.e. strategically relevant) to "you" (i.e. the practitioner)?:
1) Infrastructure-independent tooling
2) Tooling-independent infrastructure
3) Infrastructure-driven Tooling
4) Tooling-driven infrastructure

He went on to state, "My personal answer (and preference) with respect to your question would be that your service network should enable (and encourage) a variety of service-oriented programming models...each of which could be business driven in it its own right. Additionally, your service network should feature a service-oriented extensibility model that shouldn't be constrained by any one given programming-model."

I like this. The only thing I would add would be to encourage non-service oriented programming models in the network as well. Blasphemy? Perhaps, but I hope to make my case over the next several months.

"What is my Service Oriented Extensibility Model?"

Reading Material - Flight Home

Here is something to read for the flight home...

Edsger Dijkstra, way ahead of his time, "Hierarchical Ordering of Sequential Processes":

Service Programming Models

Once again, I am on holiday break, visiting relatives - tucked away between some farms in Illinois. But luckily, StarBucks is only a mile away. Each break, I attempt to write something. One year I wrote the Java book, another I wrote Service Oriented. This year, I'm feeling as ambitious as ever, but the subject seems more complicated than those that I've pondered in the past:

"Does the architecture of your service network define your programming model or does your programming model define your service network architecture? "

Well, that is my starting point. Perhaps I'll throw out both notions and go somewhere completely different.

Ivar Jacobson - Trends

What is Ivar Jacobson talking about these days?

Abstract: In this talk Ivar Jacobson will discuss four trends, major trends that he believes will change the way we develop software from being machine centric to becoming human centric.

The trends are:
- Closing The Gap - business driven enterprise application integration
- Building Extensible Software with Aspect-Oriented Programming
- Making Software Process Execute with Agents.
- Executable UML -- eliminating the two language problem

Advice from my mentor

I had a chance to sit down with my technical mentor tonight.
He had two pieces of advice for me:
1. The "true" bus is TCP/IP.
2. BPEL should be an enabler not a disabler. Services should be part of a fluid programming model - not a cumbersome extension.

Compositions, Services & Fabric

I thought I'd pull a slide from our deck...

The Service Network

I've identified 100 services that could be considered part of an enterprise "Service Network". These services can be sub-divided into two categories:
A. Services that fulfill non-functional requirements (the ilities)
B. Services that are functional building blocks to modern applications.

The service network goes far beyond queues, routers and transforms. It serves as the foundation for creating a true service oriented enterprise. Some of the items will be found in 'service fabrics' others in a 'service bus', but most will not. Rather, most of the services will end up levaraging your fabric or bus.

I will contend that is the job of the architect to begin creating your enterprise wide service network - lay the fabric, plug in a bus... do what you must, but start your network. Implement a consistent foundation for the other services. Demand that your infrastructure vendor explain *how* these services come together in their offering. Know which are in scope - which are part of the vendors roadmap - and which are not. Set your expectations - create a plan - and get moving!

Common Services in a Service Network
1. Publish / Subscribe Service
2. Configuration Services
3. Broadcast Service
4. Page Controller Service
5. Syndication Service
6. E-mail Service
7. Relational Database Service
8. Object Database Service
9. Authentication Service
10. Access Control Service
11. Routing Service
12. Encryption Service
13. Hashing Service
14. Data Caching Service
15. Business Rules Engine Service
16. Business Object Service
17. OMF Service
18. Application Deployment Service
19. Version Control Service
20. Directory Lookup Service
21. Content-based Routing Service
22. Orchestration Service
23. Choreography Service
24. Queuing Service
25. Notification Service
26. Presence Detection Service
27. Protocol Translation Service
28. Tuple Service
29. System Monitoring Service
30. Legacy Adapter Service
31. License Management Service
32. Logging Service
33. Transformation Service
34. Job Scheduling Service
35. Peripheral Services (printer, etc.)
36. Language Translation Service
37. Presentation Service
38. Workflow Service
39. Gateway Service
40. Background Service
41. Network Browser Service
42. Indexing Service
43. Search Service
44. Collaborative Filtering Service
45. File Manipulation Service
46. Hardware management Service
47. QOS Management Service
48. Telephony Service
49. Installation Service
50. Policy Service
51. Provisioning Service
52. Fail-over Service
53. Clustering Service
54. Data Aggregation Service
55. Reporting Service
56. Formatting Service
57. Imaging Service
58. Faxing Service
59. OCR Service
60. Transaction Service
61. Benchmarking Service
62. Testing Service
63. Media Conversion Service
64. Compression Service
65. Parsing Service
66. Compiling Service
67. Speech Service
68. Portal Service
69. Single Sign-On Service
70. Instant Messaging Service
71. Concurrency Service
72. Parallel Execution Service
73. Streaming Service
74. Trust Service
75. 2D / 3D Rendering Service
76. Profiling Service
77. Session Management Service
78. Synchronization Service
79. Timer Service
80. Remote Publishing Service
81. Validation Service
82. Archival Service
83. Journaling Service
84. Naming Service
85. Life Cycle Service
86. Analytics Service
87. Debugging Service
88. Obfuscating Service
89. Sorting Service
90. Merging Service
91. Error Detection & Correction Service
92. Expression Evaluation Service
93. Leasing Service
94. Marshalling Service
95. Serialization Service
96. Filtering Service
97. Content Management Service
98. Web (HTML) Service
99. Terminal Service
100. Resource Manager Service

In addition, be aware that not all web service infrastructure vendors are service vendors. Many of them focus on the "Protocol Network". That is, the layer beneath the Service Network that provides functionality through protocol realization (SOAP, UDDI, WSDL, WS-*, protocol tooling, etc.) In many cases, these vendors also have service network offerings. It is good to understand where their offerings start and stop - and how they help enable consistency in the service network.

"Yea, I'm building an ESB..."

Yesterday, I met with a consultant that I used to work with. I asked him what he has been up to and he told me that he is writing an ESB. To which I could only respond, "just you?" And he answered ,"yea.. just me, why?"

He then went on to explain to me that he was building his ESB on top of JBossMQ. And that 'what' he was building was the 'extra services' like validation, transformation and a tad of routing. He made a real interesting point. There isn't a whole hell of a lot in an ESB. The JMS based message queue is commoditized, validation services can be ripped from probably any thin client J2EE app, transformation is usually a prepackaged library, and light-weight routing isn't rocket science. He told me that he had some other services planned and that he would likely throw his version into the open source community when he was done.

Hmmm. I found myself almost speechless. But, I gave him some advice and wished him best of luck.

In the past, I've made fun of the ESB - mostly because of the pure amount of hype that has gone into it by the media and one analyst and a few vendors. But it has now occurred to me that making fun of an ESB is like making fun of 'JavaDoc' or 'log4j'. It is a very small commoditized piece of software that will work its way into many applications. I don't know if there is a big market for reselling 'log4j' nor do I know if there is a big market for the ESB. Frankly, the "Apache ESB" sounds about right. There may be a bigger market for the sophisticated ESB leverage a service fabric and can be leveraged by high-end service composition tools. But as far as I know, no vendor has gone done that road (yet).

Perhaps we could house the new open ESB here... along with handful of other buses that my partner in crime, James Higginbothamhas written over the last 7 years...

Sorry, we still need programmers...

This kind of thinking is just plain silly:
"In theory, because composite applications use software components, they can be built by a trained business analyst rather than require the services of a full-fledged programmer. "Just as you don't need to know how a Web browser works to create a Web page," says Jason Bloomberg, senior analyst at ZapThink, "you won't need to know how one software actually interfaces with another to build a composite application."

People, we are so far off from this concept... where business analysts do some magic stuff and software pops out the other side.

These kind of comments are dangerous... they just make us look like bozo's that over promised and under-delivered. We are moving into an age of contract based development, not "miracle-based development". For some period of time the complexity of putting together service oriented software will INCREASE not DECREASE. We are trading reusability and agility for a decrease in performance and increased complexity. Programming a distributed, heterogeneous web service stack with business logic spread across the network isn't simple. Adding a set of 20+ new specifications won't make life easier either.

Business analysts will create business cases.
Process analysts will create better processes.
Architects will divide up the problem.
Service analysts will encapsulate the problem into small components.
Platform coders will program them.
Orchestration designers will tie the services back together.

No magic, just good ol'fashion hard work from a variety of specialists. Hey, I look forward to the day when 'Service Oriented BPM' is here too. I'm working on it - my competitors are working on it, but promising it now is not in our best interest.

Cape Clear Takes On "Extra Baggage"

Annrai O'Toole, CEO of Cape Clear had commented, "O'Toole claimed that complicated protocols like the Business Process Execution Language for Web services (BPEL4WS) or the Web services Choreography Interface (WSCI) are extra baggage and that any functionality that those specifications are designed to handle is easily covered by something found in virtually every system: JavaScript (also known as ECMAScript)."

Since then... he apparently has changed his mind, referencing the Cape Clear suite... "The product also supports Business Process Execution Language (BPEL) for sequential workflows and composite Web services. Through this support, users can chain sets of Web services to deliver a single, composite service, Cape Clear said. "

Well, good!

The recent article went on to say, "The Cape Clear Data Interchange is a lightweight alternative to EDI and ETL (Extraction, Transformation and Loading) and allows such documents as spreadsheets, order forms and expense reports to be exchanged between applications regardless of format."

The requirements of *real* ETL software are significantly different than those of a messaging system with transformations. In the few instances where I've seen vendors attempt to build both features into a single product they usually end up with half-baked attempts at both. My guess is that Cape Clear did something more like an ESB but is casting a broader net to see what fish they catch... more investigation is necessary.

Paul Brown publishes a great BPEL presentation

Paul Brown of Five Sight has published a great BPEL presentation. He has some great insights and manages to explain complex issues using simple terms.
Well done!
See:
http://blog.fivesight.com/prb/space/BPEL/BPEL4ProgArchies.pdf

Process Oriented Uptime

As more and more applications are moved to a process-centric approach, I believe that we can anticipate a fundamental change in the systems management layer. Today, most systems management products tend to focus on measuring the uptime of a computer, a cluster or a piece of software (like an application server). As our applications become more distributed across the network (web services or other means), it will become essential to view the uptime (and other quality concerns) from a process perspective where the system would scan all of the involved services and their respective non-functional concerns. After all, a sales manager isn't concerned with the up-time of a WebLogic instance; rather, he/she is concerned with the ability to work the sales process.

Integrating software quality concerns back to the business process at hand will also allow people to prioritize the processes and re-allocate resources according to the business impact. One could see where the various 'on-demand' efforts that are being pursued could come in very handy in this scenario.

ZapThink Kicks Gartner Below the Belt

In the last fifteen years of following I.T. analysts, I am unable to remember when one analyst group called another "dead on arrival", but that is exactly what ZapThink has done.

Going far beyond their bounds of providing useful, unbiased information, ZapThink took direct aim at Gartner and called their web services framework "Inaccurate", "incomplete", "unhelpful" and an example of "a horseless carriage".

In my opinion, the decision for ZapThink to take public aim at an older Gartner report lacks professionalism and maturity. Perhaps it is time for ZapThink to reconsider their own reports, to concentrate on their customers and avoid the below-the-belt attacks on the competition.

See: http://www.zapthink.com/report.html?id=ZAPFLASH-12032003

BPM Twist

Here's an interesting BPM twist. A company called, "Clear Technology" claims:

"Tranzax is the industry’s first business process management platform designed specifically to capture and replicate the business processing behavior of an enterprise’s best employees, and then optimize and extend these best practices across the enterprise."

I have no idea if they can pull this off, but I like the sound of it. It does raise an interesting question. If you force everyone to use the same process, how can you expect process innovation to occur?

RFID Chips used for Mind Control

According to the former Chief Medical Officer of Finland, RFID chips are now being used for mind control. Now, I know what you're thinking... the transmission and storage capabilities of RFID are so low, how does it work? Well, I don't know. Apparently, the trick is to hook the antenna up to the brain stem and the electricity wires up to the cerebrum and then 'think' real hard.

The application for the device is targeted at creating a more choreographed dance line. Joseph O'Shea, the producer of 'River Dance' commented, "Gettin all these dancers to move together is a real pain. That's why we are switching to RFID Mind Control". However early tests of the device were less than successful.



We determined that midgets and fat people are less susceptible to the radio waves, as well as people that have consumed large amounts of Mad-Dog 20/20. More recent tests have confirmed that the systems works best with gay Irish men dressed in black.



The company behind this venture, "Coordinated Dancing Inc." are considering new markets. Unfortunately, most of the senior management team has been infected with gangrene of the medula which has left them without control of their bladders and urinary tract. CFO, Jimmy McDonald commented, "We've got a think tank working on the issue now - everything from more company toilets to bulk purchases of 'Depends' - we will not let gangrene of the brain slow us down. Inserting 10 cent chips into the brain of every man, woman and child is the future!"

A Service Oriented Coupling Index

In December of last year, I issued a challenge (really to myself) to work on a 'service oriented coupling index'. I was searching for a quantitative way of determining how loose or tightly coupled software entities are. I took a look at much of the academic literature but found that most of it was out of date or needed rethinking in the service oriented world. More recent work, such as that performed by Doug Kaye was a great help in the pursuit.

At the end, I produced an initial report called, "An Inter-Service Coupling Index for Lossless Exchanges"

I learned a couple of lessons in the process:
1. A coupling index is possible, however the quantitative aspect still lies in the eye of the beholder (which Doug and others warned be about). Thus, in many ways the coupling index becomes a 'best practices' in coupling guide.
2. The pursuit of the coupling index was extremely interesting. I am convinced that the value to be taken away from the report isn't the index but rather the insight on areas where coupling may still be reduced.

Please feel free to send me your thoughts (good or bad). I plan on putting out an updated version on Jan 1. And thanks again to all of you who sent me early feedback. Have Fun! jeff

MS Millennium Goals for OS

I just ran across an interesting paper from MS, see:
http://research.microsoft.com/sn/Millennium/mgoals.html
Perhaps Christian would be kind enough to blog on 'Longhorn' and how close it is to reaching some of the goals.

Service Data Objects

IBM & BEA released a new specification for Service Data Objects. In my opinion, this is a much needed and perhaps overdue specification. SDO introduces Data Objects and Data Graphs which are self-describing data containers that can be manipulated, serialized and navigated. The feature that really got my attention was the ability to create a 'change log' of the data set. In essence, this feature mimics the functionality found in .Net called, "disconnected datasets'.

It is also apparent that the specification writers have gone out of their way to plan for a service oriented world. The use of XML Schema and a SOAP binding are utilized. I'm getting the feeling that this will be an underlying work-horse for some follow-on specifications.

OpenStorm Demo

Starting in December, I will be giving some one-on-one demonstrations of the OpenStorm Suite to prospects.
Here are my initial travel plans:
Dec. 2 - Houston
Dec. 3 - Dallas
Dec. 4 - New York
Dec. 5 - Atlanta
Dec. 10-11 Philadelphia
Dec. 18 - St. Louis
Dec. 19 - Chicago

Most of these days are only partially booked. If you are considering a purchase in this space and the date/location works for you shoot me a note! jschneider AT OpenStorm DOT com. The talk will focus on Service Oriented Integration techniques, the Service Network and using BPEL as an integration mechanism.

I'll be setting up a west coast visit in January.

OCL for Web Services

Radovan wants OCL for web services (or more precisely, he wants a Service Constraint Language). I do too. As far as I know, a variation of the Object Constraint Language doesn't exist - let's call it SCL.

But be careful, just because you define constraints (pre-conditions, post-conditions, message verification - and potentially even the order of service participants), you haven't created a replacement for a fully defined business process. I love constraints - and I love fully described digital business processes - and I really love when the two are combined.

I've been having some offline conversations on the 'coupling index' - a means to quantitatively determine a 'loose or tight coupling factor'. One thing I noticed is that by having a centrally defined business process, we are able to have more fully encapsulated services (we shift knowledge out of the service and into the process). However, I've also noticed that the current state of web services fails dramatically in being 'fully encapsulated' - mostly due to the lack of constraints put on the operations. That is, a significant amount of knowledge beyond the interface is still required.

And yes, if we wanted... we could build an SCL as a web service... which means we could orchestrate the pre- and post condition calls. :-0 (not that you would want to... I'm still in that mode where everything looks like an orchestration...)

p.s., I'm a fan of point-to-point integration as well!

Service Hijacking

I've been playing with a BPEL concept that I'd like to share... I'm calling it 'service hijacking'. It goes like this:

Some company releases a web service; for example, Amazon. This web service has a wsdl describing all of the operations and messages. The wsdl is published in a public directory for consumption.

Some other company, "OnlineBooks", decides to launch a marketplace for book shopping. So they create a BPEL service that front-ends the Amazon service, but adds calls to "Barnes & Noble" and a few others. Perhaps they even kept their wsdl the same as the original Amazon wsdl in order to allow the 'consumers' to easily switch over. At this point, we have added new value to the consumer, so all is good.

Then, some other company, "MarketResearchBooks", launches a new service to front-end the "OnlineBooks" service. They support all of the same features, but also keep records of who had the lowest price and then sell that information back to the original retailer. And yes, to the best extent possible, they kept their wsdl looking like the original WSDL Amazon wsdl. At this point, we didn't *really* add new value to the consumer, but we didn't take any value away.

Then, another company, "OnlineSuckers", launches a service to front-end the "MarketResearchBooks". It adds no new value, but asks people to pay for the service on a per-use basis. Now, we have a problem. The value of the service went down, not up.

Making information available online in a structured format for public consumption is a tricky proposition. In some cases, you don't care what people do with your information, while in other cases (OnlineSuckers), you might care. In my opinion, a service is hijacked when the value of the service goes down (rather than up) from the re-purposing of the call.

The earlier examples ("Online Books" and "MarketResearchBooks") are what I call either 'service piping' or 'service chaining'. These are good things. They take information and add or maintain the same level of value. Of course, there is a different technical argument to be made against long chains of services, but I'll save that for another day.

---- after thought----
It just hit me that if you don't write BPEL scripts everday, you might be wondering what this concept has to do with BPEL. Yes, service hijacking can be accomplished through your favorite programming language (java, c#, etc.) However, BPEL facilitates this functionality with VERY little effort. With the OpenStorm suite, I can service chain Amazon in under a minute, add marketing statistics in a couple of minutes and hijack on a payment service in under a minute. I guess my point is that BPEL makes this stuff very easy to do.

BPEL Validator :-)

It might be time to get that BPEL Validator routine up and running.... just in case any company were to release a few BPEL extensions...

:-)

And congratulations on the release.

Bean-counters to Join Programmers

VentureWire had great news today. More accounting and finance jobs will be moved offshore!!

It is clear that geeky American programmers have no idea on how to influence congress. Surely our accounting brothers can help out...

=============================
Ephinay Raises $10M Series B
By VentureWire Staff Reporters 11/17/2003

Charlotte, N.c.Ephinay, a finance and accounting business process outsourcing firm, said it received $10 million in Series B financing. [full story]
http://www.ephinay.com
=============================

Outsource Partners International Raises a Potential $20M in Series B
By VentureWire Staff Reporters 11/17/2003

Outsource Partners International (OPI), a provider of finance and accounting outsourcing services, said it has raised $12 million in its Series B round. The round could potentially bring in $8 million more. [full story]
http://www.opiglobal.com

=============================

Two in one day!!! Kwe Kwe is going to get crowded!!

Junglemen Hex American Programmers

In a rare move, the normally friendly tribesmen of the Zimbabwe jungle put a voodoo-like-hex on the American programmers working there.

Frank, who was laid off from General Electric's I.T. department some time back explained, "After training my Indian replacement at G.E., I decided I was going to beat the game. If the trans-national corporations were only interested in low-cost labor, it was clear that I'd have to reduce my cost-of-living expenses. That's why our whole development team moved to the jungles of Zimbabwe."



"Unfortunately, we were found by the local tribesman. I don't fully understand their language, although it appears to be a blend of Morse-code and ASCII. From what I've gathered, they are concerned about too many U.S. programmers coming here." In an effort to remedy the concerns, Frank intends to meet with the local governing council. "I will explain to the council that we can put 'caps' on the number of American programmers that can come to the jungle - AND... they will be forced to leave after a certain number of years. In essence, we are pitching them a variation of the H1 and L1 programs!!!"



The city of Kwe Kwe, Zimbabwe is quickly becoming a technology hotbed. In addition to American programmers, recently displaced developers from Hyderabad, India are also flocking over. Amit Maheshwari explains, "Yes, we used to specialize in convincing American companies to come to India... now, we make our money selling the U.S. trans-nationals infrastructure to create wireless zones in the jungle. We have also tried to get them to subsidize the mosquito repellent, but so far haven't had any luck."

Project Liberty & WS-Federation

Project Liberty, is a federated trust & identity based scheme. It was created as a "Microsoft Passport Killer". A couple of years ago, MS was pushing Hailstorm and Passport as a mechanism to centrally control identity and schema based data. The fine folks over at Sun (and friends), came to the conclusion that they didn't want MS to control all of the user id's in the world - and for good reason. Thus, they came up with a specification to decentralize identity & trust. The program came to fruition just after the September 11th tragedy, and was given the very awkward name, "Project Liberty" - I guess they felt that they were 'liberating identity' or something like that...

Well, Project Liberty did what it was supposed to do. It created an alternative means to accomplish the same goal as Passport, without handing over the family jewels to MS. However, Project Liberty was created prior to the creation of the WS-* specifications. This means that for the most part, it has overlap with some of the newer specifications created, like WS-Trust, WS-Privacy and WS-Metadata.

I'm a huge fan of "concern-based protocols". Thus, I like having 'trust' as its own protocol - and 'privacy' as another protocol. I don't like mixing concerns in a single protocol; which I believe Project Liberty is guilty of. From a cursory view, it is appears as though WS-Federation covers the bulk of what is actually needed. I'm not an expert in this area - but so far, it looks 'good enough'.

The Project Liberty group recently published a paper comparing the approaches. Although the paper attempts to subtly convince the reader that their approach is better, for me, it has the opposite effect. They basically claim that they have successfully lumped a bunch of standalone concerns into one specification. In addition, they did it prior to the existence of the WS-* specifications, thus the implementations that are available won't be technically aligned with the needs of the next generation web service developer.

I'm not ready to say, "let's kill Project Liberty"... yet. But, I am mentally preparing for the funeral. In my opinion, Project Liberty did what it was supposed to do: force Microsoft down a standards based decentralized ID system. And this is exactly what happened... thus, I consider the project a raging success. But it served its purpose and now it may be time to move on.

New RFID Application - Knicker Surfing!

According to the Chicago Sun Times, "RFID chips could make your daily life easier, but they also could let anyone with a scanning device know what kind of underwear you have on and how much money is in your wallet".

At first I thought to myself - Wow, what an invasion of privacy! Then, I realized that the Chicago Sun Times may have just found the killer application for RFID. By targeting perverts, we will be able to sell millions of handheld readers to identify the 'kind of underwear' that people are wearing. This is genius!!! Unfortunately, I found out that there is already a growing population of what I am dubbing, "knicker surfers":



Gary, a regular knicker surfer reports, "Yea, it's cool. Me and my buddies come out here all the time and knicker surf. I just hope Walmart pushes PML. Right now, I can only get the underwear brand... with PML I'll be able to get the size too!"

I had no idea. :-)

Storing Transient Data

John Udell reports, "Today, most IT shops can't store or process massive flows of transient data. But XML message traffic is a resource that creates strategic opportunity for those who learn to manage it well. Tools for doing that are on the way. "

Hmmm... if I create a persistent store of transient data, is it still transient? Maybe there is a reason most IT shops don't store transient data - wouldn't that just be considered 'persistent data'??
:-)

Ok, I understand what he means - perhaps instead of 'transient' maybe we could call it 'inter-service message data' or just 'message data'? Still, I'm not sure that I buy into the concept. Most I.T. shops do a significant amount of warehousing and reporting off the system of records that generate the messages. The reliability side is currently taken care of by message queue journaling, and realtime inquiry on state is best handled through an inquiry to a business process engine or BAM notification.

Right now, collecting transient data sounds like a bad habit... I need a use-case, with strong, strong justification.

The Realist and the Idealist

I recently had the opportunity to engage in a technical discussion with Chris Sells. We found ourselves agreeing to disagree. He took on the role of the realist, I took on the role of the idealist.

First, Chris and I seemed to be in agreement that distributed computing was easy to screw up. As he stated, it was necessary for consultants to travel the world preaching about *round-trips* (overly verbose message exchanges).

The Realist
Now, I hope I don't screw this up (Chris, correct me if I do).
Chris is of the opinion that we shouldn't paper-over the complexities of distributed computing. By extending the object paradigm into a distributed object paradigm, we unintentionally encourage developers to think in *local mode*, when really they should be thinking in *distributed mode*. His point is that additional considerations must be met (time, reliability, security, etc.) And by forcing the developer to acknowledge these concerns, runtime disasters will be decreased. His feeling is that Indigo (from Microsoft) does a good job of making the developer acknowledge the distinction between local and distributed calls, thus meeting a need in the developer community.

The Idealist
On the hand, I am the idealist. It is my opinion that we should continue to strive towards location transparency. Thus, we should continue to use one programming model (and invocation model) for both local and distributed calls. I believe that the SOA model largely facilitates location transparency and this should be leveraged. However, Chris (and others) will be quick to point out that this is like getting half-pregnant. Either your system is working efficiently in distributed mode, or it isn't. And in virtually every occasion, computer scientists will tell me that the great hurdle in location transparency deals with the static nature of message exchange sequences between client and server. In local mode, people strive towards fine-grained calls and in the remote mode, the coarse grained method is preferred. As an idealist, I am of the opinion that we shouldn't *dumb down* the programming model to reduce developer design errors. Rather, I feel that we should take the bull by the horns and look at the real issue of automating the granularity at run time (based on costing functions). However, to accomplish this, we need to give our runtime containers mode knowledge about our *intent* (think 'use case based sequence diagrams'). Now, instead of asking for a single method to be called, we ask for a 'use case' to be fulfilled. IMHO, more emphasis needs to go on writing smart software that fulfills an intent, rather than acting out a predetermined recipe.

Does Indigo excite me? Not really - I see good concepts from P2P, AOP and Trust rolled together. The exciting part is that MS has the resources to pull it off and make it easy to use.

Chris is a smart guy - he might be right. I don't know.

This post is about the hottest enterprise technology.

I bet you think I'm talking about web services. Well, I'm not. It's time for me to start blogging about RFID and more precisely EPC.

I've considered starting a new blog dedicated to RFID, but I think I'm going to keep the posts inside of this blog. Web services and RFID will likely settle into a symbiotic relationship.

About 6 years ago, I had sector level responsibility for manufacturing systems at 3M. This involved building, buying and integrating all of the usual suspects (demand management, MRP, Lab BOM, Lab content mgmt., SCE (pick, pack, ship, label, optimize, capacity planning, etc.) Recently, I've had the pleasure of working on a supply chain project with Procter & Gamble. This has been a great experience. The first thing I noticed was that not much has really changed in the last decade. Sure collaborative planning, forecasting, dynamic safety stocks, etc. are all incrementally improving. But for the most part the changes are incremental.

RFID / EPC is not incremental. It is monumental. I'm going to blog more on this later. For now, if you want to get educated , go to the following sites:

http://www.rfidjournal.com/
http://www.autoidcenter.org/
http://www.epcglobalinc.org

Chris Sells and the Royal *We*

I just saw where Chris Sells of Microsoft was being questioned about the use of the word 'we':
In the '90s, we invented component technologies, like COM and Java, to bring DLLs into memory and we were very proud of ourselves.

Some of the readers were confused, thinking that Microsoft was claiming that they invented Java. But I understood, instead of "we", he meant to say, "people other than me and my company".

But then he goes on...
Unfortunately, we were so proud that we stretched the metaphor too far with technologies like DCOM, CORBA, and Java RMI. The problem is the idea of a proxy, which was designed to serve as an in-process stand-in for the remote code, hiding the fact that each method call was a round-trip of uncertain duration and uneven reliability. Indigo, on the other hand, is a platform technology that breaks from this metaphor to use a decidedly different way of connecting applications together. Specifically, Indigo uses services, not components, to model reusable units of code.

Holy shit - MICROSOFT USES SERVICES - why didn't the CORBA people think of that??? ROTFL
That's great... unlike DCE and CORBA, we (Chris + Microsoft + Indigo) use services.

Chris, with all due respect, it would be reading a book on the history of distributed computing, then rewriting your article.

Don Box on XAML

Don recently posted on XAML.

Take a good look at the source and the build code. Now ask yourself, are you excited to go out and whip up some XAML?

I.T. Doesn't Matter - Business Process Do

A few days back I mentioned that the new Howard Smith book came out called, "I.T. Doesn't Matter - Business Processes Do". I've had enough conversations with Howard to believe that he is one of the best minds in process driven, service oriented thinking. However, this book was quite disappointing.

Don't get me wrong. I think that Nicholas Carr is an incompetent pansy with a silver spoon stuck up his ass. As far as I can tell, Nicholas has never worked in an I.T. department, been a vendor to an I.T. department, or been the user of an I.T. department. He is a professional writer that gets paid by the word, regardless of the truth in the word.

Unlike Nicholas, Howard is an innovator, a practitioner and an evangelist. However, his critical analysis of Nicholas Carr's work was shabby at best. It appears as though he cranked out 120 pages of material while his emotions had control of him, and bitter emotions at that. It was clear to me that he was racing a publication to press while the Carr episode remained hot in peoples mind.

Save your money. Here are a couple great books:
For business dudes: "Designing and Managing the Supply Chain"
For geeks: "Non Functional Requirements in Software Engineering"

Is John Udell Confused?

Is John Udell Confused? If not, I am.

A while back, a librarian wrote to me asking how she could integrate her OPAC with LibraryLookup. I investigated and found that her vendor's implementation was based on a Java applet, and there was no way to link into it. As I mentioned to Eric Rudder and Don Box at a meeting in Redmond, this librarian later posted to a mailing list that her OPAC couldn't support LibraryLookup because it was built on the "wrong kind" of software, where "wrong" meant -- though she wouldn't have called it this -- non-RESTful. For her, the richer experience of that Java applet was a poor tradeoff, since it precluded LibraryLookup's lightweight style of integration.

Is John confusing open systems with "a RESTful" approach? I might be confused... but it sounds like the librarian had an open integration problem - not something that necessarily demanded a RESTful solution. Let's see... if the year was 1993, the librarian may have had a 'CORBAful' issue... or if it was 1990, she had a 'DCE-ful' issue. Maybe what John is trying to say is that we finally have a quick & easy way to store off profile information - - kind of like the 'Win.ini' file in early versions of Windows. It smells like John is wanting to solve some problem with REST, or I could be confused.

Novell Wakes Up

After a long, long, long sleep - it appears as though the team at Novell has woken up and decided to get in the game.

First, Novell acquires Ximian which gives them Mono, a platform for running .Net applications on a Linux platform.
Now, Novell is acquiring SuSE Linux for $210 million in cash.

The way I see it, IBM will be encouraged to remain pure Java - while Microsoft will be encouraged to remain pure .Net. This leaves the 'neutral' ground in the middle wide open.

So, where does Novell go from here? I see more acquisitions. My conversations with people close to Novell lead me to believe that Novell really believes that web services are the *network operating system*. Thus, the Ximian and SuSe acquisitions were only laying the foundation for a distributed computing platform. I would look for Novell to continue down the M&A path, but this time moving up the stack - taking a hard look at web service platform vendors and perhaps even tool vendors like Borland.

But the real question deals with timing. Is it too late for Novell? Did they already lose the hearts and minds of the developer? I personally don't think that it is too late - remember, Novell was the company that forged programs like 'Gold Certified Novell Partner'. If Novell continues to make bold moves, they will attract the talent to create new developer-community offerings.

To Novell - Congratulations. I hope all that sleep you took gave you the rest needed to get into the next big battle.

Orchestrating BPEL Validation

This is a repost from the SOA news group:

>
We have been testing our implementations against some pretty large bpel documents with good results. Also, it is easy enough to enforce syntactic validation as a web service. Thus, each vendor (OpenStorm, Collaxa, Vitria, etc.) would expose a service to validate a bpel, then we would publish a simple "validation orchestration" that tested the syntax against each vendors implementation. The customer builds the bpel schedule and then calls the orchestration, which in turn calls each vendors validator. The results are aggregated and returned to the vendor. I can not commit on behalf of any other vendors, but I can commit that OpenStorm will offer this as a service.

The service world (and orchestration) may force standards to remain, well... standards.
Jeff

I would bet that my esteemed colleagues at Collaxa are game for an open validation service / orchestration as well.

Publishing SOAP Calls

WSDL doesn't completely suck. But it isn't the friendliest vehicle for giving a person access to some piece of information.

Let's get real. WSDL as it exists today was largely designed by distributed computing gurus who have added the art of aspect oriented programming to the art of IDL design, while plopping it all on top of our latest markup language, XML.

The Interface
Now, one thing that I do like about WSDL is that I can define the contract (or interface) without specifying an implementer (the binding / port). This allows me to head down the 'contract-first' design path. It also allows me to create a service contract and easily share it with other people. Interfaces with aspect oriented, or declarative resolution of non-functional requirements are pretty damn cool too. Well done.

The Service
A service is created by binding an interface to an implementation (i.e., listen for the call on port 80, with HTTP...). Thus, a service not only specifies the contract (Types, Messages, PortTypes, Operations & Fault), but one that also specifies the aforementioned deployment considerations (Service, Port, Binding).

Design Time: When you are designing your functional solution, you knock your contracts (interfaces).

Deployment Time: When you are designing your non-functional solution (scalability, availability), you knock out your services.

Publish Time: Now, an interesting question arises when I want to publish my contract and its binding for others to consume. As developers, we tend to think that of sticking a pointer to the WSDL in the UBR or shoving it into Xmethods. Nothing wrong with this, as long as you realize that WSDL isn't easy to consume and that it is very likely that someone will give it a shot and eventually give up because the WSDL didn't give enough information to actually be used for its intended purpose. For that matter, they may just look at the 25 different operations specified in the Port Type and realize that they don't even know which operation to call.

Calls
This leads me to Calls. A Call is an instance of an invocation to a Service. Put another way; it is the SOAP envelope all filled out. In many cases, this is what people really want. Consider this: what if instead of publishing my WSDL (with many operations), I merely publish a single operation with many of the parameters all filled out (default values). Now, instead of exposing a WSDL that has a PortType holding 25 different operations to my Calendar Server, I simply publish a SOAP document that performs a call:
What: CalendarAgendaRequest
Who: Jeff Schneider
How: Use the SOAP format and send it to WS-Addressing( location XYZ)

Prior to the WS-* specs, publishing a call didn't do any good. The calls were self-contained (contractually), but were not self-contained from a service perspective (binding). That has all changed. This means that for the first time, we can pre-populate SOAP calls, save them off and make them available to our end users. This is a HUGE leap forward in usability.

Now, developers will have two options: 1. Create a WSDL will all operations and combinations (for power developers) and 2. Create a SOAP message, partially pre-populated (for business users). Now, in order for this to work, it means that we have to quit writing applications that only suck in WSDL's (like InfoPath, Excel, etc.). In addition, you will have the option to pass in a URL to the SOAP envelope (AKA, SOAP Poiner).

Again, the reason for publishing a call is to make it EASY for a non-developer to gain access to some operation.