RESTifarian?

Mark Baker declared that I was on the way to enlightenment (AKA, seeing things his way).

You know, I'm not sure if I am a RESTifarian.

When I first saw this:
http://www.extremeprogramming.org/rules.html I used it as a tool to determine what components of XP I believed in. It is tough to say, "Yea, I am an XP guy or Yea, I'm a RESTifarian". I've looked at some of the REST wiki's out there and I haven't found the simple list of rules and guidelines.
What I'd like to be able to do is say, "I support RESTifarian rules/practices 3, 5, 9, 11 and 14."

Perhaps if one of the REST supporters would give me the discrete rules for REST, I could more clearly state my opinion.

-------
Side note:
The XP guys did a great job. For the record, I'm not an XP'er. I've worked too many huge projects. Here is my stance:

Planning
User stories are written. - Support
Release planning creates the schedule.- Unsure
Make frequent small releases. - Strongly Support
The Project Velocity is measured. - Unsure
The project is divided into iterations. - Strongly Support
Iteration planning starts each iteration. - Strongly Support
Move people around. - Support
A stand-up meeting starts each day. - Partially Support
Fix XP when it breaks. - Strongly Support

Design
Simplicity. - Don't Support
Choose a system metaphor. - Don't Support
Use CRC cards for design sessions. - Support for OO systems
Create spike solutions to reduce risk. - Strongly Support
No functionality is added early. - Don't Support
Refactor whenever and wherever possible. - Partially Support

Code
The customer is always available. - Unrealistic - Don't Support
Code must be written to agreed standards. - Support
Code the unit test first. - Partially Support
All production code is pair programmed. - Don't Support
Only one pair integrates code at a time. - Don't Support
Integrate often. - Fully Support
Use collective code ownership. - Partially Support
Leave optimization till last. - Don't Support
No overtime. - Don't Support

Testing
All code must have unit tests. - Partially Support
All code must pass all unit tests before it can be released. - Support
When a bug is found tests are created. - Support
Acceptance tests are run often and the score is published. - Fully Support

No, I'm not an XP'er. I think they had some great ideas - but no, I didn't just close my eyes and say all of them were good ideas and jump on board. Am I a RESTifarian? Give me the tools to evaluate and I'll let you know.

Inserting Nouns into the Service Network

For the last 6 months I've been investigating the separation of nouns and adjectives from the verbs and adverbs. For those that don't follow my blog on a regular basis, here is a quick summary:
Nouns are people, places and things - in the software world they are our domain entities (User, Invoice, etc.) - and adjectives are the properties that describe the nouns. In the J-world we used Java classes to describe them - in the W-world we use XML schemas.

Verbs are the actions. Actions have (at least) two forms: technical (insert, update, delete, publish, etc.) and domain or business specific (ship, pack, etc.).

Technical verbs often act the same way on any given noun. Why is this important? Imagine walking into an enterprise with a set of nouns on a floppy disk. The company already has a service network in place. This ServiceNet is composed of 12 servers (portal, database, security, etc.) and runs 46 services. Each server has the capability of 'listening for new nouns' and a 'registration service' has the responsibility of: 1. Notifying the servers of newly registered nouns and 2. Notifying the servers of newly registered servers.

Ok, we just moved into a world where the network is aware of the network. No magic, just simple registration. What gets interesting is when nouns have 'default implementations' of a verb on a server. And surprisingly, you will find that this is possible and beneficial. If I add a new noun to the ServiceNet will it likely have security permissions? Will it need to be stored? Will it be viewed? Should I role all of this code by hand? Again, the goal isn't to perform magic - it is to create a network programming model where we think more about how to create smart verbs and dumb nouns. The servers are slowly given the ability to interrogate each other and eventually a common set of functions is factored out of the servers and dropped into the network.

I'm convinced that service oriented programming has the potential to get absolutely unmanageable. I talk with customers every day - they almost always start the conversation by bragging about the number of services that they have. Every time I hear this I think "Oh my God, what a maintenance nightmare! And they've only begun." In Schneider-land, the goal is to go for the smallest number of services, with the highest amount of reuse. Figuring out how to do this is a bitch.

Going from Services to the ServiceNet
The first step that I recommend is to take a look at all of the operations on all of your wsdls. Categorize what you already have (VerbOnly, VerbNoun, VerbMechanism, other). Then, determine if you could have made more re-usable services. What would they look like? Create a spreadsheet with the verbs as rows and the nouns as columns. The cells (or intersection points) are the realization of the VerbNoun. Beware of verb synonyms, or made up verbs (verbs that are really nouns). What is the smallest number of verbs you require to fulfill your needs?

Custom Metadata

I was just talking with one of my CTO buddies. He was bragging that implementing his software product at a client site requires no custom code. I paused for a moment and asked, "Does it require custom metadata?" To which he answered, "Yes, and it has a nice front-end to enter it in."

My first thought was that he was just pushing the problem from one location to another. But for some reason, it does feel better pushing the problem from code to metadata, but I'm not entirely sure why. I like the idea of not compiling everything (a benefit of metadata). I also like the idea of having domain constraints on the data being entered (versus 3rd gen languages). I guess that a system that embraces MOF/MDA/metadata concepts requires less in-depth expertise. In essence, the expertise is built into the framework, enabling a 'paint-by-number' approach to recurring pattern based problem solving (resources require less training, thus less expensive).

I'd love to hear what you think the advantages to a metadata driven approach are. Blog it. Link to this. Click through once. I'll repost referers.

Is Java an Interchange Format?

So, Microsoft believes that BPEL is an interchange format. It occurred to me that they likely perceive Java as an interchange format as well. If one really, really, really wanted to... you could take C# code, convert it to Java, then convert it into VB.Net. Yes, yes - Java is an interchange format!! I don't know why that didn't occur to me earlier. And all this time I've thought of Java as a programming language.

I'm such a fool. Officially, an 'interchange format' is anything that can be used as a temporary place holder while being 'upgraded' to the Microsoft format (X/Langs, C#, etc.)

Top Ten Interchange Formats (Microsoft perspective):
1. Java
2. BPEL
3. C
4. Perl
5. Python
6. Perl
7. C++
8. JavaScript
9. Haskell
10. Ruby

;-)

BPEL4XLang, BPEL4Java BPEL4*

It has been an interesting week in the land of BPEL.

For starters, Microsoft has been in a bit of a predicament. In August of 2002, Microsoft came out saying that they were going to deprecate their proprietary process language (X/Lang) in favor of a new open process language called BPEL. Well, Microsoft missed a couple of ship dates on their BizTalk product and eventually shipped their BizTalk 2004 with X/Lang. In order to save face, they created a mechanism to import and export portions of their X/Lang scripts in BPEL format. Now, Microsoft is saying that oh... the Business Process Execution Language isn't really an execution language - get this - it's an interchange format! Brilliant.

Now, depending on who you talk to at IBM you may find that BPELJ is a good thing. Although, some of the product groups were quite surprised to see the research guys published a white paper on the topic. Interesting. Well, it is worse. In addition to WSFL, BPEL, BPELJ, and the new JSR207, it appears as though IBM is considering additional process languages related to the Rational line of products that are more closely tied to UML. Brilliant.

Well, BEA not wanting to be 'out-dumbed' by MS or IBM has decided to throw more effort at BPELJ, and whine about the lack of progress on JSR207. As one of the BEA insider told me, "BEA is a Java company, not a web services company." What this implies is that BEA is having a hard time dealing with a 'service oriented language' like BPEL in their object-oriented, java-based platform.

So, what can we expect? Here are my predictions:
1. BPEL continues to move forward and serves as the primary foundational technology for executable business process descriptions.
2. A variant of BPELJ becomes JSR207.
3. Microsoft never budges off of X/Lang as their base language
4. The OMG guys create an enhanced Activity Diagram that is 'close enough' to stubbing out BPEL.

Next question: Is this good?
Yes. It is expected that MS will do things their own way - Microsoft will create whatever technology that they need to make things easy for their customers - sure, it will largely lock them in, but most Microsoft shops already know that they're locked in - otherwise they'd be Java shops.

And it is expected that IBM/BEA will bypass the JSR process to expedite a new Java-friendly implementation that Sun will end up incorporating into J2EE. Sun will grumble, eventually adopt it and then find a way to lose money on it.

Last question: What will OpenStorm do?
[coming soon...]

MS Posts Infopath to BizTalk Examples

Scott Woodgate has posted some examples of Infopath using orchestrations. However, I busted open the orchestration file and I'm having a hard time understand this one line of code....



Hmmm... XLang/S.... BPEL export = False? What does that line do?

More on UDDI

Jeff,

I just read your blog entry on this, and it's totally in harmony with my own thoughts on the matter. Until a few weeks ago I was in charge of directing the UDDI pilot->implementation programme for [a major investment bank], and we've pulled the plug on it because the whole t-Model thing is just far too complicated to be used in any meaningful way by the constituency that needs it.

We're now planning on building a service registry around an XML metadata store, and will probably go for an XQuery interface as the primary service discovery mechanism. We may keep what I've been terming a 'naive UDDI'
interface for compatibility with existing tools etc., but the long term hope is that we can throw this back over the wall to then vendors (and standards bodies) as something will need to replace UDDI in the web services unholy trinity

--
Chris Swan


Now Stefan states, "I have since come to the conclusion that all this taxonomy and categorization stuff is actually pretty ingenious." Stefan, I agree in a computer science kind-of-way it is ingenious. However, most people just want to quickly store, index or retrieve "Service Oriented Metadata". People don't like taxonomies or t-models.

UDDI was largely designed for external use (the UBR); the use cases that is supports are so far beyond the use cases that are required for intra-company needs that is quickly becomes too complicated.

I'd love to see someone try to explain to me why UDDI isn't a complete piece of shit. Justify it - I'm listening.

The UDDI Failure

SOA is about a triangular relationship - producer, consumer and directory. It seems simple. The UDDI represents the directory leg of the relationship, and in my opinion (and I'm not alone) has been a failure. When most people see UDDI for the first time, they usually ask questions like, "couldn't I just stick this information in a database or in LDAP??"

I recently had a discussion with an architect at a leading online brokerage house. He commented that they had looked at UDDI but thought it was overly complicated while simultaneously delivering a lack of functionality. Wow. Useless and a pain in the ass - that is hard to do. Well, these guys punted on UDDI and spent a day writing a small directory using RESTish terms with easy http access. They are very happy.

I'd like to highlight some alternatives to UDDI - if anyone wants to shoot me a note telling me how you're solving the problem or other alternatives, I'd love to hear from you.

BPEL versus Vaporware

The majority of Radovan's concerns seem to be around the structure that BPEL enforces. Sometimes this is called a "structured process". These are processes where the actors are given rules and are told not to break the rules. Structured processes exist in every corporation that I've been in.

"Semi-structured processes" are those processes that have 'wiggle room'. That is, they enforce a base structure or flow, but at key points run-time decisions can be made.

"Unstructured processes" are really more about achieving goals via whatever means. Here, the participants, the activities and the order will all change at run-time in order to achieve the end goal. The down side is that this often borders on chaos and has limited repeatability - which is a key driver behind process.

What we are finding is that BPEL can handle virtually every case for 'structured' and 'semi-structured' process descriptions and execution. Unstructured processes usually don't lend themselves well to any kind of 'ordered activity machine'. Rather, these processes are more likely to be executed via 'process liberation' tools like Groove, where the focus is on communication and inter-team task visibility.

I don't fully understand the critiques of BPEL. It really is a powerful language - although I can see if you don't work in it where you might be confused. But... we have a team working on this all day, every day... as does Collaxa, FiveSight, SeeBeyond and a host of other companies.

Agree and Disagree

Read this. Negate virtually every sentence and you will have my view.

Read this. Here, I agree on virtually everything. One thing I'd add is that the DSL is a virtual language - it consists of your base language (like bpel AND all of the services (nouns and verbs) in your enterprise vocabulary. In a manufacturing company, your verbs and nouns will relate to purchasing, picking, packing, shipping, etc. Late binding, loose coupling, interoperable schemas, etc. all create a new meaning for the domain specific language. After you pick a first order DSL, you will spend the next decade creating your second order DSL (the enterprise vocabulary).

Eric Newcomer from Iona recently stated:
I can't see that the drawing approach will really work. I don't know of any graphical software development tool that has yet to address the entire lifecycle; or that generates code of sufficient quality.

I think it's just a problem that isn't meant to be solved.

Since this statement Eric and Stefan have found some common ground:
-Models can have common ui notation and be graphically driven.
-Models can drive metadata.
-The graphical notation and the resulting metadata are two completely different concepts and should be managed that way.
-XML is a great way to drive a metadata approach.

Here are some additional random thoughts:
-Models and action semantics can create traditional 3GL code, but I'm not sure you want them to.
-Metadata can be interpreted by engines and executed.
-Engines are at the heart of reusable services.
-Nouns translate nicely into metadata.

OpenStorm Orchestrator Update

Just a quick note - next week we will be making available the 2.1 version of the suite. Updates include:

Studio Update
- Improved wsdl editing
- 'Helper' for specifying duration and deadlines
- 'Helper' for graphically creating correlations
- 'Helper' for rapid message definitions on in-bound and out-bound web service calls
- Full XPath manipulation tool
- 'Helper' for boolean expression building (switch, while)
- Ability to create rapid services (drop either Java or C# code directly into an invoke node and it compiles and deploys the code as a remote web service, then replaces the code with the wsdl that front ends the code).
- New 'Web Service Invoker' tool for calling/testing services
- Canvas supports zoom-in/zoom-out, also has a new 'thumb-nail view'

Java Server Update
- Increased support for variations of the 'assign' tag
- New console with drill down (view by bpel, by instance, by correlation, etc.)
- Reliability upgrade with full state machine persistence (test = pull the power plug, plug it back in)

.Net Server Update
- Reworking web service layer to support latest Microsoft WSE upgrades
- Creating long term architecture for Indigo support

documentation
- New BPEL Programmers Manual
- Packaging WSDOX web service documentation with software

In addition, we've added a couple people to the engineering team so we should be able to move a bit quicker on the 2.2 release.

The Promise

From the forward of, "Business Engineering with Object Technology" by David Taylor:
We stand on the threshold of a new era in business engineering. For the first time in the history of computers, it is now possible to build information systems that directly reflect and expand the way we think about business processes. The critical enabler for this transformation is object technology.

Hmm... sound familiar? Well, once again, we find ourselves on the same ole quest.... to achieve a COmmon Business Oriented Language. Yes, I'm a fan of service oriented-bpm. I believe that we need a new vocabulary... for the last several years we have seen some consistent patterns in enterprise computing:
1. A gradual movement towards specialized verbs (servers/services).
2. A desire to separate the verbs from the adverbs (factor out non-functional requirements).
3. Separating nouns and adjectives from verb implementations (XML Schema vs. classes).
4. Standardize the prepositional phrase (predicate).
5. Create a sentence grammar (composition languages).

The verbs and the nouns that we choose to program our computers is central to productivity. The separation of our terms is essential. Breaking the bad habits of the object oriented verb-noun coupling will be tough. Creating our new vocabulary will be even harder. Finding a single Business Oriented Language may be impossible. Yet, we define our programming model by the means in which we separate our concepts. Moving beyond the standards gook is essential.

Oh, and just for fun - here are the verbs of COBOL: Accept, Add, Alter, Call, Cancel, Close, Compute, Delete, Disable, Display, Divide, Enable, Enter, Exit, Generate, GoTo, If, Initiate, Inspect, Merge, Move, Multiply, Open, Perform, Read, Receive, Replacing, Return, Rewrite, Search, Send, Set, Sort, Start, Stop, String, Subtract, Suppress, Terminate, Unstring, Use_After, Use_For_Debugging, Write

If only COBOL had the verbs 'publish', 'subscribe' and 'transform' ... then it we could have called it the 'COBOL-ESB' ;-)

Tech question...

I've run across a number of web service operations that look like 'foreign-key' relationships.

As an example, look at the following web service operations:
getTeams() // returns a list of all teams, including all the team id's
getPlayers(teamID) // returns a list of players, for a specific team id

Now, this seems intuitive for a human to figure out - call getTeams(), and then use the key to make a call to getPlayers(theKey) for the team you want. However, this isn't intuitive for a computer to figure out. When you are in an interactive setting (InfoPath, Excel, etc.), and you make a call to getPlayers(..), you want the software to give you a drop-down list of all of the teams. But in order for this to happen, the software (Excel or whatever), needs to know about the relationship between the two operations (getTeams and getPlayers). This information is usually capture at the DB level and is also captured at the object level (UML), but I don't see a mechanism to publish this type of relationship at the service level.

So, here is my question: How do I define and publish computer-readable operation-to-operation relationships using a standardized metadata description language (like WSDL, or other)? Email me: jschneider at momentumsoftware dot com

WS-Metadata Exchange

WS-Mex is now published at the Microsoft site.

The spec is 975 lines long, but I can summarize it for you.

One verb: [get] and three nouns: [schema, wsdl, policy]
Naturally, this took 6 type definitions, 11 messages and 3 operations.

Suddenly: ?WSDL, ?POLICY, ?SCHEMA doesn't look so bad...

They hardcoded in three types of metadata into the spec - isn't there something IRONIC about that???

Oh, the editors are:
Francisco Curbera (Editor), IBM
Jeffrey Schlimmer (Editor), Microsoft

Verb Only (when the verb acts the same on all subjects - e.g., save (x), save(y)

Verb Noun (when the verb acts differently (different logic) on each subject processPurchaseOrder(), processSalesCommission()

Qualified Verb (Bidirectionally Link(x,y)) versus link(x,y); used to qualify the intent of the verb (changes functional requirement, unlike most adverbs which only apply to NFR)

Verb Mechanism (when the logic is changed more by the mechanism than by the subject)

Verb Mechanism

Many of the web services that are created take on the form of "VerbMechanism"; that is, they specify a verb and the mechanism in which to accomplish the task, then they usually require the 'who' and the 'what' to be passed in:
sendEmail ( 'to bob', 'the status report')
sendFax( 'to bob', 'the status report')
sendInstantMessage( 'to bob', 'the status report')

Here, the mechanism is 'Email', 'Fax' and 'InstantMessage".

Another way of writing this is by using 'via':
"send the status report to bob via email"

When specifying a mechanism, we will likely have preconditions on the mechanism. "via email" requires four pieces of information (email server, email credentials, email destination, email payload); The data related preconditions can be expressed in a declarative manner on each mechanism. Currently, we push these data nuggets into the signature of the VerbMechanism. But usually there is an overlap between the verb and the mechanism (it was 'send' that required two of the pieces of information (destination, payload); the mechanism required the other two pieces (server , credentials).

A mechanism is a way to overload an operation. The verb I'm overloading is 'send' and the mechanism is 'Email". When we overload in object oriented systems, we usually just grow the signature: send(destination, payload); send(server, credentials, destination, payload); I'm not a fan of this... I prefer the declarative OCL like approach but with an 'implied signature' based on the overloading with a mechanism.

Another issue to consider is the 'invisible mechanism' or 'implicit mechanism'. Consider:
savePurchaseOrder(...);
Here, the designer intentionally hides the implementation mechanism from the user of the operation.
Save Purchase Order via Relational Database
Save Purchase Order via Flat File

Now, I agree that it is a good practice to not burden the user of a service with mechanism. However, I'm not sure that it is always a good idea to hide it from them (burden them versus inform them).
Again we see that each mechanism will require a set of preconditions:
"via Relational Database" needs (a JDBC Driver, a connection string, credentials, etc.)

I'll save "Reliably Save the Purchase Order via Relational Database" for another morning.

The Service Network

Today, there are largely three camps when it comes to web service architectures: The SOA guys, The Bus guys and The Protocol Network guys.

The SOA guys
The SOA guys were the original 'web service' people. They viewed the new paradigm as a ubiquitous service based system that would utilize the architectural pattern known as SOA (Producer, Consumer and Directory). The SOA guys are the people that build UDDI servers and SOAP platforms. They have taken the best of the CORBA world, learned some lessons and reapplied it to a new set of protocols that are accepted by MS and IBM. The SOA guys rarely mention a 'network topology', but their implicit topology is point-to-point (my consumer directly calls your producer).

The Bus guys
The Bus guys are the people who believe in messaging. Ultimately, messages and services go hand in hand. Some people put more emphasis on the service (SOA guys), while other put more emphasis on the message (Bus guys). The Bus guys also love asynchronous communication - thus, they loves queues or any other store and forward mechanism. Unfortunately, most of the Bus guys come from the JMS world and largely their stuff doesn't interoperate (think ESB). The Bus people usually think that the web services network topology is hub and spoke.

The Protocol Network guys
The Protocol Network guys are the people who treat web services more like networking protocols. These guys put emphasis on two things: wire protocols and policies. They see everything as a policy on a protocol. These guys love specs like ws-addressing, ws-discovery and ws-policy. The Service Network guys see the network topology as being adaptive based on the state of the network. They rely on routers, load balancers and other devices that have knowledge about the running services to make informed decisions on the fly.

The reality of is that most large companies will need a hybrid of all three. They will embrace a standard SOA triangular pattern, but letting a Protocol Network make routing decisions at run-time, with the messages often ending up in queue. I continue to consult to my clients about the convergence of the three paradigms. This convergence is what I call the Service Network.

The Service Network is a message-based, service-based and protocol-based computing model.
- It leverages the SOA model to decouple producers and consumers and to provide lookup capabilities for self-describing services.
- It leverages the Bus model to provide asynchronous communications for long-running processes.
- It leverages the Protocol Network to provide runtime decision making about locating and executing a service on the network based on the service network conditions.

Another way of looking at these models is:
- SOA decouples software units (consumer and producer)
- Bus decouples software in time (synchronous = time-coupling)
- Protocol Network decouples software from hardware (run a service on some machine)

The goal of the service network is to provide all three forms of decoupling.

WhiteHorse and Virtualization (take 2)

Alex Torone the Lead Program Manager for the Microsoft Visual Studio Enterprise Tools Team gave me a gentle kick in the balls regarding my inaccurate posting around WhiteHorse. Here are his clarifications:

What the diagram really represents:
The LSAD (Logical Systems Architecture Diagram) represents "logical run time hosting environments" (hence the name). Each box represents a "logical server type". Specifically, the large blue boxes represent a configuration of IIS, whereas the endpoints on the large blue boxes represents web sites. We model the entire IIS meta base (in this example). So a user could either supply "desired configuration" in the tool, or they could simply point to a "canonical server" that has the "desired configuration" and harvest those settings. Once the settings have been defined in the LSAD model, the user can then define constraints against the application environment. For example: Suppose my datacenter policy for "front end web servers" require web apps to use forms authentication and impersonation. These constraints will be validated against the application designer (we model all of system.web for example) and can be expressed in this logical design. There are also two additional layers in the SDM model (part of DSI refer to links below) which represents the network layer, and device layer that are more in line with your comments and are slated for a much later tools and platform release.

About the Physical DataCenter:
Data Centers host many types of applications. Network infrastructure diagrams (we've all seen them) have physical machines, IP address, Vlans, switches, routers, etc. The LSAD is meant to represent abstractions over the physical data center. We want to represent types of server not physical servers. One box on the LSAD does not necessarily equate to a physical server in the data center. In fact, you can create multiple Logical web servers and place then on one physical server with SQL as an example. When we get to actual deployment releases post Whidbey (please refer to the Dynamic Systems Imitative (DSI) links below), we will then provide a logical to physical mapping. This action will populate all of the deployment parameters with they physical URL's of the web server etc.

In conclusion, the LSAD is about conveying that information which is important to the developer (such as what kinds of services are available to me, what communications pathways are open, what configuration must I adhere to, what are the boundary conditions that I must be aware of, etc) such that we will increase the probability that their design will actually work when it is physically deployed.

The more I dig into the DSI, the more impressed I am. If they can pull it off with design-time integration it will be one heck of a story. Here are some links Alex provided me to reduce my ignorance:
http://www.microsoft.com/windowsserversystem/dsi/default.mspx
http://msdn.microsoft.com/vstudio/productinfo/enterprise/enterpriseroadmap/default.aspx?pull=/library/en-us/dnvsent/html/vsent_soadover.asp
http://msdn.microsoft.com/vstudio/productinfo/enterprise/enterpriseroadmap/whitehorsefaq.aspx
http://microsoft.sitestream.com/PDC2003/TLS/TLS345_files/Default.htm
http://msdn.microsoft.com/msdntv/

Book Recommendations

It is tough to find good books on subjects like web services. The people that write on the subjects are usually writing about the current state (which is outdated by the time you buy the book) or a future state (which is usually wrong). Generally speaking, I don't buy books on web services - however, I do buy books on areas of convergence. That said, here are my recommendations:

1. Policy Based Network Management - A book by John Strassner, a Fellow at Cisco and thought leader in DEN, writes on the topic of the declarative network. The book covers basic policy models and then explores the DEN-ng policy model as an example. The book will never mention SOAP, web services or anything at the application level. It is up to the reader to draw analogies between Network Policies and Application Policies. For those that are familiar with the WS-Policy specs, you will find significant overlap between WS-Policy and PBNM (Policy Based Network Management). This book should help you understand the importance of declarative policies and their uses in application-level service networks.

2. Grid Computing - A book by Joshy Joseph and Craig Fellenstein, this book is part of the IBM 'On Demand Series'. The book covers the basics of grids, the merging of grids and web services, OGSA, OGSI and the programming model for the Globus GT3 Toolkit. This is a good book for anyone who needs a crash course in grid services. It is a high-level read - not a reference manual.

Why the Outsourcing Flap Makes Cents

Rich Miller, who runs my favorite blog, made a posting near to my heart titled, "Why the Outsourcing Flap Makes No Sense" - where, Rich asks what the flap is all about.

The flap is about...well, pissed off people that lost their jobs.
I spoke with two different people today who are old friends. They both worked (past tense) at Sabre. The first was a VP of engineering who was laid off and his job was moved to Poland. This guy was a real "do-er" - the kind of guy that made things happen - he'd cut through the BS and get things done. Does he cost more than someone in Poland? Yes. Sucks to be him. He's now remodeling his kitchen and sending resumes out on Monster.com. He gets it - lower wages for Sabre lead to a more competitive product - higher share prices for share holders and lower prices for consumers. He completely gets it - but he's still pissed off.

The second person I talked with (a project manager) told me that she was told that she had to let her whole team go because they were to be replaced with people in India. She told her manager that she didn't like the idea but went along. Eventually she was told that her job was moving offshore as well but they gave her the option of being manager of 'offshore procurement'. She did it. Then when the procurement was done she was let go. Now she is reading books on how to day-trade. And yes, she is pissed off.

The flap is about people that are great at their job who get let go based purely on cost. They weren't given the opportunity to accept a lower salary - after all, their employer has a mandate to move 25% of product engineering jobs offshore. The flap is about humans who felt disgraced by long-time employers.

Maybe the U.S. installed a minimum wage system too early. Maybe we drove up our own cost of living and that drove up our salaries. It is our own fault. I know - yelling about free markets won't change anything. But if my friends at Sabre and all the other companies that have cut U.S. workers want to bitch - I understand. Flap all you want - because that is all you are going to get.

Objects, Services and Verbs

A few years ago, I asked my mentor what the difference was between objects and services. He told me a handful of things, but the one that I really remember was, "objects use a 'noun.verb' notation and services use a 'verb.noun' notation." I asked him why this was a big deal and he told me that in most systems there are less verbs than nouns. His point was that you would end up with less first class citizens in a service oriented world than you would in an object oriented world. Very interesting. He later pointed out that the service citizen would likely treat data as meta-data, making it much more manageable than an object that tries to use polymorphic behavior to apply verbs functionality on nouns.

I've become fascinated with service / operation naming. As an example, the WSDL spec uses, "GetLastTradePrice". Immediately I break it down, "Get" "Price" ... what kind of price? "Trade"... which one? "Last"... or perhaps it is: "Trade.Price"?... Hmmm... How about :
Trade.Price.get().last();

Object, attribute, getter, ordered set operation - an interesting way to break it down. Now, why did the service people run the whole thing together (GetLastTradePrice)? Good question. Don't I end up with a ton of operations if I run them all together? What if... I didn't run combinations together, but instead identified the command components: (getter/setter) x (an objects enumerated attributes) x (potential set operations)? Should an operation name be one big concatenated string where all the potential combinations are combined at design time? SQL sure didn't do it this way - - they went command language for base manipulations - and then went stored procs with fixed names for one-offs (oh, and declarative rules *triggers* for eventing - here the name didn't matter).

Anyway... all I really wanted to do was share some of my favorite verbs:
insert, add, update, set, delete, remove, erase, get, select, fetch, subscribe, publish, receive, listen, send, notify, call, invoke, create, destroy, deallocate, dispose, show, view, hide, close, open, drop, restore, resume, suspend, pause, clear, filter, cache, run, start, execute, stop, allocate, new, advance, go, post, do, find, locate, evaluate, jump, visit, goto, exit, break, spawn, join, split, lock, unlock, process, print, transfer, throw, push and pop.

It's a pretty good list of verbs. There are some people that REST'd after only finding two or three verbs. But they just like wrapping verbs inside of other verbs :-) That's ok - I don't think they hate verbs. They just like a couple verbs a whole bunch!

I like the idea of having an enumerated set of verbs. This isn't the list, though - too much redundancy. Also, some verbs are really just combinations of other verbs. Hmmm... first order verbs. Second order verbs. How many first order verbs does a system need to have a semantic foundation for 'doing' things?

And no, I'm not a verb bigot. I love adverbs too! Nouns suck - although I admire those that have the patience to play in the noun space. Adjectives are cool - only in that they are simple ...

It is my belief that we are *slowly* moving towards a semantic and service oriented world. Operation names that are concatenated strings of verbs, nouns, adjectives and adverbs worry me. There is a better way.

OpenStorm Blog

Ryan, Dave and the crew have kicked off the OpenStorm blog. It will focus more on using the Service Orchestrator and general questions related to BPEL and service composition.

See:
http://www.openstorm.org

I don't see an RSS feed, but I'll ask them to put one up.

AT&T and GrandCentral Partner

See:
http://www.internetnews.com/ent-news/article.php/3324191

"AT&T WebService Connect allows different applications from different sources to communicate without time-consuming custom coding. And because it is XML-based (define), it's not tied to any one operating system or programming language.

Developed over the last year with partner Grand Central Communications, WebService Connect plays into AT&T broader strategy of evolving from a long-distance phone company to a provider of enterprise network services. "

...

"The service will be rolled out gradually in the coming months. It starts at about $34,000 per month, although prices could run higher depending on usage. In terms of its telecom competitors, AT&T believes it is farthest along in offering Web services (define) to its customers. "

Congratulations to GrandCentral!

Jim Waldo clarifies position

Jim does a great job clarifying his position on standards:

http://www.artima.com/forums/flat.jsp?forum=106&thread=4892

Jim states:
Point one: Just because something is called a standard doesn't make it open; and something that isn't a standard is not, because of that, proprietary.
Point two: A standards body is often a lousy place in which to invent a technology.
Point three: The previous posting was not a veiled (thinly or otherwise) attack on any particular standards group or collection of standards groups.
Point four: If there are multiple groups competing to write a standard for the same thing, it is probably a safe bet that the technology being standardized isn't ready for standardization.

Well done. I think Jim fully understands it. Sucks doesn't it? Oh well.

Now, I'd like to see Jim (the brain behind Jini) take some of his vast knowledge and write a couple new standards... for starters, I think he'd have quite a bit to add to ws-discovery: (ws-leasing, etc.)

So, here is the WS-* formula:
1. Find a concern (think separation of concerns, they usually end in "ility")
2. Find a remedy to the concern.
3. Take the name of the concern and put the letters "WS-" in front of it.
4. Use as much protocol (with XML) to describe the remedy, use wsdl and the other ws-specs to weave a full story.
5. Publish your spec.
6. Wait for either MS or IBM to "expand" the idea, change the name and republish it with a higher degree of separation of concerns and a name that has a striking resemblance to the name you gave it.
7. Bicker to the press about it.
8. Wait approximately 6 months. Feel free to knock out your reference implementation during this period.
9. Watch the MS-IBM version become popular.
10. Terminate your version and publicly support the MS-IBM version. Be happy that a spec exists.

It really is a very simple, straightforward process. Best of luck.

Supply Chain Orchestration: RFID meets BPEL

Last week I made a visit to Boston to meet the people at Connecterra. These guys are in the "RFID middleware" space. This category is often called a 'savant' - although the industry seems to be moving beyond this term.

It was absolutely fascinating to see RFID signals get picked up by the readers, be sent to specialized RFID middleware where the signals were aggregated, filtered and eventually turned into web service (soap) calls. These calls could then could be consumed by a BPEL engine for processing. Immediately, the opportunities for "supply chain orchestration" are illuminated.

I've been working on a white paper called, "Supply Chain Orchestration" with Bob Betts for the last couple of weeks. The topic is huge - the impact is significant. We should have the paper done in a couple of weeks - I'll post a note when it is available.

Adobe launches beta of XML/PDF Form Design Software

See:
Adobe Launches Public Beta of New XML/PDF Form Design Software




"Developers can easily integrate form data with core enterprise systems via XML, OLEDB and web services. Additionally, Adobe Designer allows users to design forms that can be used with digital signature technologies for facilitating secure electronic transactions."

Very exciting!

WS-Discovery and Jini

News.com and Ron Schmelzer have once again teamed up to inform the public on web service specifications:



Ws-disc and Jini have a discovery component and Jini is a 'java' only thing. A couple other things worth noting:
1. Sun made a huge mistake by not bundling Jini with the J2EE stack early - this killed Jini - it was considered a 'device only' api.
2. Jini is API not a protocol - they later rewrote this functionality as a protocol for Jxta
3. Jini bundled remedies to concerns (leasing, discovery, proxy, service matching and tuple space) in a single spec.

My belief is that ws-discovery will catch on as long as people don't say, "The idea is very much the same as Jini". As cool as Jini was, it was doomed by Sun. If you want to describe ws-discovery, say "it's a multicast framework for dynamically finding resources on a net (wan, lan, scatternet, piconet, etc.) without knowing the address of any resources ahead of time."

WS-Discovery isn't bundling a ton of things together - it is a lightweight protocol for finding stuff across a variety of networks. It isn't based on a single language and has good support. It can make it - as long as we don't accidentally kill it in the cradle.

The Web Service Dial Tone

Web Services are failing and I know why.

Most people I talk to think the lack of adoption is for one of two reasons:
1. The huge WS-* stack is too complicated and it reminds them of CORBA.
2. We haven't found a killer app for ws.

These are both interesting - but in my opinion, they are not the real reason why ws are failing. I firmly believe the answer is very simple; we haven't created a web services dial tone. When I plug my laptop into a network, it immediately sends a broadcast message to the network. Certain devices like routers, firewalls, gateways, etc. respond to the inquiry and pass on some information about their capability and service offerings. This transparent conversation creates a 'network dialtone' - it enables a device to plug into a network, discover the services and converse with them. This capability is at the heart of our TCP networks, but has not been realized in our 'web service networks'.

In order to create a ws-dialtone we need only a handful of capabilities.
1. Consumer-side applications need the ability to send a broadcast message to a network (UDP for web services).
2. Producer-side applications need to be able to listen to the broadcast and respond. They also need the capability to broadcast their availability.
3. UDP style broadcasts are limited to a sub-net. Thus, sub-net routing (via a soap router) is critical.

Much of this functionality is available via the WS-Discovery specification. However, a large number of people in the ws community view the aforementioned protocol as a tool to be used strictly in ad-hoc networks or for consumer hardware devices. And although this is a subset of the target audience, it has broader applicability in the enterprise environment.

As we continue our movement towards contract based development we will begin to see more emphasis on standardizing the contract. Today, our standardized contracts come in the form a platform like J2EE (technical contracts) or from industry working groups (business contracts). The combination of the standardized contract, the discoverable implementation and late binding will introduce a computing model that enables a service oriented system to automatically find producers for a predetermined piece of functionality and to bind to it. Imagine installing a workflow system and immediately after installation, the server found all of the other dependent services (LDAP, single sign-on, etc.) and registered the bindings to those services. This is the vision of a service oriented enterprise - it focuses on the simplified integration of systems by using ubiquitous protocols, standardized contracts and late binding on a dial tone network.

Whitehorse & Virtualization

Microsoft has been touting a next generation designer for creating services and then facilitating the deployment of the services. As Microsoft puts it:
"When creating mission-critical software, application architects often find themselves communicating with their counterparts who manage data center operations. In the process of delivering a final solution, the application's logical design is often found to be at odds with the actual capabilities of the deployment environment. Typically, this communication breakdown results in lost productivity as architects and operations managers reconcile an application's capabilities with a data center's realities. In Visual Studio Whidbey, Microsoft will mitigate these differences by offering a logical infrastructure designer (Figure 19) that will enable operations managers to specify their logical infrastructure and architects to verify that their application will work within the specified deployment constraints."

The environment allows you to drag a service description to a physical node and drop it on the node to signify deployment.


At first this seemed like a great idea, but after spending some time with the IBM grid team they quickly reminded me that "services belong to the network, not a predefined physical node". Thus, hardcoding service locations to a physical node kills the benefit of virtualization. Well - I agree - you want to drag your service to a "service network" and there should be various sub-nets that are partitioned based on resources and capability.

But, I must admit... the Whitehorse demo sure looks cool... It is very "Microsoft".

Mach3 Turbo ESB

This is a must read:
http://www.theonion.com/opinion.php?i=1&o=1

Come on... shouldn't someone do the Turbo ESB???

You're ESB only does queues, routing and transformation! Mine does all that plus [fill in blank]. Think I'm joking... just wait.

WebMethods to Close Office, Lay Off Workers

I almost missed this:

"WebMethods lost $11.1 million in the quarter ended Dec. 31. Since it went public in 2000, WebMethods has only reported a profit in one quarter, $157,000 for the three months ended March 31, 2003.

"Their inability to show profitability has been a very sore spot for investors," said David Hilal, an analyst at Friedman, Billings, Ramsey & Co. "They are doing what they can to appease investors."
See:
http://www.washingtonpost.com/wp-dyn/articles/A20530-2004Feb6.html

When is a service not an orchestration?

When is a service not an orchestration? This is the question that Christoph asks?

His point is that all bpel orchestrations 'are services' and if one wanted you could use the orchestration as an agility mechanism. For example, you know that you need a service 'foo' - why not make an orchestration 'foo' where the 'foo' orchestrations calls the 'foo' service. In essence, you treat the orchestration like a proxy to the end service. Christoph goes on to point out that like any proxy-calls, you incur extra overhead. By proxying, you don't tie the 'foo' service directly to the non functional requirements (logging, security checks, business activity monitoring, etc. ) you could add additional steps into the orchestration to perform these for you.

Overall, I'd have to agree with where Christoph landed - if the orchestration is time sensitive or doesn't look like it will need any 'decorators' - then just make it a service. If it is part of a long running business process and isn't necessarily time sensitive than consider front-ending the service with an orchestration and add the decorators.

In the long run this may kind of 'orchestration-proxy for NFR's' might go away. I'm still a big believer (as is Christoph) in using declarative aspects on services. In this manner, you would declare the decorators on the service and when the service was called and it would trigger the decorator calls (think Aspect Oriented Services). Either way... I'm not sure which one is easier to read / debug. Although it does raise a good point - we probably need a metatag to state if a service call is part of a NFR resolution or if it is a legitimate Functional step in the orchestration...

OpenStorm Launches

Today we are officially launching OpenStorm Software. See: http://www.openstorm.com We unofficially launched the company in August of 2002 - but went back into a semi-stealth mode to significantly enhance the product.

Our primary goal was simple: create a product that had the simplicity of a Microsoft tool and the scalability and reliability of an IBM tool. We think we did it. First, we took our original .Net orchestration engine and rewrote it in Java for the J2EE platform. The design leverages a message oriented architecture and facilitates logical and physical tiering between the 'communication server' (http, etc.), the message broker (JMS, MSMQ) and the orchestration server (native BPEL, exposed as a service).

Then, we rewrote the entire user interface for the developer tooling. We chose the .Net platform for the UI for a couple of reasons: 1. We believe that we can build better interfaces in a shorter period of time than on just about any other platform. 2. We intend on facilitating "Office Orchestration" - enough said.

The only issue we are having from a timing perspective is that the .Net Web Service Enhancements (WSE) remain in beta. Thus, we will be waiting on some enhancements before we make the .Net BPEL server available. We also decided that we wouldn't make the first release downloadable. Instead, we are inviting partners and qualified prospects to install the system and work with an assigned OpenStorm team to ensure success.

I think the product turned out great. It will serve as the foundation for service oriented integration (A2A and B2B). It will also serve as the underlying engine for the next generation of BPM (Service Oriented, Process Driven Architectures).

Lastly, I want to congratulate the team - you kicked ass. They are now busy working on the next release. Stay tuned.

BEA on Web Services

I largely disagree with the BEA position on web services:



Now, of course, an intelligent app server can detect if the two services are local and bypass the marshalling. I think what BEA is saying is "our web services implementation isn't smart yet..." - and that's Ok.

If you take BEA's advice - you'll likely end up with a tightly coupled, Java based application. Thus, you only integrated at the edges. I'm not a fan of this approach. Don't get me wrong - not everything should be a service - but saying, "you should place business logic in custom Java controls instead of web services" is definitely not my way of thinking! In my humble opinion, this is one of the faults with the WebLogic Integrator. It leads to 'inlining java code directly into the process code' - - which quickly turns into a mess.

The Case for Orchestration Tooling

As the Stencil Group points out, moving to a modular system has great benefits:




Now, if you were to look at the back side of the 'modular system' you would notice that there are wires running in every direction, hooking all of the modules (or services) together. If you've ever had to wire together the components in your stereo system, you know what kind of nightmare this can be.



The neat thing about monolithic systems is that they don't expose the ugly wires to the outside world. Of course, there is a huge downside in this approach as well - you're stuck with the whole elephant. The goal of orchestration is to get the best of both worlds: the flexibility of a modular system with the simplicity of a monolithic system. In most enterprises, you won't be dealing with a small handful of components, it will be hundreds, if not thousands of services that need to come together. Imagine trying to wire a stereo system with a thousand components...

Service Types?

I'm looking for some common vocabulary to describe the various nomenclatures found in service operations. Here is what I've been using:

Pass-All Service – This is a service that passes the verb and noun into a generic service entry point. An example would be: “do( Add, Employee, EmployeeInfo)” , or do("insert into Employee ...")

Verb-Only Service – This is a service that states the kind of action that will be requested in the operation name, but passes in the nouns. An example is “Add(Employee, EmployeeInfo)”

Verb-Noun Service – This is a type of service that states both the action and the nouns (the target of the actions) in the operation name. An example would be AddEmployee( EmployeeInfo ).

Is there already terminology for these different interface styles? If so, please send me a note.

BPEL Data Mapping

One of the early lessons that we learned was that data mapping between xml documents was a tough thing to do manually. BPEL leverages XPath extensively - thus to do it right, you need an integrated XML data mapping and XPath manipulation tool to facilitate the mapping between service calls. See:



In the OpenStorm product this tool gen's the code for the 'Assign' tag based on drag & drop interfaces. We are finding that it significantly reduces the potential for syntax errors. It has also reduced the time to create complex orchestrations by about 40%.

Web Service Documentation

It is great to see that products are coming out to help document web services. This is another step in the maturity process. Now... we need people to learn how to write good documentation for their wsdl's and bpel's.

Take a look at this:
http://www.wsdox.com and the samples: http://www.wsdox.com/samples/index.html

I'm very excited to see the BPEL documentation as well. The team as Wsdox are also planning on authoring guidelines about writing good documentation for web services. In addition to tooling, we need best practices.

In my inquiry, they also mentioned some interesting notes about their roadmap including: documenting the WS-* aspects of services, making the tool a web service :-), documenting some of the other artifacts (XSL, XQuery, etc.) -- very good stuff. I'm also glad to see that they are OEM'ing the software - so many products should be using this kind of stuff...

The Twenty-Twelve Stack

Patni issue oversubscribed 22 times

Offshore software development continues to be hot for the investment community:

The initial public offer (IPO) of Patni Computer Systems has been oversubscribed 22 times. The company is raising $95 million, which is the largest thus far by a software company in the last four years. In 1999, HCL Technologies had come out with $182 million issue.

Amit Chandra, member of the board, DSP Merrill Lynch, said, “The overwhelming response to the Patni book-built issue during the extremely choppy secondary market environment shows the strong investor appetite for high quality Indian paper.”
...
GE has a 10 per cent stake in Patni and relies on the US company for about 45 percent of its revenues.



This is the company that my brother in GE Atlanta lost his job to. Way to go GE! Cut costs by replacing American workers with Indians, invest in Indian companies and help them go public! GE - you're one class act. Maybe they could take their proceeds and try to get that pesky law about freed slaves overturned.

From: http://www.business-standard.com/today/story.asp?Menu=1&story=33841

More on Pass-by-Reference

Stefan Tilkov made an interesting point, " I believe anytime you make a copy of some data - which is unavoidable if caller and callee don't reside in the same address space - you can't do pass by reference."

Here are some things that I've been thinking about:

----
Pass by Value implies the creation of multiple copies of the data and any changes made to one copy do not propagate to the other copies.

Pass by Reference implies a single 'master' instance of the data; thus all changes made to one copy apply to all references.
----
Pass by Value implies that the data 'value' is created at the time determined by the sender.

Pass by Reference implies the data 'value' is populated at the time determined by the receiver.
----
Pass by Value implies that the copies of the data were a snapshot in time. Thus the data may become stale between accesses.

Pass by Reference implies that the data is 'current' - however, it may not reflect the state of the data as intended by the sender.

Both Pass by Reference and Pass by Value can use a 'leasing' mechanism to deal with stale information or stale resources.
----
Pass by Value implies that the contents of the message is data. (Out of scope: mobile agents and Jini style proxies)

Pass by Reference implies that the contents of the message is either a pointer to data, a pointer to functionality or both.
----
Pass by Value implies that the contents of the message are fully contained within the message, thus you pass *everything*

Pass by Reference implies that the contents of the message are not contained within the message and the recipient can choose at runtime which piece of the data they are interested in.
----
Pass by Value can be implemented in a single call.

Pass by Reference requires at least 2 calls; the first one sends the pointer, the second call retrieves the data (by value)
----
Pass by Value implies that the message contains the exact information desired by the sender (maintains data integrity)

Pass by Reference does not guarantee that the data being passed is that which was desired by the sender. This is due to the potential modifications of the data between 'send time' and 'retrieve time'. This problem can be overcome by using some sort of locking or checksum mechanism.
----

- I like the idea of not passing around 'fat messages' (extra information that the ultimate receive may not be interested in)
- In some cases, I like the idea of passing pointers to data.
- In some cases, I like the idea of passing XQuery expressions that point at virtual endpoints where the predicate is populated by the ultimate receiver and a lease verifies that the data/resource isn't stale. Here, I'm not sending the data - I'm sending a partial evaluation expression for the data.

However, you'll notice that this style of programming is starting to sound like what we did in the client/server days of stuffing all of our messages in our relational database and then passing around SQL statements that referenced the data. Issues here too.

Stefan is right in that remotely referenced information eventually becomes a *value*. But the Pass by Reference model has additional attributes that should be considered at design time. It is my opinion that PbV and PbR will both exist in the web services world and should be part of the toolbelt of the service designer. The aforementioned differences should be weighed when deciding between them.

I'm worried

I lived the dot-com life. My consulting customer base was every bad idea from Garden.com to BubbaJunk.com. I saw venture investors who were normally smart people throw their money away. I saw hard working employees work nights and weekends, dreaming for the next big IPO.

Well, in Austin it is now apparent that things are beginning to turn around. The first 'major' sign happened last night - the Austin Business Happy Hour returned. And unlike the last 3.5 years - the event was packed. People were optimistic - startups were emerging. Pretty much everyone had jobs and those that didn't have jobs had left on their own. The event was reminiscent of the dot-com days (minus the recruiters).

So, why am I worried? I'm worried for two reasons:
1. The startups that I'm seeing emerge look just as stupid as the ones I saw emerge in 1998.
2. The startups aren't going to get funded by VC's. Entrepreneurs may be quick to forget, but the VC's aren't.

I don't mean to be a stick in the mud, but business is still tough. Sure, things are picking up - everyone is doing better, but it is no cake-walk. Sometimes I get the feeling that the startups that are forming right now are based purely on the fact that the smart people that went into hiding (took a job with a big company), just can't stand working in the cube from 9 to 5 anymore and feel the need for speed.

Well, I hope I'm wrong. I enjoyed partying like it was 1999...

Pass by Reference : Answer #2

As previously posted, I've been looking at options for passing by reference. It has been interesting to see how people interpret this question. The first divide occurs in regard to 'what' is being referenced in the pass: pointers to data or pointers to services (or pointers to service that front-end data).

One thing that everyone seems to be agreeing on is that the 'pointer' is the URI. Last night I had a brief discussion with Dave Langworthy of Microsoft. Dave confirmed a couple of things. 1. There is no WS-EndpointReference specification, the portions of that spec that were relevant were rolled into the WS-Addressing specification. 2. I can accomplish what I want by using the endpoint reference functionality.

This threw me for a loop, mostly due to my lack of knowledge on this spec. Dave went on to explain that there are two primary pieces of the WS-Addressing specification:
1. The Message Header - This gives the from/to info for the header of the SOAP envelope
2. Endpoint References - This points to resources; these resources could be data, processors or other. Also, the Endpoint Reference is a 'type' which can be used in the body of a soap message. This means that you would define your WSDL 'message part' using an Endpoint Reference as the 'type' of a 'part'.

Within the 'EndpointReference' structure, there are a couple items of interest:
1. 'Address' - This is the URI of the data that you want to reference (or the actual endpoint referencing a service that gives you access to the data)
2. 'Reference Properties' - This is the bucket where you add qualifiers to zero-in on the information you are looking for. In my case, it will likely be an XPath expression that points to a specific element in an XML schema.

Summary - you can use an endpoint reference as standard type to act as a pointer to a resource. This standardized type can be leveraged inside of your own messages.

Sidenotes:
- The Address could specify a 'virtual' address that performs further resolution
- The Address could point to the URI of 'localhost' enabling pass-by-reference in local mode
- I'm not sure if there is a standardized way to drop in an XPath expression in a ReferenceProperty (anything special?)
- If the Address is referencing an XML document don't expect locking / synchronization, etc.

Web Services and Grid

Here's a nice presentation on web services and grid technologies (WSRF):

http://www.globus.org/wsrf/sabbah_wsrf.ppt

'Pass by Reference' in web services

I've been looking into my options for providing 'pass by reference' capability in web services. I know, I know - PBR is the root of all evil in loose coupling. But it is also the performance enabler for many applications. Thus, the goal is to identify the best mechanism that enables loose coupling while still providing performance.

My initial quest took me down a road that I'm calling "Pass By RESTference". I ran the concept by the smart guys on the Yahoo SOA group and it didn't get nuked (which is good in that group). The next issue is, how do you do it? Well, the closest example I've seen is probably this:
http://wiki.astrogrid.org/bin/view/Astrogrid/RestStyleWebServices

I'm going to chase down a couple other items... it has been rumored that WS-EndpointReference has something to say about this problem. In addition, I'm hearing that the OGSA guys have an alternative answer. More to report later.

Analysis of Orchestration Space

I know it isn't traditional to post your competitive analysis - but I'm not very traditional. Here is a copy of my internal analysis of the orchestration space (from an OpenStorm perspective).

=============================================
Don,
You were asking about competition. Here's a breakdown:

High-end EAI: (SeeBeyond, Tibco, Vitria, WebMethods.)
Big 3 (IBM, BEA, MS)
BPEL Pure Plays (Collaxa, FiveSight, Vergil, Choreology)
Web Service Networks with BPEL (GrandCentral)
Workflow (Dralasoft, Reactor)
Pure play BPM (Lombardi, Fuego, Savvion, Intalio)
JMS and ESB providers (SpiritSoft, Sonic, Fiorrano)
Web Service platform providers (Systinet, CapeClear)

The fact is... orchestration is a sweet spot and people are looking at it from many different angles. Here's my take on how it might play out:

High-end EAI - These guys are having a hard time competing on cost and putting together a message that doesn't bastardize their current high-end revenue stream. They will likely take a 'wait and see' attitude towards orchestration, putting out an offering, but not really marketing it. It is likely that they will spread some FUD about orchestration, saying it is slow or lacks functionality and that their proprietary offerings are more 'seasoned'. Ultimately they will have to play in the game or will become outdated.

The Big 3 (IBM, MS, BEA) have a different story to tell. Their story is "Orchestration is great, but it is only a small piece of the puzzle". They won't try to have the best orchestration offering, just one that integrates into their stack the best. IBM shops will want IBM, MS shops will want MS... These guys will get a significant piece of the pie; they always do - people buy here for peace of mind. BEA may have a bigger challenge; their offering lacks substance and BEA doesn't have the clout that MS or IBM have.

BPEL pure plays - perhaps a more interesting category. Collaxa came out with an early lead but then went on to put out 14 or so betas and still haven't knocked it out of the park. Vergil, Choreology and FiveSight are all still a bit early - mostly in stealth. Thus too early to tell. Expect to see these guys either partner, strategic partner or drop into a niche market.

Web Service Networks - A couple of months ago Grand Central announced that their web services network would support BPEL. Although I'm not sure, it appears as though they rolled their own implementation of BPEL - which would be odd for an infrastructure provider. The implementation had a significant number of non-standardized extensions which I believe they needed to more fully support their existing network. Maintaining a BPEL engine is a bitch - in the long run, I would expect GC to OEM an engine from somewhere else - unless they are getting out of the ASP market all together.

Workflow - Workflow and Web Service Orchestration may seem like a similar problem, but in reality they are very, very different. Some of the vendors that claim features similar (but different) will likely confuse uneducated buyers -but the customers that we want will know the difference.

Pure Play BPM - These guys have had every opportunity to break up their monolithic architectures into a component-ized (or service oriented) architecture. As far as I know, none of them have made significant progress down this path. Intalio seems to get it but doesn't seem to be delivering the message. If these guys start delivering a "SO-BPM" story (Service Oriented BPM), watch out - they could be real contenders (of course, they will have to have product to back it up).

JMS / ESB - Here there are two categories 1. Sonic, 2. Everyone else - The fact is, Sonic is a marketing machine. I tip my hat to them. These guys not only could sell ice to Eskimos - they do! In the short term, I think they will be the single biggest threat. However, they have some baggage they bring with them (JMS is commoditized, ESB will eventually be a joke) All of the other ESB vendors will have to compete based on product features/functionality/price (think bake-offs against Sonic).

Web Service Platforms - Oddly, there are really only 2 platform providers left (Cape Clear and Systinet). Both of the players will need to get an orchestration offering. Their issue will be to knock orchestration out of the park - otherwise they will have to lead with their less expensive and commoditized SOAP stacks - and then try to upsell to a more expensive offering.
===============================

If you believe in the 'web services programming model', 'service oriented integration' or 'pervasive integration' then it is only a matter of time before you realize that orchestration is the flagship of this rather complex offering. Protocol offerings should be built into the operating system (and will be). Queues are commoditized... and making 'web service enabled queues' takes about a day of engineering effort. The 80-20 rule on transformation makes out of the box, pipelined XSL offerings attractive. However, debugging a distributed, service oriented, message based, declarative policy based, multi-platform, asynchronous, concurrent, cross-enterprise process is.... "hard". The issue around orchestration will quickly turn to 'total cost of ownership' - the vendor that wins will be the one that allows the enterprise to reduce the cost of maintaining production instances (keeping the service network up, debugging issues, resolving performance problems, maintaining version, etc.) The offerings will have to go beyond orchestration and into 'composite distributed applications'. The expectations that will be placed on our new service oriented virtual machine (the service network) will be significant. As the new programming model emerges, people will be wanting a vendor that isn't a 'spot' technology but rather an enabler of the new model - that's where we come in.

Microsoft on RFID

From the RFID Journal, see:
http://www.rfidjournal.com/article/view/779/1/1/

Process Contracts

The concept of 'design by contract' was made popular in the object oriented world - or perhaps more precisely, in the component-oriented world. Much of the work that went into this study was centered around making a single module adhere to a contract and adhere in a consistent manner. When the Java language was introduced many people bumped into the 'interface' for the first time. This was the vehicle that guaranteed 'what' an object could do, but not 'how' it would be done.

This concept was extended with WSDL. The extensions included: defining the interface in a language neutral manner, allowing for multiple bindings between interface and implementation, allowing for a more message-centric design of parameters / arguments, strong support for remote calls and declaration of quality attributes as declarative policies (reliability, integrity, etc.)

In the process world, we continue to leverage the 'design by contract' mentality. Languages like BPEL expose themselves to the outside world through WSDL. This means that every process IS A service with a contract. In addition, BPEL makes all external calls VIA web services. Thus, all calls are first described via the contract.

As we continue to find best practices in designing BPEL code, we have tested a variety of means to get from concept to delivery. The one that seems to be getting the most ground is what we are calling 'process contracts'. Here, the analyst takes on a 'RAD' view of the process. The goal isn't to get all the detail right on the first pass, but rather to quickly identify all of the service calls that need to be made to compose the process. You can think of this almost like the old CRC exercises (Classes, Responsiblity, Collaboration) - except replace 'classes' with 'service calls'.

One thing that is becoming apparent is that people want to use BPEL in different ways. I usually put the usage scenarios into one of three different buckets: B2B, A2A and composite applications. In the B2B and the A2A scenario, it is common for the endpoints to already exist, but the contract to the service (type, message, operations) still needs to be created. In the 'composite apps' space, it is common for neither an interface nor an implementation to exist. In any scenario, the designer finds that they quickly need to begin creating the interfaces or contracts.

We are finding that most people are comfortable building out a complete process by 'stubbing' out the process. This involves identifying all of the major steps (service calls) and the flow/logic between the steps. At this point, the designer isn't going into any detail about the step, just putting boundaries around 'what it does'. The designer then goes back and starts to add detail around the contract of each call. This involves detailing out the WSDL interface (not binding and service). Here the designer begins to look at using common messages and types for consistent service calls. As the designer prepares the calls for invocation, he/she will usually begin to identify if the scope of the call is correct. This is a byproduct of identifying the 'fattiness' of the message, the 'chattiness' of the calls, etc.

All of this is happening up front - meaning it is happening before we begin to 'program in the small' via your favorite language to program services: Java, C#, etc. Upon completion of the process contract, you will have identified:
1. The interface to the process itself
2. The division of labor between the services
3. The interface for each service called
4. Verification that the data needed to call each service is available to the service (var scopes)

It is at this point where you can begin working with your service implementer (java guy, etc.). Here is how the conversation goes:

Jeff: "Hey Bob, I just finished my process contract for the new Fulfill Order process."
Bob: "Cool. How many service calls?"
Jeff: "Well, there were a total of 8 calls. 3 of them were canned and I will be sucking them out of UDDI, but 5 of them are new."
Bob: "Did you stub out the 5 new ones?"
Jeff: "Yea.. I did my best. I'll email you the BPEL and WSDL's"
Bob: "Better yet.. just check your process into version control and I'll pick it up there"
Jeff: "Good thinking. Will do. How long do you think it will take for you to turn around the implementations?"
Bob: "Jeff, I have no idea - I haven't even seen the WSDL's yet! Let me take a look and I'll get back to you..."
Jeff: "Understood. I'm going to move on to the Replenish Inventory process."
Bob: "Sounds good - I should have estimates in a few hours."

This conversation - and this style probably already feel familiar. The one thing that is improved is that in many cases, the process contract will become a new deliverable in the development cycle. It will force a more structured deliverable to come out of the late analysis or early deisign stage and may likely result in a more timely delivery of the end product.

Structured Process Cases

I've been playing with variations of the "Use Case" that are more process-centric and contract oriented. Overall, I think there is a good match between capturing process descriptions from an analysis perspective and tying that information back to the BPEL.

Although this is a very incomplete example, consider the following. A Process Analyst sits in front of their favorite tool (Microsoft Word) and writes down the name of the process, a small description and a handful of steps:



Now, the analyst adds a schema (or two) to the Word 2003 document by pulling up the "Tools: Templates and Add-Ins" window. In the example I'm using, they are adding in the BPEL schema but it might likely be a higher-level (more abstract) grammar that they choose.


Now, the analyst goes back to the document and "paints structure" into the content. This assumes that the analyst is given some extra training in such a process (think UML like training, "Process Case").



After painting the structure around the Process Case, the user is able 'hide xml' so that it looks like a normal requirements document. The analyst then takes the 'requirements' and sends it off to development. But, since it was 'painted' with a well known schema, the developer merely uploads the requirements document directly into their orchestration tool which stubs out the entire process for them. Now, of course there will be mismatches in syntax and scope, but the developer will make the changes and forward back the structured document for approval to the analyst.

At the same time, the developer will be creating a vocabulary of services. These will be marked up in WSDL (service:operation). Imagine terms like "checkInventory", "fillOrder", etc. These terms will then be made available to the analyst to drop into their future requirements documentation. By repeating this process, the links between analysis and design will continue to grow in strength.

This is still an early concept but our early tests indicate that we can significantly reduce development / integration costs by using the aforementioned closed-loop mechanism.

A first peek...

In the last few months, I've had hundreds of inquiries about the next version of the OpenStorm orchestration product. And although we are still in limited beta of the 2.0 release, I thought I'd share a single slide from the new deck.

The new product has a great feel to it, with some really interesting features. This is a shot of the main orchestration canvas, where you paint your process flow.



I'll post a bit more later on the product. Thanks to all of the people who have guided us through the endeavor and worked with us on the feature / function tradeoffs!

Does the Service Fabric make the SOA Obsolete?

I had an interesting question posed to me recently: Does the service fabric make the SOA pattern obsolete?

First, be clear. I have a very clear definition for the SOA:
1. It is an architectural pattern
2. It uses 3 actors (directory, consumer and producer)
3. Whereby, the consumer looks up an interface to a producer in the directory
4. The consumer creates a binding to the producer and calls it

The SOA pattern allows for dynamic lookup and binding, which means that it is a vehicle for finding the *right* implementation of a service at runtime. The service fabric often plays a similar role, but it often does it using routers. In this case, the fabric is aware of the implementations of a given service and when a message is sent to it, it can dynamically send the message to the *right* implementation. So, both techniques allow for a message to be dynamically delivered to a destination. But, does one make the other obsolete?

In my opinion, the answer is, "no". I am of the opinion that people will often mix the techniques. In essence, the service call will still be designed to do a UDDI style lookup, but the destination may likely point at a router. Alternatively, developers will continue to write services and populate the descriptions in a directory, then make their router aware of the directory. In this case, the directory is still being used, just by the router instead of the programmer.

Let's Merge!

Did anyone else catch this?



What? I say what?

Can you imagine the CEO of J.P. Morgan Chase and the CEO of Bank One Corp. talking before their merger about web services?
"WHAT??? Bank One isn't using web services - THE DEAL IS OFF!!!"

SeeBeyond does rush job?

I found this in the new literature for the SeeBeyond ESB/BPEL implementation:



For some reason, the phrase "a rapidly implemented, limited edition version" doesn't make me feel all warm about the product....

Here is how I read through the lines:
1. Product Marketing didn't give Engineering time to do it right
2. Engineering is likely pissed, and let marketing know it
3. Marketing is OK with it, because they still can't figure out how to position the ESB against their more profitable lines
4. A 'limited edition version' will give marketing more time to think about what to do about the 'creative disruption' of SOI

Features of a Service Oriented Language

Service Oriented, Protocol Connected, Message Based
Here are some casual thoughts on what I would like to see in a service oriented language. This is my first attempt at this... I'm sure I'll get some feedback ;-) and will update it.

Services
- The service is a first order concept, both sending and receiving
- Support for the strong interface (think WSDL)
- Mandatory support for Long Running Transactions
- Faults and Compensation are first order
- Protocols for transport and fulfillment of NFR are intentionally left out of scope

Messages
- Message is a first order concept
- Message is defined using a platform independent markup
- Support for message correlation properties; helpers for distributed state mgmt.
- Universal addressing scheme (assumes router)

Types / Vars
- Typing & vars are consistent with message system
- Remote variables (think REST)

Functional Containment
- The 'service' is a container (and managed)
- The 'object' continues to live. Objects are contained in services.

Invocation and Service Hosting
- WSDL's can be imported directly into the runtime. Operations and messages become first order citizens.
- Access & manipulation of binding / listener is first class.
- Language assumes that all systems are peer (both client and server)

Schema Manipulation
- DDL: Import / export / create / modify (consistent with type system)

Data Manipulation
- DML: transform (consistent with type system)

Flow Control
- Usual branching & looping
- Parallel execution; parallel joins (first order)
- Forced sequential processing

Event Based
- Events are a first order concept
- Time & activity based events

Metadata Ready
- Declarative metadata becomes a first order item (see jdk1.5)
- Service oriented loading / unloading of metadata / models / gen’d code at runtime
- Reflective knowledge of declarative non-functional polices (think ws-policy access)

Base Service & Extensions
- Concept of an extendable base service
- Service may have multiple interfaces
- Aspects may easily be applied to service
- Language is extended via ‘more services’ not ‘more syntax’

Service Network Awareness
- Service is aware of the network that it lives in (topology, routers, etc.)
- Service is aware of service-enabled remedies to non functional requirements (Virtualization, etc.)
- Service has the ability to modify the service network at runtime

OK. So a good question is, "which of these are part of the 'language', which are 'service libraries' or other?" I'm in favor of the *least* amount of required syntax possible. But, I like the idea of having *mandatory* service library extensions.

Clarification on Orchestration

In a recent ZapThink article, which has an amazingly similar name as my blog.... ;-)

I found this:


Ok. Pretty close... but I just want to clarify a bit.

First generation orchestrations are typically used as a process-integration or a system-to-system integration mechanism. This means that the orchestrations are VERY message oriented and the granularity between the operations are almost always coarse-grained.

In addition, there is a logical difference between "service orchestration" and "process orchestration". Typically, service integration is lower level. It involves the many calls to "technical services" and attempts to hide higher layer calls from the ugly technical details and are often exposed as a single business service. "Process orchestrations" are orchestrations that occur at a higher level. In these cases, virtually all of the calls are to other 'processes' and tie together a digital business process.

Now - in the future (2005+), I expect to see more fine-grained web services being used in 'composition' tools (a close cousin to orchestration). In this setting, more emphasis will be placed on in-lined services and performant service compositions. But for now, this is outside of the scope of what we typically classify as 'orchestration' or 'choreography'.

[BlogService]

[BlogService]
public class HelloWorldService
{
public String HelloWorld(String data)
{
return "Hello World! You sent the string '" + data + "'.";
}
}

=====================
Here is what I want to do:
1. Post source code (java, c#, etc.) on a new kind of blogging engine
2. Have the blogging engine compile my code and turn it into a web service
3. Host the service for execution (with wsdl retrieval)
Done.
Think Apache Axis (with .jws features) meets a blogging engine.

Automated Contextual and Conceptual Engineering

I just ran across something I wrote a few years ago... always interesting to look back at old notes...

Automated Contextual and Conceptual Engineering
I am attempting to convey a difficult concept to readers - right now. As I write this, my Microsoft Office is checking the spelling and grammar. It is putting my words and sentences into a context and using pre-defined rules to suggest areas of syntactic improvement. Dare I say it is using simple artificial intelligence (heuristics and a knowledge base) to improve my writing.

Visionaries have been promoting the concept of the Semantic Web for some time. By putting my words, sentences and paragraphs into context, the author is able to work with the software in a more advanced manner. If my word processor knew that I was writing a research paper on 'Automated Contextual and Conceptual Engineering', it could begin acting like an automated research assistant, scanning the web for applicable articles or illustrations, followed by suggestions on document structure, content, automated bibliographies, and footnotes. The more my software knows about what I am trying to write the more help it can offer.

Pushing context engineering to the next level takes us to conceptual engineering. Here, software not only understands the context of what I write, but understands the base concepts via conceptual ontologies. Once the software understands a concept it is able to look at the attributes of the concept and begin substituting alternative values suggesting related concepts. We would probably refer to this process as the elicitation of cross-domain metaphors or analogies.

Have you ever met someone that was good at connecting the dots in a business, scientific or personal problem? Typically these people are good at applying metaphors to problems. The goal of automated contextual and conceptual engineering is to create better content in less time while educating the author as he or she develops the content.

The Web has made it easy for anyone, anywhere to publish information. Browsers, cell phones and web pads are making it easy for anyone to read published information anywhere. I believe that the progress that we have made in mass-authoring content, cross-site syndication and ubiquitous rendering has made the world better. I now believe that the time has come to begin making the content better.

This is still an interesting concept. With the advances in web services, MS Office using XML and gains in the semantic web, this kind of stuff may be closer than I originally thought.

Things I'm Reading...

I'm in the middle of reading a few things. Here are the ones that seem interesting:

Short reads:
Steve Cook and Stuart Kent's OOPSLA report: The Tool Factory


Long reads:
Joe Armstrong's thesis: "Making reliable distributed systems in the presence of software errors"
Peter Van Roy's yet-to-be published book: "Concepts, Techniques, and Models of Computer Programming"

Blogging Terms

I'm going to break my number one rule about blogging: NEVER TALK ABOUT BLOGGING!

But, I read blogs - and there are a couple of things I've noticed...

BlogNosing
A variant of brown-nosing, this is the practice of always saying nice things about other people on your blog. Example: Bill did a great job of blah blah blah.. Bill always is right... blah, blah, blah. Don't get me wrong, often people will do good work and it should be acknowledged, but you know when you're BlogNosing...

Tightly Coupled Blogs
This is the practice of assuming that everyone reads your blog (and all of your friends blogs) every day and that in effect they have tuned into your mini-soap opera. I believe that Tightly Coupled Blogs were invented by Microsoft employees. Example: Don farted, then Gudge laughed, but that was cool because Tim and Dave from the Sicily project were walking by (in building 42) and it really didn't smell that bad... A person should be able to read a single entry in your blog - and it should be able to stand on its own. Long Running Conversations that carry Session and Identity between posts is a bad practice... don't worry, I won't do a Blog Coupling Index.

BlogSlapping
Although I didn't invent this... I do feel that I've mastered it ;-) This is the practice of calling someone out on something stupid that they've said/written. Quite frankly, I don't think there is enough BlogSlapping in the world today. In the near future, I'm considering having a special "BlogSlap Schneider Day", (one free pop shot) just to give everyone a little practice. :-) Damn, That will be FUN!

And no, these aren't 'blogging patterns'

LinkedIn

I'm now "LinkedIn" - feel free to create a 'connection'.
https://www.linkedin.com/

I don't know how to post a link to myself, so for now, use my name and email from the main search page.

Bob Martin Demonstrates His Knowledge on Web Services

Bob Martin, from Object Mentor decided to slam web services.

The reason I bring this up is because of how poorly he did it. Sure, there are plenty of ways to cut up web services, but unfortunately, many people still don't even have the basic concepts down. Even seasoned people like Bob Martin have such little knowledge on the subject that tend to confuse people, rather than actually making a valid point.

Bob says stuff like:
- it is rpc
- it is attached to http and we use it to get past firewalls
- it has a negative effect on coupling :-)
- it uses xml, which is "big, ugly and slow"

Ok, for those of you who are new to web services. It isn't rpc, it isn't attached to http, it has a positive effect on coupling and yes it uses xml as a typing system, which trades some performance for interoperability.

You know, I love when people have valid complaints against web services. There are plenty of them too. I'm reminded of the movie Roxanne with Steve Martin.



Do you remember the scene where he was in the bar and some big drunk called him, "Big Nose". And Steve Martin came back asking, "Big Nose? Is that the best you can do?" Then, Steve Martin went on to find 20 names for his big nose...

Hmm. Interesting. Maybe I should post "20 valid complaints about web services"... but, as always, I'll need your help! (valid complaints only, please... )

You and Your Research

An excerpt from, "You and Your Research" by Dr. Richard Hamming:

Now, how did I come to do this study? At Los Alamos I was brought in to run the computing machines which other people had got going, so those scientists and physicists could get back to business. I saw I was a stooge. I saw that although physically I was the same, they were different. And to put the thing bluntly, I was envious. I wanted to know why they were so different from me. I saw Feynman up close. I saw Fermi and Teller. I saw Oppenheimer. I saw Hans Bethe: he was my boss. I saw quite a few very capable people. I became very interested in the difference between those who do and those who might have done...

The Microsoft Drug

I just ran across a great presentation from Todd Proebsting of Microsoft. The following statement seems to sum up the MS philosophy: