Disposable Applications

In the early 90's, the company that I worked for had 3 different application platform strategies:
1. Purchase large, off-the-shelf (OTS) packaged apps
2. Build applications that were core to the business (Core)
3. Build rapid apps for non-core or non-long-living applications (Non-Core)

Back in those days, we used PowerBuilder for #2 (Core). PowerBuilder gave us enough structured techniques to build fairly complex business applications. It recognized the number one use case in business apps: Capture user information, Save it, Make it available to others. Surprisingly, this DB intensive technique could be used on a high percentage of problems. It stunk at workflow / groupware - but luckily this is when Lotus Notes was emerging. Now we had a tool that could rapidly create collaboration environments for non-structured data. We could also use the people in the company who couldn't code worth a shit (and we couldn't fire) to actually get something done. Yes, we called this people "Notes Programmers". And we had a ton of them.

Generally speaking, we considered the Notes team to be creating 'Disposable Applications'. The people were cheap, the licensing was cheap enough, and the throughput was high. We allowed them to knock out small applications - and they did - they churned through them. Then an interesting thing happened... our users started telling us that they wanted the new application to be done in Notes (not PowerBuilder). "What? You want it in Notes??? Those are the idiot programmers that we couldn't fire." I thought to myself.

An interesting dynamic had occurred. Our users realized a handful of things:
1. The Notes team delivered applications faster than the PowerBuilder team.
2. The Notes team didn't make the users feel like technical idiots; thus they became friends with the team.
3. On occasion, the users would make small changes to the Notes applications themselves.
4. The users realized that in addition to collecting, storing and retrieving data, virtually all business processes involve workflow and collaboration. In fact, the collaboration was often more important than the structured data.

I have the unique opportunity of seeing lots of service oriented offerings. Virtually all of them are of the 'PowerBuilder' classification. Most of them start off by integrating into Eclipse or Visual Studio. Ok - with this as a starting point you have already determined that you aren't a disposable application. The next thing that I see is that vendors expect people to understand XML Schema. This again, precludes the disposable community. Should they know XSLT? Nope. BPEL? Hell no. XQuery? Uh... No.

In order to create a Disposable Application environment around a service oriented infrastructure, one thing is absolutely necessary:
- The developer / author shouldn't have to know anything about SOA. No Exceptions.

Most talented engineers hate environments like Lotus Notes. They roll their eyes thinking about scripting hell, inability to enforce uniform constraints and business logic, inability to leverage a common data model and perhaps most significant, it allows dumb shits to look smart.

Talented engineers would much rather create a distributed state machine leveraging a set of 30 WS-Protocols across a messaging infrastructure that leverages a VM that facilitates heap size manipulation, while programming to a set of 17,000 classes that represent the "enterprise API". And oddly, customers are preferring to buy packaged applications that are already fully developed over having custom apps built. Who'd have guessed that one?

So, you ask, am I in favor of disposable applications? Hell yes. It may hurt your ego but it will be kind to your wallet. The real trick is to determine how to design a service network to facilitate disposable applications. It should be possible to create a constrained and structured set of services that contain the end development environment enough to allow 'power users' to do their thing. Then, it is off to the races.

A Note on BEA

The recent departure of Adam Bosworth from BEA has many in the industry talking about the overarching issues that seem to be facing that company. As an ex-BEA partner, I witnessed first hand several unfortunate but not so uncommon events take place, including:
- trouble with channel conflicts
- strong sales people being fired (even in very tough environments)
- less than capable people hired into marketing and bizdev
- inability to articulate a new vision
- failure to create an open source strategy 
- failure to answer the IBM global services threat
- inability to create vertical offerings
- chose not to acquire when their stock price was down - despite industry consolidation
Now, I get the feeling that there were more issues on the inside - but I can only tell you what I witnessed from the outside looking in.

In my opinion, BEA was failing during an absolutely critical period of change in the software industry.
- Venture funds over invested and killed the cash cows of medium sized ISV's
- This fortified the positions of the large ISV's (MS, IBM, SAP)
- Customers were moving to process based applications, not technologies
- Large ISV's moved to an agility based packaged app model

This set the stage for:
-- Microsoft building out .Net and growing Microsoft Business Solutions
-- IBM will build out WebSphere and acquire packaged app companies (Siebel, etc.)
-- SAP continuing the component/service push on NetWeaver
-- Oracle continues to build apps and grow the technology platform (Collaxa acquisition, etc.)
All in all, we are seeing a trend; major application companies are going to market with an application platform that they provide. Smaller ISV's are often picking JBoss or .Net. And medium sized businesses are being attacked by aggressive IBM sales teams.

So, where does this leave BEA?
For starters, BEA should have seen this coming and pushed a deal with either SAP or Oracle some time back. Upon seeing that Open Source offerings was having a significant effect on the low end of the ISV market via rapid commoditization, BEA needed to make a move. Going forward, variations of open source models will dominate markets that are easy to commoditize. Integration will likely be on of those markets - technologist understand the use cases. 

The departure of Adam Bosworth is only one of many issues for this company. Although their products are sound (from a Java perspective), they moved extremely slowly in their web services strategy. Could BEA acquire enough companies to be considered a 'service oriented infrastructure' company? Maybe - but the market is up and the web service startups are making sales which will substantially increase their valuations. This makes it much tougher to stomach the acquisitions. Let's say they pull it off. Is it a good move? I'm not sure. Mostly, I think it just keeps them around until the next wave of mergers takes place.

I wish that I could offer BEA some magical advice. It is a company that has been good for the software community. I don't have the answer, but I have some inclinations:
1. From a technical perspective, look beyond Java. And way beyond J2EE.
2. QuickSilver and the ESB space are already commoditized. Back an open source project.
3. Web Services are easy, "Integrated Service Networks" are hard.
4. Understand why people like the idea of a "Google computing platform"
5. Packaged apps will dominate until the build vs. buy dynamics reverse. Unless you plan on being acquired soon, your job is to reverse it. It's that simple.

As I look at these 5 items, I realize that this is good advice for any of the ISV's that plays in the app-dev, or integration space. Here are a couple more things to chew on:
- web services and service networks increase the complexity of the enterprise I.T. stack
- the more moving parts there are the more it costs to bring them together
- not all buyers need the same level of complexity (if only we had J2EE Beginners version, J2EE advanced version, etc.)
- service oriented infrastructure will be most beneficial to very large I.T. shops
- if you refactor software out of the application and into the network, you should be able to reduce the development and integration time - if you didn't; you failed.
- Remember - dBase, PowerBuilder and ColdFusion. Developers like when you make them look like rock stars - give them the tools to beat expectations.

I feel a bit sorry for them, but mostly I'm mad. As the turnover continues at BEA we can be hopeful that they'll either acquire the companies with the talent or bring in some new vision to steer the ship. Despite my frustrations with them, I wish them the best of luck.

Distributed Noun Processing

Lately, I've been very concerned about the state of Service Network designs.

The service oriented world puts an emphasis on verbs (authenticate, format, translate, etc.) This forces the knowledge about nouns to be distributed across a set of services. Service oriented programming usually looks like a pipelined or staged computing model. Often the messages that are being processed refer to the same schema. However, the processing instructions vary according to the specifics of the schema.

Let's say that we have a four stage service plan:
Step 1 - pull message off of bus and authenticate
Step 2 - transform message according to local needs
Step 3 - process business service
Step 4 - business services encapsulates a call to a data service

Each service focuses on one stage and will typically have a control language (DSL) for performing functions on the nouns (business schemas) that are passed to it. What seems to be missing from most commercial and homegrown services is the ability for a service to easily add or modify the nouns that it is aware of or modify the routines that perform the functions on those nouns.

This is a huge issue. For some reason, people continue to focus on the virtues of SOA/WS-*; loose coupling, reusability, etc. However, they are not talking about the new problem: Distributed Noun Processing. The realization of a use case is now spread across multiple services. Changes to nouns will likely result in changes to all services in the plan. It is likely that the DB schema changes, the O/R plan changes, the XSLT changes, the message type in the WSDL changes, the evaluation logic in the business class changes. The impact of noun changes must be measured (part of the agility/fragility index).

Now a seasoned person might say, "but that is the current state of art as well". Yes - this is true. Generally speaking J2ee and .Net fail to provide facilities for cross-cutting functional concerns across their tiers. So, the good news is that we're only failing as bad as we did last time. However, I think we failed pretty hard last time.

As we look at services, we must identify the best programming model associated with the functionality. Generally speaking, this means that services rely on specialized verb engines. These verb engines usually rely on their own DSL (think DML and DDL for relational databases, axiom languages for business rules, transformation languages, etc.) As the number of DSL's increases, so does the importance of coordinating changes across them.

At Momentum, we have been spending a significant amount of time looking at this issue. We have not found the magic "DSL propagation language" yet, nor do we expect to anytime soon. The MDA world has made progress but is still very disconnected from the service oriented world. For now, the effort is either done by hand or with some tooling that helps to automate impact and change management across a set of predetermined DSL's. Failure to manage the service network from a functional perspective will likely lead to excessive costs for new application introduction and change management. Trust me - get it under control now.

Composite Application: Def #1

Phil at Loosely Coupled was kind enough to send me a definition of "composite application":
 
================================
Here’s Loosely Coupled’s definition:
 
An application built by combining multiple services. A composite application consists of functionality drawn from several different sources within a service oriented architecture (SOA). The components may be individual web services, selected functions from within other applications, or entire systems whose outputs have been packaged as web services (often legacy systems).
http://looselycoupled.com/glossary/composite%20application
 
It’s a favourite term of Gartner’s Massimo Pessini, the man who also gave us ‘ESB’
================================
 
I had great conversations on this topic recently with some really smart people. Unfortunately we were all rather drunk and I don't remember what we concluded - although I think it had something to do with molecules. Ok, this in only partially true. We basically agreed that the definitions of composite apps was still too loose (SOI, SODA, integration at the glass, how many of the NFR's need to be addressed by services, how much of the app needs to be SO enabled, etc.) Bottom line is that we need to move to a 'definition by constraints' view.
 

ToDo: Create Agility Index

On my list of things to do is to create an agility index (and a fragility index). Before I go too far down the road I thought I'd post to see if anyone already has a favorite agility/fragility index in use. Lemme know.
 
Oh yea... so it isn't about being service oriented. Service networks must be evaluated from an 'ility' perspective. I'll start with agility.

Nice paper

Google led me to this nice paper by ACM Fellow, Carlo Ghezzi:
http://www.elet.polimi.it/upload/ghezzi/_PRIVATE/paper.pdf

Composite Apps?

I just caught this from one of the Oracle guys :-)

"The focus of this new blog will continue to be around BPEL and the design of composite applications." (Collaxa)

Has anyone defined "composite application" or determined the criteria for an application to be considered "composite"?

If so, please send me a note - or just post it.

The economics of integration has already changed.

The economics of integrations have already changed.

NEW YORK, July 2 (Reuters) - WebMethods Inc. (NasdaqNM:WEBM - News), whose software links different applications, expects a wider quarterly loss than previously forecast, citing an "unusual number" of large deals that did not close by the end of the period, it said on Friday.

Its stock tumbled 33 percent, or $2.74, to $5.60 on Nasdaq, its lowest level in more than 18 months.

Consider the effects of service oriented integration. The game has already changed. I'll repeat this - the game has already changed.