Orchestration in the Large versus Services in the Small

My recent thoughts on 'refactoring' and 'collective code ownership' reminded me of the seminal white paper, "Programming in the Large versus Programming in the Small". I just went back and reread the document - which was a bit depressing. The authors, DeRemer and Kron, articulate a problem that continues to haunt development shops today; recognizing the difference between programming small modules versus integrating those modules into large systems. Every once in a while I feel like my profession (software engineering) is making progress, then you read a paper like the aforementioned and realize that we are still working on problems that were clearly described in 1976.

For those of you who aren't familiar with the work, here are a few highlights: "We distinguish the activity of writing large programs versus the activity of writing small ones. By large programs, we mean systems consisting of many small programs (modules), possibly written by different people. We need languages for programming-in-the-small, i.e. not unlike the common programming languages of today, for writing modules. We also need a 'module interconnection language' for knitting modules together into an integrated whole and for providing an overview that formally records the intent of the programmer(s) and that can be checked for consistency by a compiler."

The authors go on to identify, "languages for programming-in-the-small (LPSs)" as well as, "module interconnection language (MIL)". The LPS is described as a 3GL, while the MIL takes on a bit more abstract definition, "An MIL should provide a means for the programmer(s) of a large system to express their intent regarding the overall program structure in a concise, precise, and checkable form."

Moving this discussion into the 21st century, it is easy to recognize that the world of web services is tackling the dual-language challenge. For programming in the small, web services encourage you to use your favorite tightly bound language (Java, C#, etc.) For programming in the large, web services encourage you to use a loosely coupled orchestration language such as BPEL4WS.

Concluding with thoughts from the authors,
"In summary, then, an MIL should:
(1) encourage the structuring of a system before starting to program the details;
(2) encourage the programming of modules assuming a correct environment, but without knowledge of the irrelevant details of that environment;
(3) encourage system hierarchy, while allowing flexible, if disciplined connections between modules;
(4) encourage information hiding and the construction of virtual machines, i.e. subsystems whose internal structure is hidden, but which provide desired resources; and
(5) encourage descriptions of module interconnectivity that are separate from the descriptions of the modules themselves."

I'm a huge fan of creating modern module interconnection languages (MIL75 is a bit dated). Although it is too early to tell if languages like BPEL4WS will survive, I sincerely hope that a new generation of programmers are introduced to the concept of programming in the large. More stuff here.