Sunday, June 25, 2006

Software development complexity

The whole idea of academic research is to reduce the complexity of software development by discovering and mechanizing software abstractions such as structured programming, object-oriented and component-oriented development.

The mechanization of abstractions demands equally innovative technological discoveries. Technological patches merely compound complexity without offering a solution. This note discusses the causes of increasing complexity and the correct approach for eliminating much of the artificially created complications.

Naturally, a venture capitalist (VC) invests in search of profit. A VC is not interested in anything other than the possibility of a high return on investment. One would think that an established technology leader considers factors beyond marketing twists strictly for profit.

Marketing cheap untested technological patches and ideas like MFC, COM, ATL, ODBC, SOM, XML, Beans and Aspects etc. have resulted in a chaotic mess with virtually zero progress towards solving the problems of software development. These technologies have complicated software development even on the specific platforms for which they were expected to help reduce complexity.

When attempting to solve a new problem one sets out by building a prototype. In doing so many good and bad ideas accumulate, which is acceptable for a free product such as Linux. Eventually, those who dare will weed out the bad ideas. However, expectations are quite different when the product is bought from technology leaders.

Recently IBM has been marketing its new invention “Software Governance” in conjunction with Service-Oriented Architecture. The next step will be “Software Legislature” for making laws to impede buying software from other vendors.

Technology leaders have been acting very much like VCs seeking profit without regard to the value of the products they are marketing. Indeed, there is more conscious effort to lock competitors out rather than reducing the complexity of developing software. As a result the complexity of software development has been rising with serious repercussions for the maintenance phase.

Since the solution to a problem must be expressed in a language, ultimately the mechanism for controlling complexity is in fact of linguistic nature. However, there are several important considerations in order for an abstract language to qualify as a viable candidate for a formalism for software development. Some of these factors are discussed below.

Solving contemporary problems

Automation is very often confused with adding more libraries and software-development kits (SDK). Instead of rectifying the problem these solutions compound the complexity of development. They require learning new specific skills that are not transferable to other platforms despite the fact that they are generally skirted that way.

A contemporary challenge is component-oriented development (COD). COD is distinct from multi-tiered development and Service-Oriented Architecture (SOA) but is effective in reducing their complexity. COD is also important because it presupposes platform-independence and distributed computing. Another challenge is strong mobility for Autonomous Agents. The code for an autonomous agent should be oblivious of the mechanism for transporting its state.

These challenges require linguistic solutions, and cannot be accommodated with libraries or patched with technologies like DCOM or CORBA. Furthermore, a linguistic solution for these problems will enable the compiler to trap a lot of obscure errors that generally go unnoticed.

Interoperability

It is not feasible to rewrite everything once a new language is born. The language must include a smooth mechanism for interoperability with major legacy languages. The most reliable form of interoperability is linkage. First, the compiler can catch most of the easily missed errors. Second, the linker will be able to complement compiler’s error checking.

Monotonicity

The distinctive characteristic of a formalism is its ability to grow monotonically. A programming language not designed with extensibility in mind is a dead-end. Monotonic growth means that the expressiveness of the language grows in order to absorb new scientific discoveries.

As the term monotonic implies, new extensions must not impact software already compiled. Furthermore, the term formalism connotes independence from technology. A programming language with different versions for each category of application is inherently technology dependent, and therefore not monotonic.

Device or application centric

A monotonic formalism should support two distinct views, seamlessly. In device-centric view an engineer needs proper mechanisms, such as pointers for efficient implementation of certain aspects of an algorithm. A language like C is entirely device-centric because it is intended for programming devices. However, the device-centric view of a formalism should not be used for device programming.

Application-centric view deals with class, template, COD, SOA and strong mobility among other things. The device-centric constructs are few and all well known. Future extensions are likely to be in the category of application-centric view.

The point here is that, a language with multiple versions such as an enterprise-edition and a PDA-edition is not a single language. In fact, a multi-edition language is just as complex as a language amended with library APIs for each particular purpose, and at times even more confusing.

Conclusion

For almost two decades there has been no significant technological progress in parallel to scientific research, such as COD, mobile agents and model-driven development. The measure of progress in reducing the complexity of software development is not the number of technological patches pumped out as marketing strategies. Indeed, software development has become increasingly more complex with the emergence of such patches.

Technology leaders seek to maintain their market shares at a high marketing cost for their users. Consequently, their research is limited to marketing strictly for profit. They continue to complicate software development in order to maintain control of the market. As a matter of fact, they do not even have the expertise to solve the contemporary problems of software development because their experts are saturated with a bundle of technology patches that must be preserved for profitability.

Z++ programming language is the future of software development formalism, and is freely available from ZHMicro .