Toward the end of the 1990's, it became clear that the majority of organisations embarking on large and complex software developments were being overwhelmed by the breadth and complexity of the technology confronting them and the new approaches to software engineering.
At the same time, the emergence of the Web and the drive towards customer facing applications has made software development a very visible asset of any large organisation with the concomitant damage to the corporate image when applications fail, cannot scale or perform poorly.
The rapidly moving competitive business landscape has undermined the traditional Information Engineering methods of defining, designing and implementing large software development projects in a 'waterfall' manner. The shortcomings of this approach were clear: by the time the solution was delivered it was no longer relevant – The business world had already moved on and left the original solution behind. In addition, the waterfall approach tended to hide latent and inherent flaws in the architecture and technology till late in the project. This almost inevitably resulted in "late breakage" causing significant and expensive scrap & rework very late in the project often introducing significant risk of complete project failure.
Further, the fluidity of business requirements throughout the project lifecycle has created real-world pressures on software development processes. Firstly, how do we continue to add business functionality throughout the initial development process? And secondly, how do we guarantee we will be easily able to add "newly discovered" requirements into future increments of the solution? :
This scenario has forced modern software engineering methodologies to be developed that address the shorter Time-to-Market demanded by organisations whilst at the same time providing a flexible and scalable 'learn while you go' design capability. Hence the birth of iterative development approaches to software engineering.
However, the emergence of these modern software engineering methodologies did not automatically bring with them the requisite skills and experience to leverage these new processes to the best effect on large and complex projects. Too little was fully understood in managing these complex projects and associated risks - both from a business and technical perspective.
What are some of the major issues facing large software development projects today?
Complexity: "Everything should be made as simple as possible - but not simpler" - Albert Einstien The explosion of new technologies and standards released over the last few years have combined to make development of large systems hard – and it is getting harder by the day with the introduction of “new and better” ways to develop solutions. How much architecture is too much? How much development process is too little? Will our architecture cater for the “Unplanned Success” of thousands of concurrent users? Is our project complex enough to require a “System of Interconnected Systems Architecture” or will an eXtreme Programming (XP) process be sufficient? Can we use Component Architectures and if so, how do we mitigate against the risk of inherent flaws in 3rd party components & the cost of integrating those components? What mix of technologies is appropriate and mutually compatible?
How would we make these kinds of decisions?
Risk: “Only 27% of software projects succeed” Standish Group – Chaos Report 2005 The IT industry does not have an enviable record of delivering projects on time, on budget with the original functionality agreed. One of the historical reasons for this is that the traditional methodologies and processes used are inappropriate for the manner in which modern projects evolve during their lifecycle. The “Waterfall” process is inadequate in catering for today’s complex projects where requirements constantly evolve and unproven technologies and new standards are introduced into the project under pressure. The Waterfall process will most likely lead to late breakages in the project requiring significant scrap and rework. Today’s projects typically don’t have a finality to them but are more a succession of increments that enhance functionality and extend the ROI and life-span of the solution.
How can we mitigate against these significant risks?
Time to Market: “Nothing is a waste of time if you use the experience wisely.” Rodin “Internet Time” now drives many projects. However, the Internet has not changed the fundamentals of software engineering. 90 days is not an unreasonable timeframe to deliver something of value to your business. The challenge is to accurately estimate what can be delivered and to apply just enough architecture and process to that first functional increment so that it can become the core of a future solution that is robust, extensible and functionally rich. How do we find that balance between delivering quickly to the business yet guaranteeing what we deliver remains a valuable building block?
Experienced People : "Experience is the one thing you can’t get for nothing” Oscar Wilde The IT industry attracts some of the world’s most talented individuals. However, the challenge for organisations isn’t finding smart people: We are fortunate enough in our industry to be surrounded by very bright people. The real challenge is to find appropriately experienced smart people. Few people have the opportunity to work on software development methodologies and processes – fewer still have had hands-on experience developing software & application architectures for large and complex projects. However, an inappropriate development process or an software architecture with inherent weaknesses or incorrect assumptions can doom a project to failure very early in the project without any visibility of the incipient iceberg Where can we find appropriately skilled people to deliver a "no surprises" predictable outcome?
These issues constitute a few of the major areas to consider in getting a project successfully across the line. They highlight the absolute necessity for a practical and elastic software development process that scales to the size and complexity of the project in hand and that will yield a predictable, deterministic and successful outcome. This process must be repeatable and engender an increasing ROI as it is used across multiple future projects.
In addition, any large or complex project must develop architecture artifacts that drive the software design and consider and define all the technology issues that may impact on a robust, scalable and extensible solution.
back to top
|
