Application Lifecycle Management

The key issue that needs addressing in software development today is the lamentably low project success rate that pervades the industry, particularly at the higher-scale end of project sizes and/or complexity. While technology has leapt ahead, project sociology has not kept pace. All too often project managers end up having to learn the hard way what works best, with no accepted body of knowledge in managing people developing software. Furthermore, where there is recognition of the people issue, senior management do not provide the resources to adequately deal with it. All too often there exists a cultural divide between the business and IT, with business handing over requirements to IT and the next contact between the two is made when delivery is expected. There is no involvement of the business with software development and no visibility into its activity - it is a black box to business managers.

Within IT, there is a lack of good practice, particularly with regard to using the right tools for the job. Senior corporate management skimp on tool resources but spend huge amounts on manpower, particularly contract and outsourced work. The opportunity to exploit automation and reduce the need for labour intensive work is not realised. Management also has difficulty in making the right choices on platform technologies, so that they best fit the business mission. All too often such decisions are left to low-level decision making, such as at the developer level, which can run counter to the long-term viability of the application. Enterprise applications never 'finish', unlike COTS, and a successful application will lead to enhancement requests and ongoing maintenance. These post-deployment costs, including support, represent as much as 80% of overall lifecycle costs, but developers rarely take that into account in their decision-making.

Business Issues

Application development continues to evolve at a rapid pace and with that there exists the need for clear strategies on how to move forward. Besides the changes in technology taking place, for example developments in Java 2 Platform Enterprise Edition (J2EE) and .NET in the enterprise, and the move towards Service-Oriented Architecture (SOA), there are also the developments taking place in methodologies, such as Capability Maturity Model Integration (CMMI) and Agile Software Development (ASD). Companies need to be flexible to meet the challenges in today's competitive business environment, and they need to put in place the right strategies for competitive benefits.

The emphasis in application development today is very much on management: the need to manage the end-to-end application lifecycle process, including management of applications once they are released into production, the need to manage feedback between different lifecycle phases, and the need to manage project resources and costs. There is increasing recognition that IT needs to be made transparent to the business and fit within mainstream business management. For that to happen the cultural divide between IT and the business will need to be bridged: there are new tools and methods that can facilitate that process when management is willing to take a more active role in aligning IT with the business. One of the drivers that is making that happen is compliance and IT governance.

The pressures stemming from compliance and internal governance are considerable today, leading to the need to manage the whole application development lifecycle. The code that is developed and deployed within enterprise applications is not only subject to increased control and scrutiny, but must also be aligned to the objectives and constraints of the business as a whole. In short, the application development process has become of strategic importance within the organisation.

There are two key process methodologies that help address the problems of low project success rates. The Software Engineering Institute's (SEI) CMMI for Software provides a best practice characterisation of software development management processes, and also within the context of wider organisational management processes. The earlier SEI standard, Software Capability Maturity Model (CMM-SW), now being phased out, was purely focused on managing software development but was also too document-centric - CMMI is leaner and also more specific about implementation. However, while CMMI can define the backbone activity processes for senior management, it is mainly descriptive rather than prescriptive - in-depth detail of how needs to be added. Agile Software Development (ASD) is a prescriptive methodology for developing software. At the outset it should be clear that contrary to the historical origination of the Agile movement and its initial reaction against CMM, the two are compatible: Agile activity can be performed within a CMM environment. Higher levels of CMM achieve greater repeatability and predictability of projects, and also adaptability to changes in the business. Change is the occurrence that best links the two: the Agile way thrives on a high rate of change in requirements and uncertainty in the project. These attributes are conditions that exist in the real world; the point of Agile is that it is good at meeting these challenges. A traditional Waterfall software development methodology, where each stage is tightly controlled and in a linear progression, is more likely to fail under uncertainty than an Agile approach.

Technical Issues

The Integrated Development Environment (IDE) is evolving, tool vendors are increasingly integrating their products to deliver suites. IDEs are giving way to tools that reach outside of pure coding and into the architectural, deployment, and management phases of an application's lifecycle: Application Lifecycle Management. The hallmark of these suites is a common user interface, meta model, and process engine that also enable ALM team members to communicate using standards-based architectures and technologies such as Unified Modelling Languages (UML).

The management of an application lifecycle as opposed to its development concerns much broader issues than that of project process and methodology. This is where the concept of ALM is so important. Applications exist in the organisation in multiple cases: through custom development and enhancement projects, through purchased COTS products, subscription applications from service providers, modules or whole applications provided through outsourcing, and legacy applications. These are valuable IT assets, and ALM helps to manage them all, delivering business value. ALM is being used as a means of inverting the traditional view of IT as a cost centre and changing it to one of value generator. The chief tools now available in ALM that can provide business intelligence on application delivery include Project Portfolio Management (PPM), Requirements Management (RM), and Application Performance Management (APM). The addition of these three activities to traditional lifecycle development provides a necessary and hitherto missing business perspective. Three key ALM disciplines are described next.

PPM plays a number of crucial roles in helping to manage application development and application usage as a business activity. Examples include the following:

• Rationalising the number of applications within an organisation, especially important where mergers and acquisitions have taken place, and providing a global view of application usage;
• Aligning IT with business needs and ensuring that projects in the pipeline meet business needs, ensuring that business cases are made for new applications to meet business needs, and prioritising projects in the pipeline that are urgent for business needs;
• Managing manpower resources to fit business needs;
• Running Quality Assurance (QA) software metrics on applications, and making cost benefit decisions on maintenance and support;
• Manage application delivery; and
• Provide Business Intelligence (BI) on application development through dashboards and analytics.
• This list of capabilities makes PPM play a powerful overseeing role in ALM, and providing the key intermediate layer between business delivery and application management.

RM is another activity that is now firmly part of ALM. For too long requirements has been a paper-based exercise, and/or an activity that starts and ends at the beginning of a project. The fault in this approach is all too evident when people dig deeper to better understand application development and how to improve it. In RM the first step is to build a business case for a proposed application and gather the requirements. This process must identify all the stakeholders in the application; all too often an application is built and only then is it discovered that a class of user has been overlooked.

Modern RM tools keep the requirements traced to code development - this ensures that the correct application is being built. A key advantage of these tools is to keep requirements in a central repository, so that any change in requirements is instantly propagated to all the project team: crucially, developers and testers. Managing change is often where projects fall down: project schedules seldom cater for change, whereas in practice change is the norm not the exception. Once the application is released the user experience needs to be fed back and RM provides that feedback mechanism. New releases therefore carry the changed requirements from the production environment, as advised by all the stakeholders. Finally, the increasing use of offshore development, with distributed teams separated by time zones, pose new challenges that Web-based requirements tools can meet, as well as assisting collaboration.

APM, together with Software Asset Management (SAM), is perhaps the least recognised of ALM activities, but ones that can unlock huge value from IT assets. Application development may typically take six to 12 months, but for a mission-critical application its lifetime in deployment may last over a decade. Managing the application during that period is where APM and SAM step in. Non-optimal performance in production seriously hampers business effectiveness and gives rise to a multitude of hidden costs. The complexity of today's IT environment means that monitoring and managing the hardware and network alone is no longer sufficient to facilitate the effective delivery of IT services. As a result, more and more organisations are utilising APM to drive application optimisation, more effectively align IT delivery to the strategic and tactical needs of the business, and increase cost efficiencies. In essence, APM gathers detailed information from inside applications and uses it to:

• Predict potential failures before these impact on users or the business.
• Provide rapid alerts when failures occur and isolate the root cause of these failures.
• Improve planning for system requirements.
• Speed-up the implementation and launch of new applications.
• APM can also be used in the development and pilot testing stage to detect application defects related to performance and system load early in the application lifecycle, and prevent introduction of those defects into the production environment.

SAM is another hidden secret in ensuring value from applications. Monitoring which applications are actually used in the enterprise provides valuable information to better manage budgets. Managing licensing compliance, including renewal alerts and document storage, is important for contract legality and corporate regulatory reasons.

In addition to this broadening of ALM, the core development tools are also improving. Modelling has made significant strides forward with the establishment of the Model Driven Architecture (MDA) standard, which nearly all players in the market have adopted. The exception is Microsoft, but it has embraced Model Driven Development (MDD) through its Software Factory concept, and its equivalent to MDA's Unified Modelling Language (UML), the Domain Specific Language. The chief advantages of modelling are: improved communication, collaboration and knowledge transfer; improved productivity and reliability through automation; and improved development through separation of concerns. Unlike historical modelling, MDA's advanced technology allows real-time synchronisation between models and code, so that models do not become shelf-ware after initial construction, but are an ongoing source of information and roadmap for the project.

Role-based core development tools enhance the efficiency of developers, whether analysts, architects, programmers, testers, or managers. These tools make available the options necessary for the role and prevent the overload of menu options that plague functionally top-heavy tools. Additionally, top-end ALM suites provide workflow graphical tools that enhance information access and knowledge transfer.

Market Issues

ALM as an integrated end-to-end suite is a reality today offered by the key vendors in this market, following organic development and recent intensive acquisitions and integration programmes. In addition to vendor-specific integration platforms, the players are also recognising the importance of plug-compatibility with the open Eclipse platform, as well as with Microsoft's Visual Studio.NET and Visual Studio Team System.

Acquisition activity is taking place in the area where development tool vendors are meeting IT Services Management and Application Performance Management vendors. For example, PPM, previously a specialist activity, is now in the centre of the space where governance of projects meets business needs, and this ties perfectly into ALM integration suites.

The project managers have better tools and better tool interoperability to exploit as part of their arsenal. When combined with best practice methodologies and attention to team dynamics, there is much to be optimistic about in the progress of software development.

The market forces behind the need for integrated suites are increased accountability - the result of IT governance initiatives - and the need to repeat development processes with increasing frequency across multiple projects. ALM suites exploit a common integration layer that enables better communication of project artefacts, such as requirements and documentation, across all ALM team members, and also keep an accurate audit log of changes.

Conclusion

Used correctly, Butler Group believes that ALM tools can help deliver faster-to-market, high quality software products. However, tools should not be used as a defence mechanism against dealing with difficult software development issues, otherwise the developers will bypass the tools and the same old problems will re-surface. Rather, there needs to be a better understanding by business of the nature of software development and its practitioners, and vice versa, software developers need to understand the impact their solutions have on the business. This comes down to communication as the key to bridging the cultural divide between IT and business. Used intelligently ALM tools and better people management can lead to that elusive project success.

Key Findings

• The low success rate of medium to large software development projects highlights the need for business to adopt an Application Lifecycle Management (ALM) approach to help control projects and ensure business needs are met.
• The scope of ALM embraces all types of applications in the organisation: custom, Commercial Off The Shelf (COTS), integration, outsourced, subscription, legacy, and enhancements.
• Management functionality in ALM aligns with quality processes and thereby helps achieve greater project success rates.
• Functionality in ALM improves team productivity, ability to deliver applications rapidly, improve quality, reduce risk, and align IT with business needs.
• ALM has evolved towards higher business level abstractions, making IT visible as a business unit to corporate management, and helping to bridge the cultural gap between business and IT.
• ALM tools can be misused if implemented as a defence mechanism against the difficult issues in software development. Lack of adoption results in wasted investment and worse productivity than before its introduction.
• ALM tools need to be combined with appropriate project management and development methodologies.
• Industry polarisation around key development platforms, Eclipse and Microsoft Visual Studio.NET, benefits the development community by lowering costs and providing tool interoperability that gives ALM tool investment increased longevity.
• ALM tools are an important software investment and IT asset in their own right.