This is an articled published publicly my McKinsey Company in Oct.2012, which gives a good insight about important points in delivering IT projects. Facts provided in the article are also relevant for anybody who is planning to outsource software development activities.

As IT systems become an important competitive element in many industries, technology projects are getting larger, touching more parts of the organization, and posing a risk to the company if something goes wrong. Unfortunately, things often do go wrong. Our research, conducted in collaboration with the University of Oxford, suggests that half of all large IT projects—defined as those with initial price tags exceeding $15 million—massively blow their budgets. On average, large IT projects run 45 percent over budget and 7 percent over time, while delivering 56 percent less value than predicted. Software projects run the highest risk of cost and schedule overruns.

These findings—consistent across industries—emerged from research conducted on more than 5,400 IT projects by McKinsey and the BT Centre for Major Program Management at the University of Oxford. After comparing budgets, schedules, and predicted performance benefits with the actual costs and results, we found that these IT projects, in total, had a cost overrun of $66 billion, more than the GDP of Luxembourg. We also found that the longer a project is scheduled to last, the more likely it is that it will run over time and budget, with every additional year spent on the project increasing cost overruns by 15 percent.

Staggering as these findings are, most companies survive the pain of cost and schedule overruns. However, 17 percent of IT projects go so bad that they can threaten the very existence of the company. These unpredictable high-impact events—“black swans” in popular risk parlance—occur significantly more often than would be expected under a normal distribution. Large IT projects that turn into black swans are defined as those with budget overruns of more than 200 percent (and up to 400 percent at the extreme end of the spectrum). Such overruns match or surpass those experienced by black swans among complex construction projects such as tunnels and bridges. One large retailer started a $1.4 billion effort to modernize its IT systems, but the project was eventually abandoned. As the company fell behind its competitors, it initiated another project—a new system for supply-chain management—to the tune of $600 million. When that effort failed, too, the retailer had to file for bankruptcy.

Four ways to improve project performance

So how do companies maximize the chances that their IT projects deliver the expected value on time and within budget? Our surveys of IT executives indicate that the key to success lies in mastering four broad dimensions, which combined make up a methodology for large-scale IT projects that we call “value assurance.” The following elements make up this approach:

• Focusing on managing strategy and stakeholders instead of exclusively concentrating on budget and scheduling.
• Mastering technology and project content by securing critical internal and external talent.
• Building effective teams by aligning their incentives with the overall goals of projects.
• Excelling at core project-management practices, such as short delivery cycles and rigorous quality checks.

According to survey responses, an inability to master the first two dimensions typically causes about half of all cost overruns, while poor performance on the second two dimensions accounts for an additional 40 percent of overspending.

1. Managing strategy and stakeholders

IT initiatives too often pay little heed to strategy and stakeholders and manage projects purely according to budget and schedule targets. The perils are illustrated by one bank’s transformation effort, in which its finance department became involved only a few months before the system was due to go live. This led to several complex changes in the accounting modules as a result of a recently introduced performance-management system. Coming so late in the day, the changes delayed the launch by more than three months, at a cost of more than $8 million.

Top-performing projects, on the other hand, establish a clear view of the initiative’s strategic value—one that goes beyond the technical content. By building a robust business case and maintaining focus on business objectives along the whole project timeline, successful teams can avoid cost overruns. They can also, for example, ensure faster customer response times, obtain higher-quality data for the marketing organization, or reduce the number of required manual processes.

High-performing project teams also improve the ways in which a company manages its internal and external stakeholders, such as business and IT executives, vendors, partners, and regulators. They make sure the project aligns with the company’s overarching business strategy and undertake detailed analyses of stakeholder positions. Project leaders continually engage with all business unit and functional heads to ensure genuine alignment between business needs and the IT solutions being developed.

Good stakeholder management involves foresight when it comes to selecting vendors and negotiating contracts with them. Company negotiators should proactively identify potential risks and, for instance, expand their focus beyond unit price and seek to establish “win–win” agreements. Doing so can help ensure that the company has preferential access to the vendor’s best talent for an extended period of time.

Some companies have learned this the hard way. A bank in the Middle East negotiated hard for price with a vendor and later suffered at the hands of an inexperienced vendor team. Another bank scored well on unit price with a software-package provider for the project phase of a trading-system implementation but encountered high costs for changes and support after the system was introduced and the bank was locked into the new technology.

2. Mastering technology and content

Drawing on expert help as needed, high-performing teams orchestrate all technical aspects of the project, including IT architecture and infrastructure, functionality trade-offs, quality assurance, migration and rollout plans, and project scope. The right team will understand both business and technical concerns, which is why companies must assign a few high-performing and experienced experts for the length of the program. We estimate that the appropriate experts can raise performance by as much as 100 percent through their judgment and ability to interpret data patterns.

One common pitfall occurs when teams focus disproportionately on technology issues and targets. A bank wanted to create a central data warehouse to overcome inconsistencies that occurred among its business-unit finance data, centralized finance data, and risk data. However, the project team focused purely on developing the IT-architecture solution for the data warehouse instead of addressing the end goal, which was to handle information inconsistencies. As a result, the project budget ballooned as the team pursued architectural “perfection,” which involved the inclusion of unneeded data from other systems. This added huge amounts of unnecessary complexity. With milestones and launch dates constantly being pushed back and investments totaling almost $10 million, the bank stopped the project after 18 months.

In contrast, one public-sector institution was able to rescope and simplify its IT project’s technical requirements even though most stakeholders believed doing so was impossible. To eliminate waste and to focus on the items that represented the greatest business value, the team introduced lean3 techniques. At the same time, it established rigorous testing and rollout plans to ensure quality and introduced clearly defined stage gates. Through these and other actions, the team was able to check 95 percent of all test cases, fix critical defects, and verify the fixes before continuing from the unit test phase to integration testing.

3. Building effective teams

Large projects can take on a life of their own in an organization. To be effective and efficient, project teams need a common vision, shared team processes, and a high-performance culture. To build a solid team, members should have a common incentive structure that is aligned with the overall project goal, in contrast with individual work-stream goals. A business-to-technology team that is financially aligned with the value-delivery targets will also ensure that all the critical change-management steps are taken and that, for example, communications with the rest of the organization are clear, timely, and precise.

To ensure the smooth start-up of new front-end and core systems that more than 8,000 people would use, one company team launched a massive—and successful—change-management program. The program included a regular newsletter, desktop calendars that highlighted key changes and milestones, and quarterly town-hall meetings with the CEO. The team made sure all top business-unit leaders were involved during the user-acceptance phase. The company included at least one change agent on each team. These agents received training that instilled a clear understanding of the benefits of the IT change. The actions helped the company to verify that it had the required business capabilities in place to make full use of the technology being implemented and that it could deliver the business value expected in the overall project business case.

4. Excelling at core project-management practices

To achieve effective project management, there’s no substitute for tested practices. These include having a strategic and disciplined project-management office and establishing rigorous processes for managing requirements engineering and change requests. The project office should establish a few strong stage gates to ensure high-quality end products. At the same time, it needs to strive for a short delivery life cycle to avoid creating waste in the development process.

One public-sector organization established strong project control by defining an initiative’s scope in an initial six-month phase and making sure all stakeholders signed off on the plan. Beyond this phase, the organization’s board had to approve all change requests, and the project was given a pre-defined cost-overrun buffer of less than $2 million. Another organization, a high-tech company, established clear quality criteria for a project master plan, which mandated that teams break down all activities so that they required fewer than 20 person-days to complete and took no longer than four weeks.

In yet another case, instead of following a “waterfall”4 or linear approach, a company created integrated business and IT teams that worked on an end-to-end basis in their respective work streams. In other words, the teams participated from the beginning of the project to its completion—from defining requirements to testing. This approach helps to avoid misunderstandings during stage transitions and ensures clear responsibility and ownership. It also promotes efficiency gains and fast delivery.