So here’s the question: what do IT budgets and government entitlement programs have in common? And here’s the answer: both are budgeted and funded on the basis of entitlement. While government budgeting for entitlement programs is familiar, the IT corollary, and its implications for software development processes, is what we intend to focus on here.

How can an IT organization determine and justify its software development budget? In principle, budgets should tie business requirements to the effort necessary to deliver those requirements in new or existing systems. The results should be reliable, with the business having every expectation that high-quality software will be consistently delivered within the budget allocated.

Unfortunately, most IT organizations cannot do this type of budgeting. After all, how does the developmental budget document the fact that 20% of development projects are never delivered at all, or that 40% will substantially exceed project-level cost estimates and timeframes? How many CIOs could get approval for a budget that explicitly allocates 40–50% of development resources to rework (again, the industry average)? Obviously, another rationale for funding is needed.

Our observation is that IT budgets end up being derived from past spending patterns rather than clear cost-to-deliver calculations. The budget is loosely based on head-count and fixed costs, with adjustments for the corporate cost goals or special funding for large, high-visibility software initiatives. In effect, like government entitlement, funding is largely predetermined. If the budget last year was $2 million, then, unless there is a major business disruption, it will be about the same the following year.

Unfortunately, this budget approach hides the core drivers that determine how ineffective the organization’s software development processes really are. In a disruptive economic situation such as we are now experiencing, this leaves CIOs without a meaningful way to price and guarantee delivery of high-quality software at the lowest cost. Technologies and processes that could eliminate substantial rework costs or improve quality get ignored because those costs are not directly evaluated in the budget process.

We strongly encourage our clients to start measuring and understanding the cost drivers that truly impact their software development budgets. It’s also important that they measure rework rates and root causes of major quality problems. This gives CIOs the information to make the right technology and process investments for a more reliable and higher quality software delivery. And that is the best way to ensure adequate, rational IT budgets that everyone’s entitled to.

Every year we attend several software development and testing conferences, including the STAR conferences. In principle, these events provide QA managers and CIOs with information on new advances in software quality technology. We look for important new technologies and services that truly will make a difference­—that is, new stuff that substantially raises the ability to routinely deliver complex software on time, on budget, and with few, if any, functional defects. We look for advancements that will make any IT organization more productive, not just a few of them.

And every year we come away disappointed. The solutions for software quality remain very ad hoc and situation specific. One person commented that the technical sessions all seem to boil down to “test harder and work smarter.” Training seminars represent a huge potpourri of approaches to testing, with no clear way of discerning which approach delivers reliably and under what circumstances. As a CIO, will my ability to deliver quality software benefit more from training in Exploratory Testing or the course on Systematic Testing? How will I know? On the technology side, everyone wants to buy or sell a tool. The “tool” CD has become the aspirin for relieving software quality woes. Unfortunately, aspirin is not the solution when the patient has pneumonia.

IT budgets are going to shrink in these economic times. And if we cannot deliver defect-free software within these tougher resource constraints, our companies will be at risk. The big cost- and quality-drivers are known: poor specification of software requirements followed up by poor and inconsistent testing. These issues double the cost of many software projects and add significant unnecessary cost to the rest. In the average IT budget, these avoidable costs represent big numbers.

It’s time to stop merely scratching the surface of software quality. Quality cost-drivers are not going to go away by buying a few seats of a $4,000 tool or sending a few QA folks to a training class and hoping for some incremental improvement. If you want your software development costs to substantially decrease, then it will require commitment to changing processes and skills as well as technology. The essential change is to begin integrating requirements specification with testing, particularly test case design. Create testable functional specifications and the suite of test cases that will fully exercise that functionality in the code—both at the same time, both before the code is written (with all the bugs introduced). Call it “requirements-based quality” or getting it “Right From The Start.” The processes and technology are different from traditional testing. Instead of writing test cases, test engineers evaluate the logical completeness and consistency of the functional specifications and let advanced technology write the detailed test cases. Software design and software testing proceed in tandem, not sequentially. Testing skills emphasize critical analysis of specifications over trying to “break” the software during testing.

Moving to a requirements-based quality approach does not represent risk. It works. The processes and supporting technologies are well understood and have been consistently proven in a wide range of software development environments and organizations. Implementation can be accomplished on an incremental, save-as-you-go basis. There are several very experienced resources in the marketplace to make your organization successful.

Software cost and quality pressures are only going to rise in the foreseeable future. It’s time to stop merely scratching the surface of the problem with another tool or class and get to the root of software costs and defects.

A short time ago we were at a meeting of IT executives. A major new CIO survey had been released, and, once again, one of the top ten concerns of CIOs is the impact poor requirements have on delivering software on time and on budget. So in the Q&A session, the question was posed: Since the requirements problem has been a top concern of CIOs for at least the last ten years, why hasn’t anyone done anything about it?

The answer was summed up succinctly by one of the CIO panelists: “Requirements are the elephant in the room that no one wants to talk about.” Poor requirements specification accounts for about 80% of all software development rework, and rework itself is typically half of the cost of software development overall. So it is a big elephant indeed, especially as budgets get tight in tougher economic times.

Why Is This?

After all, proven, reliable, technology-backed solutions are out there that guarantee correct implementation of requirements into applications with few defects and very low rework costs (we should know; we’ve been doing it for years!). What keeps the requirements problem on the CIO top-ten issues list? We think there are two factors.

How to Improve

First, improving requirements cannot be done by simply buying another tool and some training classes. It also requires changes to roles, responsibilities, and skills, especially in the QA and testing organization. Many IT organizations are simply unwilling or unable to make these changes. Managers are willing to take risks with new technology but unwilling to do so when it comes to changes in how the organization does its work.

Second, the financial rewards for delivering higher-quality software at less cost are difficult to recognize. Suppose the annual software development budget for an IT organization is $2 million. They implement proven requirements validation and testing solutions and remove $500,000 in testing and rework costs (a typical result). How does that “profit” benefit the IT organization? Better salaries? Probably not. More likely, it translates into additional projects or goes to mitigate our chronic tendency to under-estimate software development costs in the first place.

There is, however, an overriding incentive to recognize the elephant: customer satisfaction. When applications and releases are routinely delivered on time and with few, if any, functional defects, the business customer is spared a tremendous amount of frustration and anxiety, not to mention cost. The trust between the business customer and IT goes up substantially. In these times of budget austerity and limited investment, this can be critical to the success of the CIO and the business as a whole. So pack up the elephant gun and let’s go on a safari!