After five years of retrenchment and cost cutting, senior executives across a variety of industries share the conviction that innovation — the ability to define and create new products and services and quickly bring them to market — is an increasingly important source of competitive advantage. Booz Allen Hamilton surveyed 50 companies in early 2004, and more than 90 percent of top managers at those firms, in fields such as aerospace, automotive products, pharmaceuticals, and telecommunications, said innovation was critical to achieving their strategic objectives. Indeed, for the next two years alone, they are setting aggressive performance goals for their innovation and product-development organizations, targeting 20 to 30 percent improvements in such areas as customer value, time-to-market, development cost, product cost, and quality.

But a vast disconnect lies between hope and reality. Our survey shows that companies are only marginally satisfied that their current innovation organizations are delivering their full potential. Worse, executives say that only half the improvement efforts they launch end up meeting expectations.

Several waves of improvements in innovation and product development have already substantially enhanced companies’ ability to deliver differentiated, higher-quality products to market faster and more efficiently. However, the degree of success achieved has varied greatly among companies and even among units within individual companies. The survey confirmed what we have observed in our consulting work: The differences in success stem from the difficulty of managing change in the complex processes and organizations associated with innovation and product development. The survey also suggested that four factors make or break innovation programs.

Senior leadership support. Clear and unwavering support from senior management ranks first among factors that determine a company’s ability to effect complex change. By a two-to-one margin over other considerations, competing internal priorities and insufficient time and resources — issues that can be resolved only with top management guidance — were cited as the chief barriers to achieving improved innovation and product-development performance.

Continuous improvement. The survey suggests that a company’s overall approach to innovation, rather than the amount of time and money it spends on process improvements, correlates with the results obtained. Companies with an established process for continuous improvement coupled with periodic transformational initiatives had the best overall results and the highest level of satisfaction in the progress of their innovation initiatives. Still, some 40 percent of companies either have no formal process for improving performance or describe their improvement efforts as opportunistic or ad hoc.

Organization as enabler. Different models for innovation programs work effectively for different companies in different industries. But, although variety may be the spice of life, there is a downside: Since there is no standard innovation-organization model, many companies end up changing models frequently in search of better performance. Fifty-five percent of the companies surveyed said they had reorganized their innovation organizations during the prior two years.

The traditional “functional” organization model for innovation was aligned according to activities, such as R&D, marketing, and operations. But this model’s inherent weaknesses — it hinders customer understanding and time-to-market — generated a mass migration toward “product-focused” organizations, in which resources from relevant functions are assigned to a specific product or product group. Realizing, however, that with the product focus they risk losing technical excellence and functional engineering standards, many companies have begun to use a third model. The overall trend now seems to favor program-management organizations. These combine strengths from the functional and product-focused models; personnel increasingly are reporting both to a functional head (for standard methods as well as skills and career development) and to a program manager who has significant authority for a specific program’s resource allocation. Companies operating with a heavyweight program-management organization tend to be more satisfied that their innovation organization is fostering creativity, efficiency, and continuous improvement.