On the cultural side, data warehouses push marketing and other management professionals into new roles, and often force changes in the relationships between vendors and buyers.
For the technologist, "it's not only uncharted territory, it's uncomfortable territory,'' said Larry Greenfield, president of LGI Systems Inc., a data warehouse consulting firm based in Northfield, Ill. "There's a lot of ambiguity in the roles of the information systems people when you start giving end users data warehouse tools. Are you a developer, a cheerleader or a tutor? It's a bit of a puzzle.''
For the business manager, "the upfront cost for developing the data warehouse can be huge, with an unknown return,'' said Jim Schroer, a lead partner in Booz-Allen & Hamilton's consumer products and retail group. And when two organizations must cooperate on a warehouse project, "there's a lot of fear that the benefits will accrue to the other party.''
It sounds self-evident, but it's worth noting that data warehouse returns will vary depending on the purpose for which a system was created. The warehouses can be used to improve the effectiveness of an existing business process, to change a process or to foster a new process. In many cases, traditional measures such as return on investment may not be meaningful.
The earliest adopters of data warehouse technology were in consumer packaged goods and retail, driven by the narrow profit margins and fierce competition that have always characterized those businesses. Companies like Procter & Gamble, Wal-Mart and PepsiCo's Frito-Lay began implementing data warehouses before the term existed, often with highly specialized systems costing in the tens of millions of dollars. Now that data warehouses can be built with standard hardware and software, at an average cost below $3 million, their use is spreading to many more packaged goods and retail organizations. The most common use, as at the Longs drug store chain, is in category management, which is the fine-tuning of product placement, purchasing and promotion to maximize sales and profits.
Retailers like Longs have always had to manage myriad products in a highly competitive environment, but that's a relatively recent phenomenon for industries like telecommunications and financial services. Telco's now want to sell Internet access or paging along with basic long-distance services, and banks want to augment the meager returns on checking or passbook accounts by offering mutual funds and financial planning. Both groups need to work harder than ever to retain customers. To better understand their customers, and to pitch the right additional products to the most receptive potential buyers, these businesses are rapidly turning to data warehouses.
"When customers buy more than one product, the churn rate goes way down,'' said Lance Boxer, senior vice president and chief information officer at the MCI Communications Corporation, the second-largest long-distance provider, which began building a data warehouse in 1993 using software from the Informix Corporation. "We keep very detailed records on our buyers, so that we can sell them contiguous products,'' Mr. Boxer said. Thus, if analysis shows that customers with pagers are most likely also to want Internet service, MCI's sales representatives will concentrate calls on those buyers.
The system also allows MCI to respond quickly to competitive threats, according to Mr. Boxer. "If AT&T one morning were to announce a campaign for international callers, we could go in very quickly and start scoring people in our database who were likely candidates, and the marketing people could put together a response very quickly,'' he said. The data warehouse lets MCI avoid pitching customers who change carriers often, and who thus generate little profit, and instead target those likely to buy special services, who generate a lot. The system stores nearly 3 terabytes of data, or 3 billion bytes, enough to hold more than 100 million customer records.