An advertising agency met with a client—who happened to be a U.S. Marine Corps colonel—and the conversation turned to the topic of reliable data. “Look,” said the colonel, “if I’m on a battlefield trying to defend a hill and I get a piece of intelligence, even if I’m not 100 percent sure that it’s accurate, I will make decisions based on that intelligence.” He strongly believed that it’s better to have some information than none—and that you’d be a fool to disregard it just because it falls short of being definitive. One could say that the colonel was a proponent of “little data.”
There is, of course, a great deal of discussion about the potential of “big data,” the high-volume, high-velocity, high-variety information assets that require new forms of data processing to enable companies to make better decisions and operate more efficiently. Giant data sets are being created by aggregates of individuals’ behavior (on social media sites such as Twitter and Instagram, for example), by transaction logs, and by automated information-sensing devices. Companies are increasingly mining these data sources to understand more about their customers’ behavior and preferences, and even to anticipate stock market movements. Early successes by a few companies have caused others to start investing in the infrastructure, software, and talent required to mine big data.
There is, however, one important caveat. Many companies—probably most—work in relatively sparse data environments, without access to the abundant information needed for advanced analytics and data mining. For instance, point-of-sale register data is not standard in emerging markets. In most B2B industries, companies have access to their own sales and shipment data but have little visibility into overall market volumes or what their competitors are selling. Highly specialized or concentrated markets, such as parts suppliers to automakers, have only a handful of potential customers. These companies have to be content with what might be called little data—readily available information that companies can use to generate insights, even if it is sparse or of uneven quality. For these companies, the U.S. Marine colonel’s words will resonate more than the latest data-mining algorithm or social listening platform.
Several commentators have made the point that the implications of big data go beyond new data sources, analytical techniques, and technology. Rather, a paradigm shift—away from management based on gut feelings and toward data-driven decision making—is already under way, and accelerating. The shift is so profound that companies lacking complete or clean market data can no longer use this deficit as an excuse to rely on the status quo. They must make a concerted effort to use the data that is available to them (imperfect as it may be) or to explore innovative, low-cost ways to create new data.
Companies lacking complete or clean data can’t use that as an excuse to rely on the status quo.
In one example, a large beverage manufacturer wanted to improve its sales to bars, restaurants, and entertainment venues. For years, this company had been buying syndicated data from an established source, which covered more than 100,000 establishments. Unfortunately, the data was collected and structured to serve a broad set of clients and featured a standard segmentation scheme that did not provide enough insight for the beverage company into how to serve different segments. So the company decided to adopt a series of little data techniques to come up with a solution customized to its needs.
It started with observational research, visiting bars and restaurants and qualitatively cataloging the clientele and their consumption patterns. Synthesizing this information resulted in more actionable segment definitions. The next step was to quantify the segmentation—determining how many establishments were in each segment. The beverage manufacturer developed an algorithm based on observable characteristics, then asked its sales professionals to classify all the bars and restaurants in their territories based on the algorithm. (This is a classic little data technique: filling in the data gaps internally.) Finally, for each major segment, the company designed tailored product assortments, pricing, and marketing programs. Pilot projects in two large cities have shown significant lifts in total sales and share penetration, and the company is now rolling out the initiative nationwide.