Bottom Line: Asking online communities for help with data analysis comes with many potential benefits and pitfalls for companies. One particularly successful crowdsourcing campaign used a contest, geared toward scientists, to predict the future.
A few years ago, leading U.K. retailer Tesco wanted to better predict its customers’ shopping patterns. The grocer already tracked which stores consumers visited, what they bought, and what their preferred payment method was, through its loyalty card program. But it was stuck on how best to crunch all these numbers and turn them into usable data. Tesco decided to take the problem to the people—a bunch of really, really smart people—and came up with a solution that exceeded its own benchmark models by more than 100 percent.
This type of crowdsourcing—where companies eschew their own in-house data mining or analysis techniques in favor of inviting large groups of people outside the firm to weigh in—has become more common in recent years. As the Internet increasingly enables rapid exchanges of information, firms have begun embracing the concept that making sense of big data occasionally requires the help of external experts and their fresh perspectives, untainted by internal biases, entrenched viewpoints, or conflicts of interest between corporate divisions.
But according to a new study of Tesco’s crowdsourcing effort, pulling off a successful campaign isn’t easy, and can quickly become a drain on money, resources, and reputation if companies don’t follow a few guidelines. Namely, they must find the right audience to consider their problem, motivate this crowd in ways that produce the best answers, and remain in the good graces of customers who have grown weary of data breaches and other threats to their personal information. If companies can meet these conditions, they can exploit crowdsourcing to facilitate innovation, provide more relevant solutions to their customers, and even look into the future, the authors write.
Tesco framed its campaign as a contest, posting its problem and proprietary data through an analytics firm and dedicated website. A network of scientists continually on the hunt for real-world data to test their techniques had access to the details of every grocery store visit made by 100,000 customers during a yearlong period. Participants were asked to predict the date and purchase amount of each customer’s next visit.
The true goal, of course, was to develop a predictive modeling method—one unknown to ex-employees and therefore not easily replicated at other firms—that could be used on an even wider scale than just this subset of shoppers. One of the presumed advantages of crowdsourcing is that it can attract the attention of researchers with widely varied skill sets and professional backgrounds. In this case, 537 different participants submitted more than 2,000 different models. The winner was a statistics professor in Moscow; the runner-up was an actuary from Indiana.
Extensive interviews with executives involved in the competition revealed some of the keys to its success. The first lesson: Get some help. Companies that try to build and maintain their own online communities face a needless hurdle, the authors suggest, and they should use one of the many established crowdsourcing platforms to reach a wide crowd with varied analytical abilities.
The right type of motivation is also crucial. The Tesco project utilized a live leaderboard that allowed participants to compare their results (but not their actual algorithms), thereby encouraging the scientists to continually update and revise their models in an attempt to vault up the standings. The aesthetics of the contest’s web pages and the ability to interact with others trying to solve the problems also contributed to an enjoyable challenge that people wanted to circle back to.
Money, however, proved far less tantalizing than the authors had initially assumed it would be. The prize money was divided among the top three submissions to discourage a winner-takes-all mentality and ensure that the company could gain the rights to three models instead of just one. But scientists, it turns out, care more about competition than cash, and the presence of a real-time leaderboard was cited as the most motivating influence in their continued efforts. In fact, money acts as a demotivating factor if the crowd deems the reward inadequate in light of the company’s stature, the authors write, creating a “backlash against the competition that will result in fewer, lower quality submissions and less commitment from participants.”
Scientists, it turns out, care more about competition than cash.
But scientists aren’t the only group capable of unleashing a backlash. As Netflix and AOL have painfully learned in recent years, crowdsourcing can turn problematic when customers become concerned about the safety of their personal information. Fishing around for new ideas is one thing, but when businesses lay bare their customers’ data to inform analysts, they risk losing the trust of the very people they’re hoping to better serve. For this reason, companies must be careful when crafting their crowdsourcing campaigns, weighing the benefits of launching an open competition using real data against the perils of alienating consumers. Private or limited contests can provide a way around this concern, but won’t produce as many diverse ideas.
The authors also advise companies to keep their efforts tightly focused, avoiding open-ended or sprawling tasks. By its very definition, crowdsourcing should elicit experimentation, trial and error, and several different solutions. In the Tesco example, a relatively straightforward question—when will customers come back, and how much will they spend?—encouraged scientists to refine and elaborate on their predictive models, and they quickly and clearly reached the limits of what was possible to analyze given the data set.
Finally, companies can’t be so focused on external opinions that they neglect internal morale. Aside from questions over data protection and commercially sensitive information, companies must consider the effects on their own analysts when looking to outsiders for advice. In the Tesco project, managers ensured that staff members would have a prominent role in analyzing the results and choosing the winning models.
It can also be difficult to integrate business models devised by people with little knowledge of the company’s internal structure. For this reason, the authors advise companies to follow the example of the Tesco campaign, which centered on a non-critical business issue that enabled the firm to test the crowdsourcing approach without causing upheaval in its day-to-day operations.
Source: The Wisdom of Crowds: The Potential of Online Communities as a Tool for Data Analysis, by Marian Garcia Martinez and Bryn Walton (both Kent Business School), Technovation, Apr. 2014, vol. 34, no. 4