The Dangerous Necessity of Relying on Bots
Algorithms and AI can help a company achieve massive scale, but they bring with them new risks.
For businesses, scale isn’t everything. It’s the only thing.
Scale confers massive advantages. When you’re manufacturing physical goods, being able to move more materials through an assembly line reduces the unit cost. When you’re serving customers at a burger chain, having 20,000 outlets confers a huge cost advantage over a competitor with 200 units — you reach lots more people and make far more effective use of supply chains, advertising, and other investments. In a highly competitive industry with razor-thin margins, scale is what enables you to grow profits. And for an online or digitized business for which the cost of adding or serving a new customer is close to zero, scale is a necessity.
From an economic perspective, everybody benefits from scale, which leads to greater efficiency, convenience, and choice for consumers. Today, you can summon virtually any consumer good, watch any movie, trade any stock from the comfort and privacy of your own home, cheaply and at the click of a button.
But there are some indications that scale has its limits and poses significant challenges — to the people running those businesses, and to society at large.
There are, of course, physical barriers to scale. The world contains a finite number of people who can sign up to a social media platform or afford a cup of gourmet coffee. And once you’ve reached all of them, you start to lose some of the benefits of scale. The scientist Geoffrey West, profiled so well in these pages by Lawrence Fisher in 2015, notes that there are systemic barriers to scale. That is, the laws of physics and biology dictate that organisms can only get so big and live so long — and this logic, West has found, applies equally to companies.
But we’re also learning that there are practical barriers to operating tech entities that have massive scale. Customer service is a tough discipline for a small business to master. It’s exponentially harder to do when you have hundreds of millions of customers with myriad grievances. Sure, AI and bots can help in managing billing and assessing complaints. But setting things right often requires human intervention. What’s more, the greater the scale of your business, the more likely it is to attract the attention of regulators, anti-trust officials, and hackers. Hackers stand to collect a much bigger payoff for busting into a fortified system containing 200 million accounts than for picking on a system that is smaller and more vulnerable.
There are some indications that scale has its limits and poses significant challenges.
There’s another issue. The only way you can run complex systems with hundreds of millions of users, customers, and potential advertisers is to turn over a large part (sometimes all) of the operations to algorithms, automation, and software. There’s no way you could profitably run such a system with humans overseeing every account opening, every posting of content, every ad purchase. There’s no way Amazon could function if a person had to ring up every sale and physically swipe a credit card every time a purchase is made. And so it is left to artificial intelligence to run the show, to determine on massive global social media sites what type of content is offensive or troubling, to decide which ad purchases it should accept. Although that’s not problematic in the overwhelming majority of instances, it takes only a small number of outliers to expose the difficulties inherent in operating at a gargantuan scale. And those instances tend to lend force to those who would like to limit the size and influence of those large-scale organizations.
Pointing out that our artificial intelligence needs more emotional intelligence is another way of saying that we need human judgment and sensibility to intercede more frequently in the hundreds of millions of transactions and decisions that get made every day. But doing so simply isn’t possible when organizations operate at the scale and speed that they do today.