AI in the announcer’s booth
Artificial intelligence could soon deliver sports commentary customized to a viewer’s preferences. And if robots can learn to do this sort of talking, the applications could be far-reaching.
Photograph by krisanapong detraphiphat
I like to watch rugby, even though I know very little about it. The commentators don’t help. They rightfully believe they’re talking to people who watch rugby a lot, so they feel no need to address me, personally, with rugby-for-dummies spiels that might give me an appreciation for the game.
But emerging technology could soon solve my problem. Some companies are working on AI that will generate custom sports commentary, which means I could potentially tune into a streaming rugby game and listen to a human-sounding, AI-driven robot commentator that already understands my level of rugby savvy. Maybe my robot commentator will patiently explain the difference between a blood bin and a tight head.
The most interesting thing about emerging technology like this is that it allows us to reinvent stuff that we’ve long taken for granted. The autonomous car, for example, won’t just make driving more efficient — it will redefine the notion of personal transportation. And so it goes for sports commentary, which has implications well beyond sports.
Current sports broadcasts are a 20th-century leftover, a relic of an era dominated by economies of scale and mass production, when the way to rake in profits was to make the same thing for as many people as possible.
Sports broadcasts started in the 1920s on radio, and the commentators had to describe literally everything going on in the game. When sports moved to TV in 1939, so did commentators, because that’s what people were used to. Now we can’t imagine a televised game or match without all that chatter.
Yet we have to endure a single mass-market feed of commentary — one size fits all. The commentator doesn’t care about your individual interests. Maybe you’d rather hear much more about strategy, or be told details of interpersonal relationships. (“You see, Joe, this player hates that player, so I anticipate a fight.”) Well, too bad. You have to listen to the same thing everyone else listens to.
There hasn’t been an alternative, or the potential to profitably customize sports commentary for every viewer, until recently. Companies including IBM and Amazon have been working on using AI-powered robots to narrate personalized sports broadcasts. IBM’s technology will debut soon. And last year, Major League Baseball announced it is partnering with Amazon to develop AI-based technology that will “create analysis for live games in the tone and style of iconic announcers.”
What’s new here is the commentary and the ability to customize it. AI has been moving deeper into sports production over the past few years. It’s been used to make predictions during cricket matches in India and develop stats-based insights for Minor League Baseball. Moving from these sorts of AI applications to customizable commentary has been a progression. For instance, IBM’s path to AI commentary started in 2017 at Wimbledon. The tennis tournament has 675 matches over two weeks, many of them simultaneous. For producers, finding video of the most dramatic moments in every match and assembling highlights for fans is a nightmare. “No human production team can keep up,” says John Smith, the IBM scientist who is now guiding the AI commentary project. But in situations like these, AI can assist.
Developments in AI could have ripple effects. How long before a couple of laptops host the New Year’s Eve ball drop in Times Square?
To make AI smart about sports, researchers rely on an AI technology called end-to-end learning. Instead of getting 100 engineers to write algorithms to track every aspect of a game, end-to-end learning programs AI to learn on its own. The researchers set up the machine, then make it “watch” thousands of matches or games — the video and the commentary. Once it can comprehend what’s significant and what’s mundane, it can instantly put together highlights.
The next step in getting to commentary is adding natural language speech technology. Scientists have been writing papers about this for years. For instance, Greg Lee and Vadim Bulitko of the University of Alberta in Canada published “Automated storytelling in sports: A rich domain to be explored” in 2010. By now, the technology has advanced “to the point where we can feed it a game it hasn’t seen yet and it will start talking, describing what it sees,” Smith says.
It’s not perfect yet. Smith played me a clip of IBM’s commentary, and although the voices sound quite human and usually sound excited or critical at the right moments, weird glitches creep in. For instance, the soccer AI gets stuck on the term cross. “It says every long pass is a cross,” Smith says. Another challenge is that sports broadcast teams don’t just talk — they converse. Creating that kind of conversation is a complex and still unmet challenge.
Still, this is valid technology, and machine learning and natural language processing make it possible to “unscale” sports broadcasting — taking apart the mass-market approach and instead generating commentary personalized to every individual viewer. You like your soccer games to be announced by a woman with a French accent? You got it. Want to watch a Washington Capitals hockey game in which the only player the commentators talk about is Alex Ovechkin? (“Now he’s sitting on the bench drinking water. Oh, wait, he’s cursing the referee in Russian!”) Go for it.
Of course, such developments in AI could have ripple effects. What might happen to all those sports broadcasters’ jobs? And if AI can do this in sports, why would we need humans to tell us about any kind of activity, including the events covered in a political convention or a news story? How long before a couple of laptops host the New Year’s Eve ball drop in Times Square?
In fact, China’s Xinhua News Agency debuted the world’s first AI news anchor in 2018 and now has created another AI-powered avatar. And the conversations this technology can handle are only going to increase in complexity. This year, IBM unveiled new research, called Project Debater, that the company bills as “the first-ever AI technology to meaningfully engage in a live debate with a human.”
If AI gets good enough at debating, we could get to the point when we listen to commentary and no longer know if a human is even participating. We might tune into a sports talk radio station and end up listening to two machines arguing over whether the coach should get fired.
Or, if by then the coach is also a machine, get unplugged.
