[Ticking] Okay so it's 2004 and Serena Williams is playing Jennifer Capriati in the US Open. What you're about to see is a referee call so bad it changed a sport forever. So she hits the ball.
. . it bounces just inside the line.
. . the crowd cheers.
. . and then suddenly the umpire calls it out!
"Advantage. . .
" "Whaaaat? " "No way! That was way in.
" "That ball was so in! " "That's one of the worst calls I've seen. " And Serena's right.
You can clearly see on this review, the ball is in. But the umpire wasn't using that review system. Only the TV stations could see it.
The system is called Hawkeye and you can hear the commentators begging them to use it: "Oh no" "OH NO" "HAWKEYE PLEASE. " Within just a year from this controversial game, Hawkeye was added as an official part of the US Open review process, and today it's replaced over 200 of their human judges. And this whole robot ref revolution, it isn't just happening in tennis.
It's taking over the sports world: Soccer, basketball, baseball, they are all using machines to try to improve the calls being made on the field. The stakes here are high. The difference between a right and a wrong call could be a championship or millions of dollars in prizes or endorsements or ticket sales and for the most part fans like tech that makes calls more objective.
However, there is a big chunk of sports fans who would strongly disagree, who think that this tech is ruining the game, who believe that human error is key to sports. The thing is though, this tech isn't going back in the bottle. Stadiums are now being decked out with dozens of cameras that can track people and balls and even use AI to make calls so accurate that one day human refs might be a thing of the past.
. . "Footballers and coaches have complained about referees and lack of accuracy -" "The fans hate it" "I don't think people are enjoying football as much -" "- incredibly frustrated with the Hawkeye system" "YOU CANNOT BE SERIOUS" "The receiver did not maintain possession -" "Could Hawkeye technology replace humans?
" Okay, look at this: This is the ball from the most recent World Cup. It looks pretty normal but inside it's anything but. For the first time ever, hidden inside this ball are special sensors that collect positioning data in real time.
The sensors are hung on a suspension system so the ball doesn't actually feel any different to the players, all the while it's sending data 500 times a second back to a room filled with refs you never see. That room also receives data from 12 cameras mounted around the stadium that track 29 data points for each player on the field and can automatically alert the rest the moment a player is offsides, here meaning past the last defender. [Whistle] They called this the new semi-automated offsides technology and it uses that tracking data and AI to build a virtual 3D playing field so the refs can see the play from basically any angle.
This is the cutting edge of live sports tracking tech and it was featured in the World Cup with lots of hype. In this system, they're still human refs involved but they only need to validate the calls being made by these robots. That's why they call it "semi-automated.
" A ton of amazing engineering slo-mo footage. These systems work surprisingly well, like in the recent World Cup match between the US and Sweden, the video review system decided the game. Just watch this: [Cheering "Did it go in?
? ? " The ruling was too close to call with the human eye.
. . "Look, they're waiting.
. . " "They're going to check.
. . " With the help of the automated system, you can see that ball just barely makes it fully across the line.
And Sweden won. [Cheering] "Sweden wins! " "Wow, that is close!
" "That is the margin. . .
" These robot ref systems are clearly helping make better calls and taking more human error out of the game and while many see that as a good thing, some fans hate it. wouldn't you want a system that makes the right call every time? Because while people do voice concerns with specific calls with this automated tech, "it's not accurate" doesn't seem to be the main complaint.
Do people just like fighting with the ref? Well maybe. .
. "REF YOU SUCK" "REF YOU SUCK" "REF YOU SUCK! " I get the fun of getting all riled up.
But I also just don't think that I could look an athlete that I admire in the face and say, "we think this tech will judge you more fairly, but we don't want to use it because injustice for you is entertainment for me. " I don't think that's right. And besides if I think a call is wrong I can totally yell at a machine.
"We're a --- embarrassment, mate" "ABSOLUTE RUBBISH. RUBBISH. " It turns out though that wanting to argue with the ref isn't actually the biggest issue here either.
The main complaint is something that I kind of actually agree with: AI-powered robot ref systems don't understand the natural gray areas of sports. To really see this in action watch this clip: this is Leverkusen. .
. Leverkusen? Let me check that.
. . this is Leverkusen.
. . "Leverkusen" Leverkusen.
This is Leverkusen versus Club Brugge - "Club Brugge" Yeah I got that one right. In the second half of the game, Leverkusen scores. .
. "Ohhhhh it's all the way in! " But the play gets flagged by the semi-automated system because an inch of his toe was over the line.
Like just look at it. . .
You can hardly see this. . .
so the goal gets disallowed and the other team wins the game. [Cheering] [Angry tweet] [Many angry tweets] I get why people dislike this. The player probably reasonably believed that he was on sides.
With a more human system, we had a natural buffer because we couldn't be more accurate. Now that we can, we have to decide if we actually want to be or if we actually like the sort of human nuance. You totally could in theory build the buffer into an automated system but you'd have to admit that you want it first.
This is where it gets complicated because at the start of this story I was like, "yeah 100 accurate, that's the way to go" and then you give me one example and all of a sudden I'm feeling conflicted. But on the other hand not having precise consistent calls affects the game as well. Remember that tennis match with Serena Williams?
Here's what she had to say about it years later: "The reason Hawkeye became a thing was because they were calling my balls out and they weren't even close to the line and it became impossible to play. " We ask athletes to spend their whole lives training to perform better than anybody else and then give their all in front of us in every game [Tennis grunts] [Cheering] I think we owe it to them to use the tools that we can to judge them fairly and to try to do better at that all the time. Tennis is a good example of that effort going well.
Today the Hawkeye system at the US Open tracks the ball by using 12 cameras capturing 340 frames every second. For each frame, it identifies the ball, triangulates its position, and creates a combined model of exactly where it's been to predict where it's going. ".
. . OUT!
" Hawkeye boasts the margin of error less than two millimeters. The best study that I could find comparing human refs showed that the margin of error for line judges is around 40 millimeters. In tennis, it's gotten to the point where championships not using Hawkeyes to the fullest are the ones becoming controversial instead of the other way around.
And besides it's just fun. I really like seeing the Hawkeye replay and trying to figure out if I was right. And while tennis and soccer have really helped pioneer some of these new systems a lot of other sports are coming on board as well.
The NBA just announced that it's going to use Hawkeye tech to track the ball and the players in upcoming basketball games. Other sports like Minor League Baseball are already using it to let players challenge human calls. It's clear that we're entering a new world of refereeing and a new world of being a fan too.
With these tracking tools, what we're really doing is creating super accurate models of the world and those models give us a superpower: We can understand what's going on much better than we can with just our eyes. That superpower brings with it questions that extend far beyond sports like, "how much better does the technology need to be than humans for us to like it? " That's a question that we're exploring with technologies like self-driving cars and AI medical diagnoses.
Or "how precisely do we actually want our rules enforced? " Or "when do we actually want human judgment and why, especially if we can measure that it's less reliable? " But for all of the talk about tech replacing humans what struck me most about this story is that all of the cutting edge tech in sports is actually going into just measuring humans.
We're not interested in replacing the athletes with robots I mean a robot basketball game sounds fun but it's no NBA. Sports are fundamentally about pushing forward what humans can do. And all of this tech, it's just a new way to appreciate that.
