Mind and Iron: The AI future of football
What Super Bowl LXVIII could look like. And...musical mind control?
Hi and welcome back to Mind and Iron. I'm Steve Zeitchik, veteran of The Washington Post and Los Angeles Times and Trevor Noah of this techie Grammys.
Every Thursday here at Mind and Iron we bring you a whole bunch of news and insight about our future world — what AI and innovation have in store for us, for good and for ill. Our mission is critical, but it is alas not free. Please consider supporting it.
This is Super Bowl week, so we’ll bite on some of the recreational themes of our national holiday.
Sports can be both an intensely human and highly analytic pursuit. Which makes it ripe for the machine vs. human age we are entering. Until recently, the humans have won — it’s been all mind, no iron. But tech is starting to bring the pressure. We’ll look at the coming coaching machine-outsource.
The halftime show will also serenade us with the sound of Usher (and the sight of Taylor Swift?) So we’ll take a moment to examine music’s AI future — one in which the tech could go from just enabling sales to shaping creativity itself. A look at the effort to engineer songs that can nestle right into our brains.
And finally, the AI that can (gulp) learn like a baby learns.
First, our future-world quote of the week. It comes from one of those people bringing science to music.
“Some people get goosebumps, chills, have an elevated heart rate…When you get it, it feels like a magical thing.”
—Eric Dubowsky, a music producer who has created “Frisson Trigger,” a song bioengineered to make us feel
Let's get to the messy business of building the future.
IronSupplement
Everything you do — and don’t — need to know in future-world this week
Future Super Bowl robot coaches; building the perfect musical beast; an AI that can learn like a baby?
1. FROM JOE NAMATH TO BROCK PURDY, sports generally and the NFL in particular have always at heart been a people story. Players gritting it out in the name of our allegiance is, on many days, what makes this dark woebegone existence worthwhile. Messy humanity compels us, and we’d no sooner strip it from our sports than we would dehydrate the flavor from our meals. Sports is a microcosm of the struggle and hope we all face in our daily lives, and long may it live.
In one sense, none of that should change anytime soon. Player execution on the field hasn’t evolved much since that first Super Bowl when Bart Starr was dropping in completions to those Packers receivers — nor will it.
And yet every decision happening out-of-sight behind them — what plays are run, what philosophies are employed, what players are even on the field in the first place — has been transformed since that 1967 day. And it could be about to change a whole lot more.
First things first. AI is going to bleed into games more than douse it. Consider the history: The notion of machine-led statistical analysis overriding human decisions was a non-starter for many decades of mainstream American sports, entering only in the early 2000’s when a desperate Oakland A’s GM named Billy Beane began making decisions on the basis of what a computer-model type named Paul DePodesta told him. (“Moneyball,” a practically holy tome.)
The idea of deploying this brand of computer thinking in real-time on the diamond naturally followed (sign a guy who walks a lot and you’ll end up taking a lot of pitches). It spread at varying rates to different sports after that but really only came to football, with its exponentially larger number of players and variables, in the last decade. That’s when we started seeing coaches make decisions about fourth downs and two-point conversions based on the machine-crunched statistical evidence, not cliches and butt-protecting.
And even then it’s been selective. Plenty of NFL coaches, like Bill Belichick, still don’t love listening to a computer in the middle of the game. Sure, there’s less resistance in the front office, with advanced analytics factoring into personnel decisions, but pushback abides even there. The most important NFL player choices every year are made at draft combines, where humans take a stopwatch and judge other humans based on how they see size and speed, not a microprocessor in sight.
Yet the seeping-in is happening. Here’s a sample of some of the machine-led changes occurring right now in various corners of football.
—A Northwestern engineering major named Michael McRoberts a decade ago put together a company called Championship Analytics that basically has a computer tell a coach what to do in ambiguous situations. No one listened at first. Now he has nearly 200 clients across every level. (The majority of college programs that played in conference championship games last season were McRoberts clients.) There’s a limit to how much coaches defer to his system; they may listen to it on the simple decision of going for it on 4th down but not on what to play to call when they do. But copycats are springing up, and the infiltration is growing.
—The forward-thinking Jacksonville Jaguars, which already has used AI in everything from stadium design to game-week marketing, has a deal with a digital company named Tech Mahindra to serve as its technology and analytics and digital strategy partner. Expect more such deals, and more influence over on-field decision-making.
—In (overseas) football, a company named Soccerment embeds microchips in shin guards that, as the game progresses, monitor everything from the speed of a player to the strength of a ball-strike. Then it radios that back to coaches on the sideline, who can understand their players’ performance at a level the human eye cannot process. The NFL itself has teamed with Amazon to process player info and experience in real-time. For the moment, it’s mainly for health-and-safety purposes. But the system is easily ported to coaching.
—Success breeds success. The three teams that have used analytics the most in the past several years, according to a survey last year, are the Browns, Ravens and Eagles. All three made the playoffs this season. The Ravens were the No. 1 seed. The Eagles made the Super Bowl last season. The Browns GM has a Masters in computer science. Oh, and DePodesta is the club’s chief strategy officer. The NFL, as it is often said, is a copycat league. So expect more like them.
Most of these of course are incremental shifts, not revolutionary ones. But in a league so disinclined toward change, they’re notable. Increments are how revolutions happen. Sports-minded AI types talk about a day not far off when coaches feed info into models right in the middle of practices and even — the holy grail — in real-time game situations.
According to this vision, the laminated takeout menus that currently fill the hands of NFL head coaches with all those plays and scenarios (a static approach that sends shudders down the spines of quant types; I mean, would you conduct a million-dollar stock trade based on what Gordon Gekko wrote on a Post-It?) will be replaced by digital devices that are constantly updating with the best packages and plays.
Because for all its balletic beauty and sophisticated strategy, football, like most other complex systems, is at bottom a set of data points, and it can be broken down accordingly. (It kills me as an NFL romantic to say that. But it’s true.) We’ll see how soon teams embrace that.
Of course, even if you could get past coach intractability, such systems are still much easier to imagine than implement. One big question is how much platforms can be updated in real time. AI models take a long time to train, but a player can tweak an ankle and wash out on the next play.
And a football game is filled with intangibles — a coach sensing a player has lost his confidence; crowd noise that’s going to make a long snap count more offsides-vulnerable — that are harder to program. Then again, no one is saying get rid of the coach. The idea is just to let the AI have first crack at it.
There’s no question a model could enhance play-calling and package-alignment — that is to say, there’s no doubt that an AI could be, at the very least, a solid coordinator, weighing in on crucial plays. Thinking about a machine omnisciently spitting out plays instead of stoic men covering their mouths is a big stretch. Coaches glancing at said machine or being fed what it says in their headset? Less big.
As fans, these changes feel a little alarming, a dehumanized vision lifted right out of Fritz Lang’s “Metropolis.” Is nothing safely out of Silicon Valley’s clutches, we find ourselves asking, that we need to turn football coaching into a giant algorithm too? As one fan said in a recent Reddit thread on the topic, “taking this idea to the extreme, we hypothetically end up with one team sponsored by (e.g.) Amazon AI, and the other by Microsoft AI, and suddenly it's a technological optimization challenge just as much as a human sport.”
Well said. And scarily said. The idea of football as a kind of reverse BattleBots — in which instead of humans controlling the machines we have the reverse — doesn't exactly enthrall. Fortunately, there are too many humans with too many gut feelings (and dollars) invested for this to ever get that far. Also, players still need coaches to, well coach. And while it’s at least conceivable play-calling gets more mechanized, computers will never be running two-a-days and motivating linemen. Machine Vince Lombardi isn’t Vince Lombardi at all.
Maybe even more reassuring, I actually think this machine handoff could encourage more spontaneous playcalling. AI thrives on predictability — it learns how to adapt by seeing the same situation over and over again — which means when you’re playing against an AI coach you want to be as unpredictable as possible; otherwise the opposing coach(‘s AI) will have you figured out quickly. Basically you want to be the machine equivalent of Bill Belichick, bobbing when they think you’ll weave. (Perhaps one reason he’s one of the coaches so opposed to machine-learning — he’s already doing it himself.)
Defenses might think twice about using that exotic blitz on third-and-long because the AI is expecting it — but then the AI, knowing that the defense wants to outsmart it, will conclude the defense won’t use it and so won’t expect it…prompting the defense to use it. We’re in Vizzini-in-Princess Bride territory here.
Technological change happens gradually in sports. No NFL coach ever thought to use game film until 25 years into the professional sport, and it took more than a half-century before the idea of a specialist radioing in plays from the sideline started to become common. But it does come. And when it happens it tends to take hold, the same resistant groupthink now pivoting to embrace it. Because ultimately the one tendency that trumps even grumpy conservatism in professional sports is competitive edge. The most Luddite curmudgeon could be open to AI’s Siren song when rival teams start singing it.
Will this machine-led sports future make the game less fun? Like so many American professions, it might not be as much fun to work in, less a pant-seat human enterprise than a factory-machine operator deciding which outputs to discard. But the question most of us have is more personal: will it make the game less fun to watch?
For all the ways AI will reduce the game to math, I can’t help feeling like this era will actually bring a new kind of joy. Sure, without the human subjectivity, the games, at first glance, could be less pleasurable. Yelling about the wrongness of the human who happens to be wearing your team’s headset is one of the sports fan’s purest releases; lamenting a machine’s decision just lacks the same oomph. Hollering about Dan Campbell’s fourth-down decisions is fun because he’s a real person with a mix of swagger and remorse. I don’t want to throw my stein at an image of computer code on the TV.
But there will be humans executing the computer-code’s wishes — which means there will be humans deciding whether to override it. Imagines the AI systems becoming not just more employed by coaches but more transparent to us fans. Imagine a TV broadcast ranking the kind of play a coach should call by order of probability of success — the 3rd-and-four from midfield against this undersized defensive line is generating a call of a halfback draw at 63 percent, a jet sweep at 26 percent and a screen out to the tight end at 11 percent.
And now imagine a coach calling for a sideline bomb.
We’ll get half the fan base hollering about how smart he is not to trust the computer and the other half saying who does this ham-and-egger think he is, convinced he can out-think a machine that can make 500 trillion calculations per second?
We will argue, in essence, whether machines or humans can do the job better. In that sense sports will become the truest microcosm, and long may it live.
2. EVER SINCE IT CAME OUT LAST WEEK, I’VE BEEN LISTENING OBSESSIVELY TO BILLY JOEL’S “TURN THE LIGHTS BACK ON.”
The song has wormed its way into my head. When the track finishes playing, I ask Alexa to play it again. Listening to it fills me with all those pangs of hope and regret evinced by its singer — giving me both an emotional fix and a craving to hear it constantly. It’s a musical mini-addiction.
(Other pop songs that have done this to me over the years: Taio Cruz’s “Dynamite,” College and Electric Youth’s “Hero” from the “Drive” soundtrack, Chumbawamba’s “Tubthumping,” Charlie Puth and Wiz Khalifa’s ‘See You Again” and Fountains of Wayne’s “Valley Winter Song.” My therapist is also confounded.)
What is happening for me with these songs is what scientists call “frisson” — the idea that the auditory cortex, in response to a particular piece of music, sends signals to the emotion-controlling parts of our brains. (You might have heard of ASMR, the relaxed feeling one gets from a Billie Eilish or Sigur Ros song, that is similarly biological.)
Researchers don’t fully understand how this works — “the neural link between sensory experiences and pleasurable aesthetic responses remains unclear,” Harvard researchers concluded in 2016 — just that it does. When Usher plays on Sunday he might do the same for you; when you hear “Yeah!” or “Climax” or “Think of You” it will press all the biological buttons that make you feel.
(By the way, this is not the same phenomenon as an old song that fills you with emotion because of its association with past experiences; this is music that triggers biological responses simply by dint of its notes and arrangement. Also, it appears that only half of us are capable of it. We should pity the other half. It must be like having no sense of smell at Hersheypark.)
Given this, a question pops up: can frisson be engineered?
That is, if a piece of creativity brings on emotion by way of biology, can’t we just build the biology right into the piece of creativity?
Now, before you go all get-outta-here-with-that-robot-nonsense, music-isn’t-science (as I first did), realize that this already happens more than we think. The biggest producers, especially of the tech-enhanced pop that fills our ears so much of the time, rely on patterns that are proven to have effects on our brains. (See this video on how Swedish uber-producer Max Martin does this, given by a guy who looks weirdly like Max Martin.)
The issue now is whether someone can go step a further and build a song entirely on the basis of that biology — in other words, can optimize music for frisson.
One recent effort comes from a producer named Eric Dubowsky. At the behest of audio giant Sonos, Dubowsky created something called “Frisson Trigger” that basically reverse-engineers the science into a piece of music. (You can listen to the five-and-a-half minute track on Apple Music. )
Here’s Dubowsky talking a little about how he did it — by engineering melodies for maximum tension and release, e.g.
“To me music really is all about creating emotion,” he says. “[So the] piece “was engineered to create frisson in as many people as possible.”
His goal, he adds, is those goosebumps and elevated heart rates. “I hope that people feel a physical connection to music,” he says.
The track is marketed with an even bigger boast: “Frisson Trigger, a groundbreaking track engineered to give listeners chills.”
Which might make us roll our eyes. Even if it does that, so what? You can screen a horror film in a hot room and give moviegoers sweats. That doesn’t mean it’s a scary movie.
Still, I can’t help feeling like Dubowsky and Sonos are on to something. The history of music is one of technology’s steady march. Dylan going electric at Newport 1965 was a shot fired in a technological revolution, while the autotuning and sampling of this century has injected the digital right into the music. Why couldn’t new forms of biological understanding also drive music-creation?
Incidentally, the reverse — the usage of biology to undo musical consumption — is something some of us are already familiar with via the well-researched trick of chewing gum to rid ourselves of an earworm; that’s about hijacking neural pathways for a different purpose.
The Sonos track didn’t do much for me. Maybe because I’m a weird dude who likes Taio Cruz and Fountains of Wayne. Or because it may not work. But there’s a third possibility, and it’s what makes this field so promising but also so fraught. It’s that frisson could vary by listener. That is, though fully half of us may be able to experience frisson in some manner, the actual ways we get there can be far from unified. In this regards musical frisson works like any other drug. Some people react to it; some people don’t.
So should we want to engage in frisson engineering? I’d understand if you said no. Music for most of human existence has functioned more like a dish than a drug; we try different combinations for what tastes good, not engineer different compounds for what’s most effective. Perhaps that should continue. (We probably should also look at the health consequences of mass-designing music for brain responses.)
But that doesn’t mean some artists won’t try to follow this budding science. And it doesn’t mean listeners won’t like it.
The creation of music will always be personal, instinctive, ineffable. But there’s no denying it is made for humans who are also biological, psychiatric, chemical. Until now we’ve largely ignored the latter in favor of the former. Don’t be surprised if a new niche emerges changing that tune.
3. JUST BECAUSE IT’S INTERESTING — NOT BECAUSE BIG TECH IS TRYING TO DO ANYTHING WITH IT (YET) — CHECK OUT THIS STUDY PUBLISHED IN THE JOURNAL SCIENCE LAST WEEK.
It involves how an AI was taught to learn language like a human baby, with the many thrilling/frightening implications thereof.
Most AI learns by training on pure data — you show it a thousand different images of an apple and punish/reward it until it can recognize one.
But this technique doesn't mimic how humans learn in the real world. No one "trains" a child on a thousand objects. You just speak and eventually it picks up meaning from context.
So these researchers from NYU tried to train an AI system by replicating that. They put a camera on a baby a few hours per week the first couple years of its life. Then they fed the AI the words and images from the footage — no training, just what the baby saw and heard puttering around the floor with its parents nearby. (Nature has a good summary of the study.)
Surprisingly, the AI seemed able to learn language this way. In a subsequent four-choice test of tens of thousands of words, the AI got them right 62 percent of the time, a lot higher than guessing and about what's comparable with more traditional AI training.
The researchers here focused on what this teaches us about language acquisition in humans (there are all sorts of implications). But it’s not a big leap to see the implications for AI too. Because if the system doesn’t need a separate formal training system to understand — if it can just “experience” the world like a baby — our whole notion of how an AI gets smarter changes. It goes from something that has to be taught to, well, more like a human — just sponging up meaning from being around people.
That sounds a little terrifying and sentient-y. It doesn’t have to be. It could simply be a means to make AI more adaptive in real time. The dream of natural-language conversation with AI assistants — as opposed to the stilted and misunderstood communiques with Siri — could get closer, for instance.
These are very early research underpinnings, not actionable applications. Still. AI has long had to be formally taught before it knows something. Now hints are coming it can understand simply by hanging around. We’ll see what it picks up when it does.
4. SUPER BOWL ADS IN RECENT YEARS HAVE BECOME A KIND OF BAROMETER OF OUR TECH-Y FUTURE.
We’ll wait until next week to see what’s worth weighing in on. (Fortunately no crypto ads bouncing around like a DVD screensaver seem to await this year, nor similarly sketchy spots for a firm whose AI is absolutely one-hundred-percent guaranteed to help you increase your sales.)
But I am intrigued by OpenDoor, a company that will do a live virtual house-showing during halftime — not for the 2020-era housing froth, but for how effectively/ineffectively it will demonstrate telepresence, the idea that we can be there without being there.
So keep an eye out for that and anything that marketers want to promise; they’re usually a good indicator of the kind of future we want to hear about (or, they think we want to hear about).
In the meantime please enjoy this 2022 Super Bowl throwback, when Larry David tried to warn us about crypto (while also getting paid to endorse it). “It’s far. It’s too far!”
The Mind and Iron Totally Scientific Apocalypse Score
Every week we bring you the TSAS — the TOTALLY SCIENTIFIC APOCALYPSE SCORE (tm). It’s a barometer of the biggest future-world news of the week, from a sink-to-our-doom -5 or -6 to a life-is-great +5 or +6 the other way. Last year ended with a score of -21.5 — gulp. But it’s a new year, so we’re starting fresh — a big, welcoming zero to kick off 2024. Let’s hope it gets into (and stays) in plus territory for a long while to come.
AI FOOTBALL COACHES: Get me the popcorn. +2.5
MUSIC BIOENGINEERED TO MAKE US FEEL: Just as long as it doesn’t take over all the other music. +1
AI CAN LEARN LIKE A BABY: A notable breakthrough, but the jury’s out on whether this will improve our world or warp it. So call it a wash for now. 0
Steve does a wonderful job in explaining complicated matters in a simple manner in a simple manner, so everyone can appreciate his great ideas