Mind and Iron: A machine just understood the words a person was thinking
A breakthrough too stunning to believe. Also, original summer fiction!
Hi and welcome back to another fine episode of Mind and Iron. I'm Steve Zeitchik, veteran of The Washington Post and Los Angeles Times and lead eldrazi of this magic gathering.
Every week we give you all the intel you need on how tech and AI are shaping our future — no filter or corporate agenda, just all the news and context you could want.
It's summer, so this week we're going to give you a little treat — summer fiction.
That's right, an original short story, on-brand with our themes of humanity and tech, delivered fresh to your door. Maybe you'll like it, maybe you won’t, maybe you'll think, ‘hey I can get on board with that kind of Substack.’ We’ll likely do it again before the summer’s out. This week’s tale has a young woman paying a visit to a mysterious shop and getting more than she bargained for.
Also, something crazy cool — a machine just plucked thoughts from an ALS patient’s mind and translated them into words, essentially restoring his ability to speak. The New England Journal of Medicine reported the results Wednesday, and the effect on a man and his daughter are astonishing.
Finally, a new series of deepfake scams involves Elon Musk. And it highlights an intractable problem. (Besides Musk.)
First, the future-world quote of the week:
“In a person with ALS and severe dysarthria, an intracortical speech neuroprosthesis reached a level of performance suitable to restore conversational communication after brief training.”
— Well I would have used snappier language. But the researchers in the New England Journal of Medicine are still saying something pretty cool: an ALS patient can now speak using only his thoughts.
Let's get to the messy business of building the future.
MindBendy
A short story about the long road ahead
One of our goals here at Mind and Iron is to give you a sense of where our world is headed. Sometimes that’s with a feature. Sometimes that’s with an interview. Sometimes that’s with a piece of analysis. And sometimes it’s with a piece of fiction.
The story below might feel like a surprise because, well, we’ve never run fiction in this space before. But the themes — how to pursue the very human need to make our lives better without sacrificing what makes us human in the first place — will be instantly recognizable. Give it a read and let us know what you think.
Gabirol
Inbal felt like visiting the file-reloader today. Though, if you wanted to be truthful, it was less a feeling than an imperative. And she couldn’t even say why – she simply woke up with the directive in her head. Like how you feel the urge to buy a pair of sneakers even when the thought of running repels you; you just know life would be paler without them. When you have so strong an impulse you need to act on it.
Anyway, she went to Solomon, who ran the nearby file-reloading machine, and asked if the machine was working today. “Yes,” he said. “So who will it be?”
“First, how do you know I want to bring anyone back?
“Even if I did,” she added, “aren’t you supposed to warn me about the dangers?”
“Even if I warned you, would you listen?” he asked.
Legion in Heaven, he had a point. Solomon knew he couldn’t really stop anyone — when you felt absence so acutely reason had nothing to say — so why even try? Love was potent like that, even when it was a love for someone who didn’t exist.
“Is it worth the pain of fresh grief just to experience this person again for a few minutes?” Solomon asked her. “Knowledge of the risks is much more effective than a warning,” he added, explaining his process.
“Everybody doesn’t experience grief all over again,” she countered.
“Some don’t, true. Could you see yourself being one of those people?”
“I think so,” she said.
“Each must make the decision for themselves,” Solomon said gnomically, and went to retrieve the files Inbal had written on her admission sheet.
Now that they’d gotten that out of the way, Inbal could concentrate on why she came. Creating — re-creating — the person she really wanted to see. Extremely needed to see.
“For argument’s sake, let’s say I wasn’t sure I could be one of those people,” Inbal said, circling back as Solomon warmed up the whole-brain emulator machine. “I still want them in my life now—to share my week, my problems, my life. Camaraderie brings consolation,” she added, reasonably.
“Tell me, is it consoling when you know they’re not actually listening? It just feels like they’re listening. Only a trick, made possible by a few projectors, some diligent file-keeping and one extremely powerful processor,” he said. “Nothing more.”
“Ugh, you don’t have to keep reminding me,” Inbal said.
“No matter,” Solomon answered, “here you go.” Turning a few dials, he made sure the machine was up and running and left the room.
Inbal gazed at the form as it took shape. Looking and seeming so much like her, even though every rational cell in her body knew it wasn’t.
“What have you been up to, my sweet girl?” the figure said as it filled itself in. “Everything in your life – your marriage, your children, your work – how is it?”
Rocking back and forth involuntarily, Inbal sought to compose herself. Echoes of her mother were somehow stronger than the real thing; how was that even possible?
“Utterly great. Perfectly fine. Ontologically wondrous. Never better.”
“Inbal, the more sarcastic you are the more I know things are not fine,” her file-mother said, firm but sympathetic. “Tell me what’s wrong.”
“Liam is staying ‘late at work’ a lot. I don’t know what to think but I think I just don’t want to think it. Kids, well, kids are great, but why it is always great in the abstract?
“Even work, my saving grace, is tiring, with all the politics, the bootlicking.
“Utilizing every trick in the book,” Inbal said as she took stock of it all, “I still feel like I can only carry half the load. Somehow everyone else carries the whole load. I carry half.
“Not even that sometimes,” she added, erupting into tears.
“God, what am I doing,” she said, wiping her face as the figure in front of her remained silent, “I see you for the first time in I don’t know how many years and I just start going on about my problems?”
“Tell me Inbal, my sweet girl, do you ever think of not being with him?” her file-mother finally said. “Effortless, that’s how I would describe my marriage to your father each of those 16 years, as though one of us could say something and the other could just say ‘ditto,’” using a popular family word, “like we not only saw the world the same way but could shorthand our agreement.
“Can’t say everyone’s is like that, though,” her file-mother continued. “Hoo boy, certainly not,” she said, getting a faraway look in her eye. “Not when you look at so many of our friends.”
“Or Aunt Harriet,” Inbal said.
Laughter burst from her file-mother’s face. Overwhelming laughter, the kind Inbal recalled from childhood. Gosh, it was hard to remember this wasn’t her. You sit at the re-loader and, well, it’s true what they say — you get taken right back to when everything was still OK. Those childhood years, so many childhood years, and yet not nearly enough. Oh, to be back there right now. Right at this moment.
Even though, Inbal realized, in a way she was back there — did an intellect or soul need tangible substance to have form; what was an algorithm but a mathematical distillation of the truth; what did it matter how someone came to say what they said if you loved the words they were using?
“So what else can you tell your mother— you know, to bring her joy, unbelievable joy?
“Unbelievable joy! Rudimentary happiness, maybe.”
“Ripples of brightness? Explosions of bliss?”
“Could you the set bar just a little lower?”
“Touches of mirth?”
“This has been so good,” Inbal said, smiling through her tear-streaked face.
“Here, you want to know how to find your purpose, to return from this sadness, to access the light— the source of life, really?” her file-mother said, segueing to the philosophical, like Inbal remembered from childhood. “Easy – you just strive to be true and good; that will give you centeredness, the purity to return from this dark place.
“Don’t ever forget you are who you are Inbal,” her file-mother said. “Even when the world is doing its best to make you feel like you’re not who you are.”
“And most important — you don’t forget that I love you,” Inbal whispered.
“Ditto,” her file-mother replied, before dissipating.
“Bye Solomon,” Inbal barely managed as she sprinted past him and onto the street. Under her sunglasses she could feel tears welling up again. The man was right. She felt a loss so sharp it was as if it had just happened. Her mother, taken from her all over again. Only now she had all the awareness of what happened since. Underwater for so long. Love hard, and vulnerability too. Death victimizing her twice, as it did many who lost a parent young.
What an incredible feeling to talk to her again though. Ecstasy, even just for a few minutes.
(Footnote: An acrostic was a device used by Medieval Hebrew writers to tag a poem or otherwise comment metatextually on the work in question)
IronSupplement
Everything you do — and don’t — need to know in future-world this week
When fake Musk comes for your real money; when AI restores a voice
1. HAVE YOU BEEN VICTIMIZED BY ELON MUSK SMIRKILY SELLING YOU A FICTION ONLINE?
Well, we all have. But have you been victimized by Musk selling you a bogus investment online? Well ok, that happened too. Maybe we should be more specific: have you been victimized by something that is actual video of Elon Musk but saying words he never said?
A new series of deepfake scams involves video of Musk with words AI-ed in. According to a New York Times report this week, Musk has appeared frequently in some 2,000 deepfakes that slowly separate users from their money with a slew of crypto and traditional investment “opportunities.” Video Musk suggests that you deposit your money in this account for untold riches, and then the money Houdinis away. The videos are very convincing, because even the casual tech is that good. Estimates have tens of thousands of users falling prey, and billions of dollars lost.
So what should we do about it?
Inevitably the conversation has turned to how to stop its distribution — what detection tools might block or identify scam videos; what penalties might make scammers less likely to undertake such attempts. So far nothing is working very well. YouTube has pulled down millions of videos, and yet still they proliferate. Meta is using detection software, but it’s a slog — the videos keep popping up as paid ads on Facebook.
Still, sooner or later, those methods will gain traction and we’ll start to roll back these cons. Also helping will be our level of sophistication. The Nigerian prince scam worked for a while in the 1990’s because we never saw email before. Then we did, and it became a joke.
Except stopping these Musk deepfakes doesn’t solve the problem long-term. Yes, that’s because the tech tools will keep getting more sophisticated. But it’s also for reasons more fundamental.
As with so many scams the focus will continue to be, I fear, primarily on the tools and not the causes. And that’s a mistake. How big is the income gap that people will get this desperate? What is the fetishization of easy wealth that makes users drawn to such videos? What is the cult of personality that make Musk (or Warren Buffett and Jeff Bezos, also deployed) such viable stalkinghorses? All these questions require a much harder look in the mirror and much more cultural work. Easier to write some new code and call it a day.
A few years ago I reported on an unfortunately popular crypto-romance scam. I was struck by how even savvy security-conscious people fell for it — including someone who was literally in charge of security for casinos. But a Tinderized culture has made love seem easily swipable — convinced folks that what was happening to them could only be a relationship and not a pickpocket. Thieves located this psychic loophole — no less real than a bank leaving a back door unlocked — and exploited it.
To stop these scams from victimizing us we need a multipronged approach. If a doctor only pushed defibrillators and stents and didn’t tell you to exercise and eat healthy, you’d look at them funny. So why are we doing that with scams?
2. SOMETIMES A FUTURE-WORLD STORY COMES ALONG THAT’S SUCH STRAIGHT-UP GOOD NEWS YOU CAN BARELY BELIEVE IT.
So it goes with a breakthrough this week from the New England Journal of Medicine, which reports on the instance of a 46-year-old man named Casey Harrell.
Harrell was diagnosed with ALS several years ago, which went on to take so much from the father and climate activist. Among the larcenies was his voice, preventing him from socializing, singing and making impassioned pleas on behalf of the environment. Also, quality time with his daughter, who was just one when he lost the ability to speak. He read her one last nursery rhyme, never to read to her again.
Then AI came along.
Harrell was a victim of an encroaching paralysis, which took his ability to move his facial muscles along with many others. But researchers from University of California, Davis had the idea that they might be able to use AI to restore that voice by identifying sounds he was trying to make — circumventing his facial muscles and plucking the speech right from his brain. So last summer they implanted an array of tiny electrodes in his brain to nab crucial signals and plugged hm into a giant database (actual large pieces of computer hardware set up in his house). Then they watched what happened.
Now, there are many, many ways this approach might have failed — for starters, the odds that the machine could identify attempted vocalizations by the pattern of firing neurons were long in their own right. But somehow the AI could do that; it was able to recognize sounds that Harrell was merely thinking of trying to make. Then it took the sequence of those sounds and, with a large trove of words it had been trained on, managed to piece them together, sound by sound, comparing each agglomeration of sounds to words in its training set. (Makes Alexa seem like the horse-and-buggy it is.)
First it could do this successfully with just a handful of words, but within a few days it was adept at searching a database of 125,000 words. Yes, the machine was able to decipher sounds from neurons, piece those sounds together and locate them in a massive dictionary — with over 90 percent accuracy.
Harrell was thinking of making sounds, and the computer could understand words — it could read his mind. The machine used AI deepfake technology (the good kind) to produce his voice to speak those words, too.
We’ve told you about similar types of implants — known generally as brain-computer interfaces — a few times before. Such tech has been deployed, for instance, to enable a tetraplegic person to move their hands again, like this research subject we interviewed last year. And Elon Musk’s Neuralink is working on similar applications. Incredible use-cases in their own right.
This lands in a different but equally impressive category, in which instead of reading a brain’s desire to move a muscle and then translating it back to the body, the machine can extract words from a person’s brain and bypass facial and speech capabilities entirely. The AI uses a Large Language Model — literally, for the words, but also for the neuron firings, recognizing each one based on so many it has been trained on before.
What we have is an evolution that goes from the text-to-speech program of Stephen Hawking, in which he used eye movements to type; to the more recent tech of a user being able to move a cursor with their mind to type out words; to machines now extracting language directly from brains.
The game-changing nature of the results for anyone suffering from ALS, stroke or other serious neurological condition that affects millions of Americans needs no adornment. It’s astonishing.
In fact, each paragraph in a New York Times story about the benefits the breakthrough conferred on its subject is more happy-tear-inducing than the next.
Like:
Or:
We write a lot about the dangers of technology in this space, the care with which such a spiky tool must be handled. When it comes to protecting our humanity, AI is often the stranger at the door we must properly vet before letting in.
But sometimes it’s already inside the house, driving out all the evil forces showing up outside.
The Mind and Iron Totally Scientific Apocalypse Score
Every week we bring you the TSAS — the TOTALLY SCIENTIFIC APOCALYPSE SCORE (tm). It’s a barometer of the biggest future-world news of the week, from a sink-to-our-doom -5 or -6 to a life-is-great +5 or +6 the other way. Last year ended with a score of -21.5 — gulp. Can 2024 do better? The summer hasn’t been great. This week? That’s a different story.
AI RESURRECTIONS ARE NOT AS FAR AWAY AS YOU THINK: Good? Bad? Crazy? We’ll lean positive: +1.0
ARE WE ATTACKING THE ROOT OF DEEPFAKE SCAMS?: -3.0
AI COULD GIVE ALS, STROKE AND OTHER VICTIMS THE ABILITY TO SPEAK AGAIN VIA THEIR THOUGHTS: +6.O