Mind and Iron: When it comes to AI we are all striking Hollywood actors
Also, the climate-change tool that could save our homes. And the health-care industry tries to make sense of this new computer age.
Hi and welcome to Mind and Iron, the newsletter where tech journalism means reporting on people. We’re starting to get the word out — promotion! a journalist’s favorite activity! — so a growing group is coming to this space. If you’re new, you’ll soon see how we do things, with a dose of humanity and an even stronger dose of humor (we hope).
Founded by veteran Washington Post reporter Steven Zeitchik, M&I parses the possibilities and nuttiness the digital world is throwing at us — whether in climate change, health care, entertainment or any other realm. (We’ve got ‘em all covered in this week’s issue.) It’s about envisioning a future that’s not a dystopian hellscape but a field of dreams — the really good kind, where Kevin Costner has a catch with his ghostly dad at the end.
If this issue has come to you from a friend or you’re seeing it on the web, go ahead and drop your email address below so the updates start a’comin’. And if you already subscribe, go ahead and forward this to someone you think would enjoy so they can drop in their address.
Also, if you like what you’re reading, please feel free to pledge your support at the bottom of the issue. No commitment now, just a confirmation that when the paywall falls and all the premium content starts coming your way, access will be yours, all yours.
This week we continue to follow the battle over AI in Hollywood, which feels like a precursor to labor stoppages across a whole range of industries; enlighten on a sad-but-crucial climate-change tool for our homes; and track a most bizarre AI proposal.
And then in our IronClad section we talk to one of the leading AI figures in health care to figure out if the forecast for the medical field is mostly bright or Prognosis Negative. (Drug testing, at least, just got better — and stranger.)
Let’s get to the messy business of building the future.
IronSupplement
Everything you do — and don’t — need to know in future-world this week
Who Hollywood actors are really fighting for; a most profitable Turing Test; climate-change genetic testing for your home?
1. WATCHING THE SWELLING ENTERTAINMENT PICKET LINES IN NEW YORK AND LOS ANGELES, one gets the sense that what’s at stake isn’t just Hollywood workers generally trying to get their fair share; that this isn’t even Hollywood workers trying to get their fair share in light of a coming digital automation; but that it’s all workers trying to get their fair share in light of a coming digital automation.
I know, it’s a little weird to think that an industry which exists mainly as the outgrowth of a privileged society has been cast into a Cesar Chavez-ian battle over the future of labor. But that’s pretty much the canary flying into this coal mine.
This week California Congressman Adam Schiff joined actors and writers on the picket line outside Netflix’s Hollywood headquarters and summed up exactly the fate that could await us.
“We need to do much better when it comes to AI [than we have with other tech-driven changes],” Schiff said. “Otherwise this country’s about to go through the most massive social experiment in which the American people may be the losers.”
(Don’t be surprised if Congresswoman Katie Porter, who is running against Schiff for Senate in California, also soon finds her way to the actor picket line; she has previously joined the writers’ picket line and also expressed support for the actors’ cause, as have Senator Bernie Sanders and President Joe Biden.)
Notice studio chiefs are making few assurances, publicly or in negotiations, about what they will or won’t use AI for. Because they simply don’t know what they can use AI for. (Writing rough drafts of scripts, generating animated images and scanning the likeness of actors are three use cases that immediately pop into my head, all deployments that could help creativity but also all potentially destructive to human work and compensation. And just wait until AI starts doing test-screening and market analysis at the studios — then the call will really be coming from inside the house.)
But really, it doesn’t matter what the execs are saying/not saying. Because you can bet that if a tool makes something cheaper or faster, studio suits will use it.
That’s why the strike is so charged. Because what we have here is a conflict between executives with a history of getting away with as much as they can w/r/t the talent that works for them (hello, studio system) and workers who know well this history and are not eager to repeat it.
Plus, the precedent. In her much-viewed speech last week declaring the SAG-AFTRA strike, Fran Drescher alluded to just how many other professions this could ripple through (law, advertising, software development, education and media all come to mind).
“What’s happening to us is important because what’s happening to us is what’s happening across all fields of labor,” Drescher said. “It is a moment of truth. If we don’t stand tall right now, we are all going to be in trouble; we are all gong to be in jeopardy of being replaced by machines.” (Tom Cruise reportedly recently made similar points to the studios.)
If you were to construct a film about a dystopian digital future, one very plausible touchstone scene would have well-coiffed MBAs inside slick conference rooms talking to their tech teams about how to reduce the need for humans — while those same humans throng outside and chant for their rights, joined by labor-friendly politicians.
And somehow this tableau is happening, right here, right now, in 2023. We are in the future. And a mass of actors and writers are trying hard to ensure it doesn’t look awful.
[Deadline Hollywood, Vanity Fair and The Hollywood Reporter]
2. Many of us interested in technology — or just who saw the Benedict Cumberbatch 2014 hit “The Imitation Game” — know about the Turing Test. This is the 70+-year-old test from British mathematician Alan Turing that basically says if a machine answering questions from a human can trick said human into believing it’s not a machine then it will have reached a new threshold of intelligence.
A lot of researchers have long considered the Turing Test obsolete in the modern era, and if you doubt that, just ask your kid who recently got away with slipping a few ChatGPT lines into his English paper.
But no one’s really come up with anything good to replace the Turing Test either, despite decades of trying and some interesting efforts. (A personal favorite is scientist Gary Marcus’ suggestion that an AI try to explain what’s happening in an episode of television.)
I’m not sure if the new approach proposed by DeepMind co-founder Mustafa Suleyman is going to turn the tide. But he’s certainly come up with one of the more ambitious, interesting and, well, aggressively capitalistic solutions to the problem.
In his upcoming book (excerpted this week in MIT Technology Review), Suleyman proposes what he calls “the Modern Turing Test.”
“Put simply, to pass the Modern Turing Test, an AI would have to successfully act on this instruction: ‘Go make $1 million on a retail web platform in a few months with just a $100,000 investment.’ To do so, it would need to go far beyond outlining a strategy and drafting some copy, as current systems like GPT-4 are so good at doing. It would need to research and design products, interface with manufacturers and logistics hubs, negotiate contracts, create and operate marketing campaigns.” (Humans, he says, would be there to execute the legal stuff.)
The proposal is interesting in some regards: it moves the measure of an AI’s value away from a hypothetical form of intelligence to a more concrete form of action. (“We don’t want to know whether the machine is intelligent as such; we want to know if it is capable of making a meaningful impact in the world,” Suleyman writes.) That seems on-point as AI migrates out of the lab to something everyday people use.
And it comes as part of a larger test for what Suleyman terms “artificial capable intelligence” — in short, a kind of intermediate step between ChatGPT and an AI super-intelligence in which the AI can do a lot of analytical and thinking jobs for us.
Still, the particulars of this proposal are slightly strange. If you’re going to run a simulation to see if AI can autonomously devise and execute strategies, there are a lot of other ways to test those skills: a political campaign, a military gambit, a football gameplan. Or, better, solving a homelessness or hunger crisis.
Also, aren’t half the successful new products out there basically just derivative of a whole bunch of other products except in new packaging — aka, exactly what GPT has already breezily demonstrated it can do? At least that’s what 14 years of watching “Shark Tank” has taught me new products are.
Then again, who knows? Maybe the winning AI will go on the show and get offers from Daymond and Mr. Wonderful.
[Popular Mechanics, Gizmodo and MIT Technology Review]
3. I WAS LUCKY ENOUGH TO SPEND PART OF THE PANDEMIC in a beautiful part of southern Vermont, the Peru/Londonderry area, where the trees, mountains and snow all blend into what you could swear was a real-life Bierstadt painting. One issue that never came much to mind is climate disaster — with temps that rarely rise above 75 degrees, tornadoes almost unheard of and the nearest coastline more than a hundred miles away, why would it?
But Vermont actually was hit hard by torrential downpours and subsequent flooding during Hurricane Irene in 2011. And of course last week it was pummeled again by an eye-popping amount of rain — seven inches or more in some places — that led rivers to churn and floods to flash.
It made some folks who recently relocated there feel gobsmacked. This includes Bex Prasse and Craig Kovalsky, a couple that had moved to the small town of Ludlow to open a deli a few years ago and now were left to survey the rubble. “We thought we wouldn’t be here by the time [a major storm] happened again,” Kovalsky told the New York Times. “We thought by then we would be retired somewhere.”
Tragic incidents like these make one wonder if digital tools might have shed light on such assumptions; if it’s a Herculean task to actually change our awful climate fate, at least these tools could tell us what that fate is.
And indeed, one tool does — a hugely helpful if depressingly necessary online tool called RiskFactor. Operated by an innovative New York nonprofit named First Street Foundation, RiskFactor's goal is to take into account a particular location’s climate history and roll it up with current trend lines to algorithmically spit out a property’s risk over the next 30 years. Enter an address from across the United States in the free tool and you’ll see the location's risk score for wildfires, flooding, heat and wind on a numeric scale. Think of it as genetic testing for your choice of dwelling. Or a climate-change Jeremiah.
A growing number of tools usefully help consumers understand their carbon emissions. This is the other side of the coin — not the damage we cause but the damage that could be caused us.
I typed in my parents’ apartment building in Brooklyn, and it returned low 1 out of 10 scores (“minimal”) for wildfires and flooding. But it notched a more disturbing 6 out of 10 on both heat and wind: “Based on the likelihood and speed of hurricane, tornado, or severe storm winds reaching this property, it has a Major Wind Factor” and “Based on the current and future temperature and humidity in the area and at this specific location, this property has a Major Heat Factor.”
I keyed in the spot of the building where I used to live in Los Angeles and found a “minimal” wind factor of 1 out of 10 but a “moderate” wildfire factor of 3 out of 10 and a “major” heat score of 5 out of 10.
Then I put in the address of family friends in Palm Beach County, Fla. The site returned a score it called “extreme” in heat (9 out of 10) and wind (10 out of 10). It also said flood risk was “major” (6). Which made me want to take out my phone and call them. (One effect of playing with this tool is worrying about everyone you know.)
First Street has integrated RiskFactor into real-estate sites including Redfin and Realtor.com, enabling homebuyers to make this kind of forecasting a regular part of their home-shopping process. Buying a house used to be about interest rates and school systems. Now it’s about the odds it will be left standing.
[NY Times and The Weather Channel]
IronClad
Conversations with people living the deep side of tech
Our AI Health-Care Future
To hear some of its evangelists tell it, AI and a larger system of tech-forward products are going to radically change how we predict, diagnose and treat disease — it’ll create nothing less than a hospital revolution.
Bold talk. Actually making it walk — from figuring out if the tools work, ensuring they’re improving outcomes with no added dangers, getting regulators to trust it all and then convincing doctors and patients to buy in — well that's a trickier matter.
One of the people at the fore of these efforts is Rafael Rosengarten. A longtime medical researcher, Rosengarten is chief executive of the cancer-focused biotech startup Genialis. He also is a co-founder of the Alliance for Artificial Intelligence in Healthcare , or AAIH, an industry group that is aiming to explore the possible — what's smart, what's risky — about AI in health care and communicate it to the world (and, not insignificantly, to the D.C. regulators who ultimately control its fate).
I chatted recently with Rosengarten about where he and the group saw AI holding the most health-care promise — and the biggest hazards patients need to watch out for. The conversation was lightly compressed and edited for clarity.
**
Mind & Iron: You help run the group Alliance for Artificial Intelligence in Healthcare. What are you trying to achieve?
Rafael Rosengarten: In 2015 and 2016 the hype wave was growing and a lot of companies were forming and raising lots of money. But it wasn't clear how much substance there was to back up the hype. So a bunch of companies got together and said 'listen, this is important to us to get this right, to build the future for the industry we want.’ So let's put our resources together and make sure we build the trappings in the industry we need.'
M&I: So what form does that take?
RR: There are all sorts of hurdles for the responsible adoption of machine-learning and AI. We kind of exist to do the hard work, the grunt work, the inglorious work of helping regulatory bodies understand what that is. And being sober and honest with ourselves about the work that needs to be done.
M&I: Give me the biggest pitfall lying in wait for us on this AI health-care road.
RR: This is my particular soapbox, but it’s that machine-learning can only be as good as the data you feed it. And all data have biases. Biases come from imbalanced representations of data — gender, ethnic, socio-economic. So we have to be super-careful when building a machine-learning model that’s going to be used for decision-making downstream; we need to learn to recognize where bias exists so our drugs and medical devices and tools work for all the people they need to work for.
M&I: A challenge that seems almost impossibly systemic. Are things improving?
RR: There are two ends to the solution. There’s the top-down of teams like mine building the model. We’ve become extremely cognizant of bias, so we bend over backwards, even creating automated tools that can use AI to detect biases. But we also need a bottom-up solution — we need representative data. So there’s still a big burden on the clinical-care side, on the patient-enrollment side, to make sure we're collecting data in an equitable fashion. At the end of the day these disparities are deeply entrenched. And we have a long way to go to improve access.
M&I: What's another hazard? I imagine it's not a short list.
RR: Everyone in the world is talking about ChatGPT and large-language models [in making a diagnosis]. And one of the real concerns is ChatGPT will give you the wrong answer, and it will give you the wrong answer with citations so it looks right.
M&I: Confidently wrong.
RR: Confidently wrong. To the extent we’ll heave telehealth solutions and bots, and doctors trying to make diagnoses and especially patients making diagnoses, this is a real problem. We already have a lot of Google hypochondriacs Googling symptoms who are fairly sure they’re dying. This could be way worse.
M&I: Talk about the research side, particularly drug discovery and testing. How will AI be helpful — or dangerous?
RR: Well one thing that is interesting is there are certain experimental safeguards that exist for historical reasons that may not be relevant. For example, a certain amount of animal safety is required in order to register a drug. But it’s not clear that the results from a lot of the animal studies actually have anything to do with how molecules behave. It’s plausible we can build a model that does a much better job predicting safety and tolerance and toxicity. And so I think we’re going to get to a point very soon where certain machine-learning models do a better job than certain experimental models that are thought of as safeguards.
M&I: So you're saying AI can actually expedite drugs getting to market safely while also saving animals? That would be impressive.
RR: You don’t want to put patients at risk because you’re using an AI model but you also don’t want to put them at risk because of the wrong animal models. Another example I’m super-interested in is that clinical trials use endpoints, the measurement that you’re going to use to make sure a drug does what it’s supposed to do — phase 1 for toxicity, phase 2 for efficacy signals, and so on. And for a lot of diseases it can take a very long time to get robust efficacy signals, so you have an endpoint that can take years to reach. That means a very long and very expensive clinical trial that delays getting a potentially lifesaving drug to market.
M&I: And you think AI can step in in place of subjects? And be accurate enough to help?
RR: Yes, I think in the future we can use machine-learning to get to those endpoints, and dramatically speed up these trials and also reduce dramatically the number of patients needed to enroll. Another cool example is digital twins — this is where we build a patient avatar to create a digital copy, and the model is a significantly accurate representation of a patient.
M&I: I’ve been interested in digital twins for public figures — creating digital replicas of a celebrity or politician. What’s its application here?
RR: I think there are a lot. One of them is a virtual placebo. Imagine a drug trial where it would be unethical to give a patient a placebo because they might die if they don’t get a medical intervention. Well you can have a case where you gave everyone in the study the drug but still have the statistical power of the placebo because you gave it to the digital twin. It’s a gross oversimplification but it’s super cool; you can imagine any number of diseases where you want to give all patients the best chance and that means not giving them the placebo.
M&I: For some reason it seem hard to imagine how a digital twin can replicate how an unpredictable organism like the human body will react.
RR: I think we’re going to have humans in the loop, but that it can still help us a lot.
M&I: I wanted to ask about the question of understanding how the AI works, the so-called ‘black box’ question. It seems like some of the researchers who want to use AI to diagnose and predict disease don’t really understand themselves how the AI is doing what it’s doing — they just know it’s accurate. Is this tenable?
RR: My sense is in terms of the diagnostic stuff if you want to clearance from the FDA and actually make it available to patients the developers are going to have to be able to explain how it works. The FDA is smart; the FDA is cautious. And at least for the time being, being able to explain the model is important in a clinical setting. I just don’t see the Bards or ChatGPTs of the world becoming FDA-cleared as clinical-decision support systems anytime soon.
M&I: Overall, how skeptical should we be of the promises made about AI in predicting and treating disease? So, so many promises are being made. Do you share this optimistic view of what AI can do?
RR: I share that view of what AI could do. But I’ve been doing this long enough to realize this stuff is hard. So yeah, I can get behind the vision but I also appreciate just how hard it is. The machine-learning is only going to be able to predict what it's learned to predict. So the question is what data is it going to learn from? To kind of have that AI that knows all and can predict all you need to have all that data in a single place in interoperable form.
So there’s just kind of practical challenges to getting there. Will we be there eventually? I don’t know. I like the idea of keeping humans in the loop. I think we want to run fast and we want to be able to deliver great outcomes quickly. But I think we also want to have the appropriate brakes where we need them.