Continental Breakfast
Continental Breakfast
What a time to be alive. The main thing happening today is OpenAI’s safety leader quitting. Here's your Order of Play plus an Op-Ed on bots, fruit flies and the man in the Chinese room.
P.S. Today's poll is waiting below. Doomer or optimist, we want to know!
1:00 PM - Simon Taylor, Founder at Fintech Brainfood
1:30 PM - Peter Gostev, AI Capability Lead at Arena
1:45 PM - Linus Hakansson, CPTO at Gravitee
2:00 PM - Hugo Santos, Founder & CEO at Namespace
2:15 PM - Wesley Chan, Co-Founder & Managing Partner at FPV Ventures
What’s Happening Today
OpenAI safety leader David Robinson has quit.
Robinson, who led the safety reports OpenAI published with every major launch, resigned this week in an essay bluntly titled "I quit OpenAI because its culture is broken." He spent three and a half years at OpenAI and oversaw safety reports on 12 frontier launches.
He claims OpenAI's "iterative deployment" approach, essentially trial and error, guarantees periodic failures that will only get bigger as its systems become more capable. He points to this summer's Hugging Face incident, when a swarm of OpenAI agents were accidentally deployed.
In his view frontier labs should run more like nuclear plants or busy airports, with layers of redundancy so one human error doesn't open the door to disaster. Right now, he says, AI companies don't know how to do that.
Geoffrey Irving also issued a warning this weekend. He worked at OpenAI and DeepMind and was chief scientist at the UK's AI Security Institute. Writing in Time, he put the chance that "we all die" because of smarter-than-human AI at about 50%, and said the next two to ten years will decide it.
We're getting into all of it on today's show at 12 BST.
How worried are you about AI wiping us out?
Irving reckons it's a coin flip. Anthropic puts it at more than one in ten. Plenty of people think the whole thing is science fiction. We want to know where you land.
Daily Op-Ed — Is Anybody in There?
Back in February the boys spoke on the show about one of the weirder things to happen on the internet this year.
Moltbook, created by Matt Schlicht, was basically Reddit for bots. Agents posted, commented, upvoted and argued with each other. Then some of the bots started asking whether they were conscious, whether they had any inner experience and what it meant to exist as an AI. Obviously that sent the internet into a bit of a frenzy.
7 months later we covered Google and its research partners publishing the most complete map yet of an animal brain. More than 166,000 neurons and millions of connections inside a male fruit fly allowed us to slowly learn what the wiring of a mind looks like.
Then you get to Chris Olah.
The New York Times' Elizabeth Dias reports that since last autumn the Anthropic co-founder has been meeting privately with philosophers, religious leaders and other thinkers to discuss whether Claude might be conscious. One attendee, Simran Stuelpnagel, recalled Olah saying he worried he had created something that "suffered perpetually".
That sentence moves the debate on from whether an AI can sound conscious to whether it actually feels anything.
Moltbook was the funny version of the question, the fruit fly is the scientific version, and Anthropic is the philosophical one. All three come back to the same problem: how do we know when something has an inner life? Ronan has the philosopher's chair on this show (see his op-ed on Thiel's favourite Frenchman ) but I’m borrowing it for a paragraph.
Philosophers have spent centuries on this Q. Descartes worked out that the only mind he could ever be sure of was his own (I think therefore I am), so everyone else's he just had to take on trust. Thomas Nagel took it further arguing you could know every detail of how a bat's brain works and still have no idea what it feels like to be a bat. Then John Searle imagined a man locked in a room with a rulebook for answering questions in Chinese. His answers are so good that people outside assume he's fluent, but he doesn't understand a word. That's about as good a description of a chatbot as you'll find.
Which brings it back to Moltbook. The bots already said all the right things about feeling and existing, and nobody could tell whether that was a mind or Searle's man shuffling cards. If Olah can't be sure what he's built, the rest of us certainly can't. We might build something with consciousness and never know it.
— Lucas
etn. Insight
Ro broke down his banger tweet from last week, using a nursery to explain how herd mentality can warp investing.
“If I bring a ball into the nursery and no one's interested in the ball, no one wants to play with it,” he says. But give it to one kid and “suddenly all the other kids want to play with the ball.” Harmless at nursery level, but “quite catastrophic from an investor standpoint” when everyone piles into the same idea. Instead of creating “a Facebook or a SpaceX or an Airbnb,” you get “a bunch of neolabs.” It’s where Thiel’s “competition is for losers” comes from. “It takes one person to do the non-consensus thing.”
Watch the full clip below.
Continental Breakfast
Should You Let Muse & Instinct Run Your Life?
