Nancy Pelosi’s replacement is ready to regulate AI
Now, after winning a tight congressional race for former Speaker of the House Nancy Pelosi’s open San Francisco seat, he could play a key role in setting federal AI policy.
- The previous day, the AI nonprofit METR had released a report that had been widely interpreted as evidence that Anthropic had achieved recursive self-improvement, a phenomenon in which AI meaningfully contributes to its own advancement and AI progress potentially spins out of control.
- SB 1047, a bill that he authored in 2024, was set to become the first real piece of frontier AI legislation in the country before it was vetoed by California governor Gavin Newsom; he then sponsored SB 53, a lighter-touch bill that Newsom signed.
- “I think it’s really important to have a strong, empowered, nimble federal regulator,” he says, noting that Representative Ro Khanna, a fellow Democrat from California, has proposed legislation to establish such a regulator.
He wants regulation with real teeth: Wiener is pushing for a powerful federal AI regulator, whistleblower protections, kill switches, and auditing requirements — and he's firmly against federal laws that would strip states of their own ability to regulate.
“It’s very concerning,” Wiener said, citing the report as evidence that we are quickly running out of time to regulate AI.
- He also highlights the importance of whistleblower protections, testing and auditing requirements, and a kill switch requirement, and he wants to see Congress leverage its oversight powers to hold the AI labs to account.
- “That does not appear to be happening.” Wiener has been increasingly focused on AI in recent years, and he’s made many of the AI safety movement’s concerns his own: During our 20-minute conversation, he used the phrase “recursive self-improvement” half a dozen times, for example.“We need to slow this down,” he says.
- “Public First supported me because I was the first lawmaker in the country to introduce AI regulation and the first to pass it and get it implemented,” he wrote in a message after our interview.
- Wiener’s congressional campaign received over $3 million from Public First Action, a nonprofit in favor of AI regulation that receives substantial funding from Anthropic.
- That contribution led opponents to criticize him for being too close to the AI industry, but Wiener says that couldn’t be further from the truth.
How this summary was made: extractive summary — built from the sources' own reporting. It is drawn only from the reporting of the sources listed below — read the originals for full detail. How we summarize →