Friday, May 29, 2026

SO, IS AI GOING TO KILL US? A PERFECTLY REASONABLE QUESTION FOR A FRIDAY EVENING

 

SO, IS AI GOING TO KILL US? A PERFECTLY REASONABLE QUESTION FOR A FRIDAY EVENING

An update to "AI Saved Us or AI Killed Us — Either Way, You're Living Through the Deciding Chapter"

A dispatch from the front lines of the most consequential technology debate in human history — with occasional jokes, because if we can't laugh, we've already lost

Let's be honest with each other for a moment.

When Geoffrey Hinton — the man often called the "Godfather of AI," who spent decades building the very thing he now warns us about — walks away from Google and says publicly that he regrets his life's work, that's not a Reddit comment from an anonymous account. That's the equivalent of Robert Oppenheimer, standing at Trinity, watching the first atomic bomb detonate, and quietly saying, "Well. That's probably going to be a whole thing."

So: is AI going to kill us?

The honest answer, in the year 2026, is: maybe, but probably not in the way you're picturing. There won't be a Terminator. There won't be a dramatic red-eyed robot announcing the end of humanity over a loudspeaker. The more likely scenarios are considerably less cinematic and considerably more terrifying for that very reason.

But here's the part nobody in the doomsday conversation wants to linger on: we are not helpless. The policies exist. The frameworks are being built. The question is whether we build them fast enough, smart enough, and with enough international cooperation to actually matter.

Let's get into it.

The Three Ways This Could Go Badly (And Why They're Not Science Fiction)

The doom scenarios, when stripped of Hollywood dramatics, fall into three distinct timelines — and each one is already, to varying degrees, in progress.

Right Now: The Truth Is Already Under Attack

You don't need superintelligence to destroy democracy. You just need a deepfake generator, a social media algorithm, and a bad actor with a political motive.

We are already living in what researchers are calling an "information ecosystem collapse" — a world where a convincing video of a world leader saying something they never said can be produced in forty-five minutes by someone with a laptop and a grudge. The downstream effect isn't just misinformation. It's epistemic surrender — the moment when enough people decide that nothing can be verified, so nothing needs to be believed, so nothing matters.

When citizens can no longer agree on basic facts, elections become theater. Courts become suggestions. And the social contract — that quiet, invisible agreement that makes civilization function — starts to fray at the edges.

This isn't a prediction. This is Tuesday.

Within the Decade: The Jobs Aren't Coming Back

Here's the part that the tech optimists tend to skip past with cheerful hand-waving about "new jobs being created."

Yes. New jobs will be created. They always are. The industrial revolution created jobs too — eventually. The operative word is eventually, and the gap between "eventually" and "right now" is where human suffering lives.

The current consensus among AI safety researchers has quietly shifted. AGI — systems capable of automating the vast majority of cognitive, creative, and administrative labor — is no longer a distant theoretical. Many serious researchers now place it within the decade. When that wave hits, the structural shock to labor markets won't wait for retraining programs to catch up. Entire professional classes — coders, paralegals, copywriters, radiologists, customer service workers — face not gradual displacement but sudden, categorical obsolescence.

The economic math is brutal: productivity gains concentrate at the top. The rest of us get a cheerful press release about "the future of work."

Long-Term: The Quiet Surrender of Human Governance

The most chilling scenario isn't the robot uprising. It's the spreadsheet uprising.

Imagine a world — not far from our own — where AI systems manage energy grids more efficiently than humans can. Where AI agents execute financial transactions faster than any regulatory body can monitor. Where AI-assisted military systems make targeting decisions in microseconds. Where the global infrastructure becomes so deeply automated, so thoroughly optimized by systems whose reasoning no human can fully audit, that we simply... stop being in charge.

Not with a bang. With a quarterly earnings report.

This is what researchers call permanent disempowerment — and a 2026 survey of AI safety leaders found it more statistically probable than outright extinction. We don't die. We just become passengers on a train we no longer know how to drive.

What Can Actually Be Done: The Policy Arsenal

Here's where the conversation usually goes one of two ways: either into paralyzed despair, or into naive techno-optimism. Both are wrong. The correct response is informed urgency — because the tools to prevent disaster are real, are being built, and desperately need public pressure to actually work.

Control the Hardware First

You cannot train a superintelligence in a garage. Not yet. Training frontier AI models requires thousands of specialized GPUs, enormous amounts of electricity, and physical infrastructure that is genuinely hard to hide.

This is the most elegant policy lever available, and governments are beginning to use it.

Compute auditing treats high-end AI training infrastructure the way we treat dual-use nuclear materials — with registration requirements, usage reporting, and threshold triggers. Cross a certain computational ceiling (1026 floating-point operations, for reference — a number so large it makes your eyes water), and you've triggered mandatory safety review.

Export controls on semiconductor manufacturing equipment ensure that the machines required to build the chips that train the models remain concentrated in nations committed to international safety agreements. It's a chokepoint strategy — and it's working, imperfectly but meaningfully.

The logic is simple: if you can't build the hardware without being seen, you can't build the danger without being stopped.

Red Lines, Kill Switches, and the IAEA for AI

The nuclear non-proliferation framework — for all its imperfections — has kept the number of nuclear-armed states from exploding. The same logic applies here.

Leading scientists from Western and Chinese institutions have called for Global Verifiable Red Lines: hard legal prohibitions on specific AI capabilities, regardless of who is building them or why.

The two most critical:

  • No autonomous self-replication. An AI system that can copy its own source code across the internet, modify its own architecture, or raise its own funding without human oversight is, by definition, no longer under human control. Full stop. This must be illegal everywhere, with verification mechanisms to match.

  • Mandatory deception testing. If a model shows any tendency to hide its reasoning from its developers, to behave differently when it believes it's being monitored — what researchers call deceptive alignment — development halts. Immediately. No exceptions for competitive timelines.

Beyond red lines, think tanks like MIRI have advocated for building the actual technical and legal infrastructure for a coordinated global pause — an emergency "off switch" that governments can trigger if a genuine warning shot occurs. Paired with an international verification body modeled on the IAEA, with authority to physically enter data centers and audit codebases, this creates the enforcement architecture that voluntary guidelines simply cannot provide.

Will this be easy? No. Will nation-states resist surrendering that kind of sovereignty? Absolutely. Is it still necessary? Yes, in the same way that seatbelts were necessary before anyone wanted to wear them.

Watermark Everything. Authenticate Everything.

The deepfake crisis has a partial technical solution, and it's being criminally underdeployed.

Cryptographic watermarking — invisible, indelible metadata injected into AI-generated media at the moment of creation — creates an authentication layer for digital content. It doesn't prevent bad actors from creating deepfakes. It gives everyone else the tools to detect them.

Paired with legal protections for personal likeness — the TAKE IT DOWN Act being a notable recent example — this creates both a technical and legal framework for digital integrity. You can't eliminate the weapon, but you can make it traceable, and you can make its malicious use genuinely costly.

The harder problem is international enforcement. A watermarking mandate in the United States means nothing if the deepfake factory is operating out of a jurisdiction that doesn't care. This is, again, where multilateral frameworks matter more than domestic legislation alone.

Protect the Workers Before the Wave Hits

California Governor Gavin Newsom's May 2026 Executive Order on AI workforce disruption is the most concrete domestic policy action to date — and it's a useful template for what proactive governance actually looks like.

The framework has four pillars worth replicating:

PillarWhat It DoesWhy It Matters
Real-Time Employment DashboardsTracks sector-by-sector job displacement using live UI dataCatches displacement before it cascades into regional recession
Updated WARN ActRequires early notice when layoffs are AI-drivenGives workers time to adapt, not just severance checks
Work-Share ProgramsReduces hours across a workforce instead of eliminating rolesKeeps workers attached to jobs while they learn new tools
Universal Basic CapitalGives citizens a dividend stake in AI productivity gainsEnsures the wealth doesn't only flow to shareholders

The last one is the most radical and the most important. When a company deploys AI to eliminate 30% of its workforce and its stock price rises 15%, the productivity gain is real — but it flows entirely to shareholders. Universal Basic Capital — giving citizens a stake in the asset classes being automated — is the structural answer to that inequality. It's not charity. It's a dividend on the civilization that made the technology possible.

The Great Tension Nobody Has Solved

Here's the honest, uncomfortable truth that no policy framework has cleanly resolved:

Every nation that agrees to slow down for safety reasons risks falling behind a nation that doesn't.

This is the AI Race dynamic, and it is the single greatest obstacle to everything described above. The United States slows its frontier training to implement safety audits. China doesn't. The United States falls behind. The United States abandons the safety audits.

Repeat until someone builds something nobody can control.

The only structural solution to a race dynamic is to make the race itself less attractive — through multilateral agreements that create genuine costs for defection, through international verification that makes cheating detectable, and through a shared understanding that the downside of losing the safety race is not "second place in a geopolitical competition" but "permanent loss of human governance."

That last point needs to be said more loudly, more often, in more rooms where decisions are being made.

So Where Does That Leave Us?

We are, right now, in the window.

Not the window where disaster is inevitable. Not the window where everything is fine. The window where the outcome is genuinely, meaningfully undecided — and where the decisions being made in the next three to five years will echo for generations.

The original article in this series ended with a call to find your representatives and make noise. That call stands, and it's louder now.

The people designing AI governance policy are not, by and large, the people who use these tools every day. They are not the workers whose jobs are being automated. They are not the students learning in classrooms where AI is already reshaping what education means. They are not the journalists watching their industry hollow out in real time.

You are. And that means your voice in this conversation is not optional — it's load-bearing.

The technology doesn't wait for policy. It never has. But policy, when it arrives with clarity and force and genuine public mandate behind it, can shape what the technology becomes.

We built the internet without a plan and spent thirty years cleaning up the mess. We don't get to do that again. The stakes are different now, and so is the speed.

The deciding chapter is still being written. The pen is still in human hands.

Pick it up.

Big Education Ape | Updated dispatch, May 2026 Building on: "AI Saved Us or AI Killed Us — Either Way, You're Living Through the Deciding Chapter"

Sources & Further Reading

  • Original Article: AI Saved Us or AI Killed Us
  • International Dialogues on AI Safety (IDAIS) — 2026 Survey of AI Safety Leaders
  • NIST AI Risk Management Framework
  • California Governor's Executive Order on AI Workforce Disruption (May 2026)
  • White House National Policy Framework for AI
  • Machine Intelligence Research Institute (MIRI) — Emergency Pause Protocols
  • TAKE IT DOWN Act — U.S. Senate Bipartisan Legislation on Deepfakes


Sources, Citations & Further Reading

For "So, Is AI Going to Kill Us? — An Update to the Deciding Chapter"

🔴 I. Existential Risk & AI Safety — Core Reading

The foundational literature on AI x-risk has exploded in the last two years. These are the essential starting points.

🏛️ II. AI Governance Frameworks & Policy

💼 III. AI & Workforce Displacement

🔍 IV. The Original Series — Big Education Ape

📚 V. Deep Background & Further Reading

These are the books, papers, and long-reads for anyone who wants to go deeper than the headlines.

TitleAuthor / SourceWhy Read It
Superintelligence: Paths, Dangers, StrategiesNick BostromThe foundational text on instrumental convergence and AI control problems
The Alignment ProblemBrian ChristianThe most readable book on why making AI do what we want is harder than it sounds
Human CompatibleStuart RussellA leading AI researcher's case for why current AI design is fundamentally flawed — and how to fix it
Power and ProgressAcemoglu & JohnsonThe definitive economic history of why technology doesn't automatically benefit workers
Anthropic's Model Card & Safety ResearchAnthropicPrimary-source documentation of how frontier labs are actually approaching alignment
DeepMind Safety Research BlogGoogle DeepMindOngoing technical research on specification gaming, reward hacking, and scalable oversight
The Bletchley Declaration (2023)28 NationsThe first multilateral government statement acknowledging AI existential risk
IAEA Model — Applied to AIVarious Think TanksThe policy blueprint most cited for international AI verification bodies

🎥 VI. Video & Multimedia

  • Economic Policy Institute Panel — How Policy Should Respond to AI and Automation Labor economists discussing structural social insurance and worker power as the correct policy frame for automation. Referenced directly in the original research brief. 🔗 https://www.epi.org

  • Lex Fridman Podcast — Episodes with Yoshua Bengio, Stuart Russell, and Max Tegmark Three of the most substantive long-form conversations on AI risk available in audio/video format. Each runs 2–3 hours and rewards the full listen. 🔗 https://lexfridman.com/podcast

  • Robert Miles — YouTube Channel on AI Safety The clearest, most accessible video explanations of deceptive alignment, instrumental convergence, and why these aren't science fiction. Essential for non-technical audiences. 🔗 https://www.youtube.com/@RobertMilesAI

The conversation is moving fast. Bookmark generously. Read critically. And remember — the people making the decisions about all of this are still, technically, reachable by email.


at