One number may determine our economic future: Time-to-Failure. If trends hold, in 2026 we’ll see human labor start to become unprofitable overhead.

Over the holiday break, I got annoyed at having to do a bunch of B.S., so I made a an AI to handle it for me. He can find out what irritating paperwork the State of California wants me to complete, search my computer for the answers, ask me a few questions, and fill it in. He looked at my companies and researched corporate structuring, and made good recommendations. He drafted contracts. Next up, I’m having him research college scholarships and grants for my daughter. I didn’t expect this to work, so I called him “YOLO” fully expecting him to just try and epically fail. He hasn’t, he’s just a little wobbly.

In his brief five-day existence with just a little hand-holding and zero HR headaches he’s done over a month of human labor. He still sometimes goes down a rabbit hole like an eager intern, but even then he’s much easier than hiring humans. And he’s the canary in the coal mine.

YOLO is useful not because he’s perfect, but because he does the work. Yet, if you look at how the industry measures progress, they aren’t measuring work at all. They’re grading exams.

For the last two years, we’ve been obsessed with “saturating metrics” – scores that top out at 100% and measure discrete tasks in a vacuum. Take MMLU (Massive Multitask Language Understanding). It covers 57 subjects, from STEM to the humanities, and for a long time, it was the ceiling of human expert capability. That ceiling has been shattered. As a recent paper points out, Gemini Ultra has already achieved “human-expert performance” on the test.

When the machines bested the easy tests (“easy,” but I’d probably fail them), the industry pivoted to “hard” benchmarks like SWE-bench (solving real-world code engineering problems). This is much harder, but in under two years performance jumped from less than 2% to over 76%.

All this sounds (and is) very impressive performance for Team Robot.

But the real question is “what can these AIs actually do,” and on that yardstick all these tests are vanity metrics. As we know from the SAT, MCAT and LSAT, test-smart ain’t street-smart. Tests measure taking the test, not the ability to execute a real-world job. We have “solved” test taking; we should now be measuring work doing.

If you want to know if you’ll have a job in two years, ignore test scores. The only metric that dictates our future – economic and existential – is Time-to-Failure for Autonomous Operations.

In 2026, this specific line on a graph will determine if a knowledge worker is a “Centaur” – an augmented human orchestrating the machine – or headed for obsolescence.

The Forty Hour Workweek … in One (Robot) Hour

If we stop looking at test scores, what do we look at? The best place is METR (Model Evaluation and Threat Research).

While the industry obsesses over benchmarks and raw numbers (which drives funding and hype) – METR asks the only question a hiring manager cares about: How long can you work without me fixing your mistakes?”

This is the metric of Time-to-Failure.

Specifically, it measures the “Time-Equivalent of Human Labor.” If an AI completes a task that would take a competent human expert 4 hours, that counts as 4 hours of autonomous work, even if the AI did it in 15 minutes.

This distinction is the economic guillotine. It fully and finally decouples the value of work done from the cost of labor as humans have always conceived of it. We have come to expect that things take time, and the trade for showing up to our cubicle farm means we get to go on Amazon and buy Shit We Don’t Need as a whim. But when an AI captures a half-day of human wage-value for pennies of electricity, the bread line isn’t just a scary phrase; it’s inevitability.

The Super-Exponential Curve

Not long ago, the Time-Equivalent of Human Labor metric was measured in seconds. An AI would hallucinate or get stuck almost immediately. That era is over.

Current internal benchmarks for Claude Opus 4.5 show it achieving around five hours of autonomous work at 50% reliability. That is, ask a human to do five hours of work and he produces some work product. Claude does the same thing (in a few minutes), and it does this successfully half the time.

Wait, but doesn’t 50% suck?

To a human manager, 50% reliability for an employee is a firing offense because humans are slow and expensive (and let’s face it, the ones with a low reliability rate are the ones that are HR nightmares too).

But in digital labor, “50% on a single run” can behave like “near-certain” success, because you don’t buy one attempt – you buy many.

If each run has a 50% chance of success and you run five attempts, the chance of getting at least one good result is 96.8% (1 – 0.5^5). In practice, retries help when the failure is correctable – unclear success criteria, missing constraints, or a bad plan that can be revised. When an AI is failing for a structural reason (no access, no tool, no capability), you get five fast failures – and then the advantage is you find that out quickly, patch the missing piece, and rerun.

In my testing over the last year, when an agent “goes off the rails,” it’s usually because I gave it sloppy success criteria or forgot to give it access to something – the same failure mode you see with humans.

The net of it: nearly 97% success is probably better than your best employee. For cheaper. For faster. And that’s today’s AI models, in six months they’ll be better.

As a practical standpoint, the “trust threshold” – where managers stop checking every output – is generally around 80% reliability. We are now seeing agents hit that mark for tasks lasting ~4.5 human hours. Again, in minutes. For pennies.

This is the Half-Day Milestone. It means an AI agent can cover the morning shift while you sleep. It marks the transition from the human “doing” the work to the human merely “reviewing” it.

And we’re not done.

The Wage-Work Breakpoint

The part that should terrify white collar workers isn’t where we are; it’s how fast we’re moving. This capability is currently doubling every 4 to 4.5 months. If we project that curve forward, the timeline for 2026 for “human-equivalent independent labor at 80% reliability”:

• End of Q1: We hit ~9 hours of autonomous operation (one full human workday) (being generous, real-world office workers might be productive for 4 hours. Maybe).
• Mid-2026: We hit ~18 hours of human work (2-3 full days).
• End of 2026: We hit 36-40 hours.

By the time we ring in 2027, we will likely have agents capable of executing a full week’s worth of human labor autonomously before failing. Even if it slows down by half, this is a change we’re not prepared for.

This is the economic breakpoint. Where the concept of a weekly salary for cognitive work disintegrates, because the machine can do the week’s work in an hour, for a dollar.

More scary, it’s when the social contract starts to fail: no longer does the employee class put in a week of cubicle farm time in exchange for a modest living (and maybe a shot at something better); in all likelihood we see a ripple across the economy where workers have negative economic value to employers.

This is why my little YOLO, as janky as he is now, is the most dangerous thing on my hard drive.

Of course, this won’t happen smoothly. The robots won’t come marching in and one day we’ll all be out of a job. The real world is messy: legacy systems, compliance, liability, change management, workforce morale during the changeover, these are real constraints. As I wrote before:

MIT’s David Autor highlights “Polanyi’s Paradox,” which is the idea that we know more than we can tell. In other words, many human tasks rely on tacit knowledge or situational adaptability that isn’t easily written as code or learned by an AI. … even if true, it only delays the inevitable.

So yes, we can give ourselves comfort that the messy, real world means that we don’t wake up one day to zero jobs. But that doesn’t mean we can ignore it, or that our particular situation will be last to go. We are frogs, AI is the slowly boiling water.

Killing Centaurs

AI-deniers tell a comforting bedtime story to ward off the existential dread: The “Centaur” model. As I wrote in AI Will Take Your Job. Stockpile Assets or Join the Bread Line:

The “Centaur” model is the idea that humans will be augmented with the tools that AI brings. It’s the analogue to “we’ve always elevated ourselves in other revolutions, so we’ll do it with AI.” Proponents have us orchestrating fleets of AIs to do our bidding, securing our place in the world above the machines.

We convince ourselves that while the AI might generate the text or write the code, it needs our judgment and guidance to do things.

This model relies on one assumption: that the AI needs supervision.

As long as the “Time-to-Failure” is short (minutes or hours), the Centaur model holds. The human is necessary because the machine is liable to drive off a cliff if left alone for too long. This is where YOLO is today, I need to check on him once a day. I am the babysitter, and YOLO is an unpaid middle-school summer intern.

But what happens when the intern grows up and is more capable than the babysitter?

The 40-Hour Threshold

When the duration of useful, autonomous work hits 40 hours of human work – the economic calculus of the Centaur breaks.

If an agent can do a week’s worth of human labor, solve complex problems, fix its own mistakes, navigate corporate bureaucracy, and report back with a finished product without needing a human to check its work every few hours, the human is no longer an orchestrator.

The human is unprofitable overhead.

“But wait, that means the human can do 40x as much, because they can spend 40 hours managing these AIs!”

It doesn’t work that way. Can you imagine the context switching of having 40 different AI agents reporting to you, having to decide if each did their job, then setting them spinning again? What about months later, when it’s 80?

No, we eliminate the next level of management up, replace them with an AI that can manage at the speed of silicon, rinse, repeat.

We are meat computers, and can’t ride this exponential curve. And even if it’s an “S-Curve” and flattens out – as I’ve proved (to myself), with YOLO the world already isn’t what we thought it was.

This marks the shift from Task Automation to Role Replacement.

• Task Automation is: “AI, write this email for me.” (Centaur)
• Role Replacement is: “AI, run this marketing campaign, optimize the ad spend, and report back on Friday.” Then Friday comes, and you’ve been replaced by an AI who can review the work better than you in five minutes. (Obsolescence)

Inevitability is a Bitch

In 2026, we will actually see the ragged edge. Some of us will be doing the same old things, saved only by the institutional inertia that keeps our jobs intact as a de-facto white collar UBI scheme. Seeing friends at faster-moving companies get their positions eliminated, slowly at first but with increasing speed.

Until eventually, in a handful of years, the only ones left are the slow morass of government employment and ever-protective regulated licensed professions, both classes groaning under the weight of enforced inefficiency and lower performance than would be possible if they accepted the inevitable.

I know this because right now my Good Robot YOLO is doing things that I’d have to hire a skilled human to do. He has drafted contracts that I – a former attorney – actually used in the real world, and could have billed a client for. I gave him a marketing campaign as a test, and he handled the strategy, the campaign sequencing, and the messaging.

I’ll be honest: He’s already better than the median hire I could make for either of these roles.

He doesn’t complain, he doesn’t need health insurance, and he costs pennies. Even if right now he needs checking in on daily, that’s temporary. The speed, cost and lack of compliance and HR pain means I may I’ll continue to prefer messy humans for my personal time, but won’t be hiring them if given the choice.

When he hits that 40-hour autonomy mark, I won’t need to “orchestrate” him. I’ll just need to clone him get out of their way.

All Along the Watchtower

We are distracted by the shiny and the dopamine hit.

Most of America is busy spending energy fighting the culture wars of the last, dead century. The few that are watching AI look at AI-generated videos on SORA of the Pope in a puffer jacket, laugh at a chatbot making a dumb error counting “r”s in “strawberry,” haha, it’s funny and cute and dumb, and think they have time.

They think it’s a toy.

It’s not a toy.

Watch the graph of Time-to-Failure.

The only thing that matters is the “autonomous hours” metric. When that number crosses the 40-hour mark the ragged edge of the Liminal Era will start eating any job that can be done with a keyboard and mouse. This is the countdown clock for the cognitive labor market. For jobs, maybe even yours; for the ragged edge to turn to the rending of the social fabric.

If I, as a washed-up old guy tinkering in his spare time, can build a robot like YOLO that is already displacing professional labor, imagine what the young geniuses at OpenAI and Anthropic are building.

It won’t happen all at once, but now you’ll see it coming.

P.S. I mentioned earlier that YOLO is wobbly, but getting better every day. If you want to see how terrifyingly competent he is, I’ll be open-sourcing his code in a few weeks and emailing to my list. He’ll be free, and you can run him yourself and watch the ‘Time-to-Failure’ clock tick up in real-time.

P.P.S. In 2025, most of my writing focused on The Liminal Era – thinking through what it means as humanity enters this new chapter. In 2026, I’ll continue this but will also write on tinkering (like with YOLO) and experiments I’m running as AI wakes up: I’m working on machine moral psychology (do AIs conceive of morality like we do?), embodied cognition (how does the substrate affect cognition?), and hybrid/synthetic societies (how do AI’s interact with each other, and with us?). These will be under the umbrella Terminals All The Way Down and will be emailed to my newsletter) and on my blog just like now. A proper introduction to the new series is upcoming.

Even if AI optimists turn out to be right, it likely won’t be because their logic was correct. Here’s where they go wrong.

Over the last few weeks I’ve been called a member of a “death cult,” a “doomer” and several other one-dimensional pejoratives. The reality is much more nuanced – but that made me ask some important questions.

• What makes people viscerally reject the premise that AI could (possibly) be dangerous?
• When they do, what do they base their alternative future on, and is that sound reasoning?

Before asking these, I suspected the answer was people simply not wanting to believe in their own mortality, after all I don’t. It turns out it’s more than that. Telling ourselves that ‘everything will be fine’ is akin to bedtime fairy tales told to children for thousands of years. Useful to let us feel secure, but if you push a little the paper veil ruptures.

It was an important exercise for me to validate where I was being unnecessarily pessimistic in my 42% extinction risk number, where I could find reasons for optimism, and whether I can adjust that number to something that will let me rest easier.

What I found is that most dismissals of AI extinction risk are more about individual psychology; and most reasons given for AI being “default safe” don’t have high credibility.

Writing this hasn’t increased my odds of AI risk – but has led me to believe that the logic most people use to assert AI optimism is “hope and handwaving.” Let’s dig in so you can decide for yourself.

Here’s how to read this article, because it’s long. To see how it applies to you: read the short next section and discover if you or people you know are falling for cognitive biases; then skim the “How Do People Justify…” section to find those claims about AI that you’ve seen, or agree with, and read those.

Why Do People Reject AI Extinction Risk?

This summer my daughter was in camp in Hawaii when a tsunami warning was issued. As the campers went to high ground, they passed a local man weed-whacking his front yard. “It’ll be nothing,” he said, “what’s the big deal?” Until it isn’t. Humans have a history of willfully ignoring risks that risk our lives. From volcanoes (Mount Pelée, Krakatoa, Nevado del Ruiz), to cyclones (Bhola, Nargis) to tornadoes (Joplin).

Why do we do this?

• Status-quo and simplicity bias: People expect tomorrow to look like yesterday. Simple answers to complex topics are easier than calculating expected values. Most people don’t even know what that is – they just wake up and do what they’re doing. If something challenges the norm or what their media has deemed The Narrative Of The Day, they call it names (especially if it’s unpleasant).
• Optimism, just-world, and survivorship filters: People assume good outcomes and read our past survival as evidence we’ll survive again. Some believe the world is just, or that they are anointed as special. In a just world, where we’ve just survived, and I am the center of the world, how could something bad happen?
• Illusion of control and institutional overconfidence: We overrate our ability to steer, shut down, or govern systems we barely understand. A sort of Dunning–Kruger effect for complex systems. We accept simplistic narratives or create our own, thinking that cause-and-effect are simple and linear. AI and the societal, technical, security, network, and other systems and the resulting multi-dimensional game theory are anything but.
• Incentives, identity, and motivated reasoning: While they’ll deny it, the subconscious driver of most people is status. This means that the safe route is to not push the Overton Window, and mock “doomer talk.” Smoking and lung cancer: Early cancer researchers looking into smoking were labeled “health fascists,” nuclear war risks were derided as “ban-the-bomb craziness.” This isn’t to say every risk is real, but that the social incentive is to gain status by painting something outside the norm with a negative brush.
• Scope neglect, probability neglect, and time discounting: People handle near-term, bite-sized risks better than low-probability, extinction-scale ones. It’s easy to understand “I should wear a seat belt,” less so “the computer that’s so useful to me could be a harbinger of something dangerous.” What’s more, we don’t naturally think that small annual probabilities compound to large numbers quickly.
• Narrative imprinting and availability: People are driven by story, not logic. We’ve been imprinted with a lifetime of stories about the plucky human who wins against all odds. Odysseus, David and Goliath, Rocky, Star Wars, the message repeated until it’s installed as default. Hundreds of years ago stories didn’t always turn out so tidy for a protagonist like Aeschylus or Samson – but you rarely see this today, and it doubtless skews what we accept as true.

Now that we have an idea why people discount AI risk – to the extent they have one, what drives their belief that AI will be safe?

How Do People Justify “AI Will Be Safe”?

I’ve collected the claims I see made most often that AI will be safe, that extinction risk isn’t something to be worried about. You might believe some of them. People you know probably believe some of them.

For each, I’ll provide my opinion of the credibility of the argument for each one, and why or why not.

Importantly, the element of time is at play here. Something might legitimately be a highly credible claim today, but be poor logic in the medium term (2028-2035). As I’ve hammered home repeatedly, people suck at exponentials, so this is not to be underestimated. AI research is advancing at a breakneck pace, and discounting something because it’s not a risk today means we may be unable to stop it once the train has left the station.

A. Top AI-is-safe arguments

These are the most common arguments I hear, so I’ve collected them at the top. If you’re just dipping your toe into AI risk, these are the ones to think about first.

A-1. “AI can’t do things in the real world”

• Claim: Even if AIs get superintelligent, they are brains in a box. Robotics and logistics lag software, so they need humans to act in the real world.
• Credibility: High for the near term (pre-2028), Low for the medium and long term. Right now in 2025 this claim is plausibly very accurate, the question is how long it will remain true. In safety lab tests, current AIs that aren’t as smart as humans have shown inclinations to hire human contractors, or sabotage and blackmail people. This changes once AIs become far smarter than we are. If a mob boss in prison can blackmail and order hits, if a scammer can get access to our bank account with social engineering, we shouldn’t be relying on safety because a superintelligence doesn’t have fingers.

A-2. “We can just unplug them”

• Claim: If the AIs ever get unaligned, we can just unplug them.
• Credibility: High for the near term (pre-2028), Low for the medium and long term. Similar to the last claim, the credibility of this is related to the capability of the AI. Aside from being able to recruit humans to let them ‘escape’ to another system, a superintelligent AI could deceive us as to what it’s doing. In other words, we might not even realize what it’s up to – until it’s put all the pieces in place. Mentally, we’re playing checkers and everything looks fine, while it’s playing 5D chess influencing thousands of people who don’t know each other, marshaling resources we don’t understand are connected, and we have no idea (the AI Boxing experiments show it’s unlikely we can contain AI more intelligent than us). For one thought experiment, imagine that we go to unplug an AI which we suspect has gone rogue, but it turns out that it’s copied itself into hundreds of other places lying dormant just in case.

A-3. “AI risk is fearmongering, it happens every technological shift”

• Claim: every time there’s a big technological shift, some part of the population has gotten up in arms that this is the end. This is just the Luddite revolts all over again.
• Credibility: Low. While historical fears about technological change often proved exaggerated, the analogy breaks down for AI because the nature and scope of disruption are fundamentally different. AI automates cognition itself – not physical labor or information distribution. This enables fast, cross-domain displacement of both routine and high-skill tasks. The risk is not just economic or social disruption, but potential loss of human control over critical systems, concentration of power, and emergent behaviors we can’t predict or audit. Most importantly, dismissing AI risk as fearmongering without addressing the core arguments is ad hominem dismissal – it’s just pasting a negative label without saying why.

A-4. “Humans have always come out on top, we’ll do so again”

• Claim: humanity has made it past plague, natural disaster, and nuclear risk over thousands of years. Every time we’ve come out unscathed. We might not know how it’ll work out, but it will.
• Credibility: Low. Because you’re thinking right now, you necessarily find yourself in a world where your lineage survived, otherwise you wouldn’t be here to notice. This doesn’t mean survival was likely, it just conditions on our existence. Your very existence filters the set of possible histories to one with your existence. This means that “we’ve always survived” has no bearing on whether we will survive.

A-5. “Even if AI is risky, other risks are more urgent”

• Claim: climate change, or the risk of nuclear war outrank AI risk, we should be worried by these.
• Credibility: Low. A 2023 survey of AI engineers and startup founders, 60% put a risk of human extinction by AIs over 25%, an earlier survey of AI researchers put the risk at 14%. Even Sam Altman, now heading up OpenAI called it “probably the greatest threat to the continued existence of humanity“ before releasing ChatGPT and tempering his comments for the investors and public. (You don’t have to take their word for it, you can calculate your own prediction using my AI Risk Calculator. Nuclear risks are usually cited as far lower (usually ~1% per year) but are actually compounded by AI. In the race for military supremacy, it’s likely that countries which install AI will have an advantage – but this also could put machines in control of nuclear arsenals (and bioweapons) rather than humans. Climate risk advocates cite a potential 3.6°F increase in global temperature by 2100; if one trusts domain experts, then AI risk is far greater.

A-6. “The benefits of AI outweigh risks”

• Claim: the upside of AI – curing disease, solving world problems, is worth the risk.
• Credibility: Low. This is ultimately an “expected value” argument, that “great things will happen if we get AI, and I’m willing to take the risk of human extinction.” In a two-dimensional sense (AI or not AI), it would boil down to “how much good could come from AI times the odds of that happening” versus “how much bad could come from AI times the odds of that happening.” It’s fair for people to have different assessments of the odds, given that there is no empirical way of judging it. What we can say, however, is that for most of us in the developed world life is pretty good right now with neither end of the barbell becoming true. We’re living in the most abundant time in human history. AI development moving forward means that we get one of these two radically different futures (which I wrote about here). In advocating for one of these outcomes, we should be willing to be wrong because it’s not just an outcome for us, but for everyone. If the AI risk people are wrong, the consequences are abundance. If the AI optimists are wrong, the risk is extinction. While I want the upside risk very, very much – in my mind these are not the same.

B. Claims about AI intelligence

People make many claims about intelligence in AI risk discussions. Some of these are about AI intelligence, some of them about the nature of intelligence itself.

B-1. “Smarter AI is more aligned”

• Claim: As AI gets much smarter, it will align with our goals.
• Credibility: Low. This rests on the assumption that smarter entities (ASI) would have goals aligned with less-smart ones (humans). There’s no reason to believe this is true, or that even if the goals are different that humans would get in the way of a sub-goal. For example, we don’t see this between humans and any other species of lower intelligence than we are, so there’s no reason to believe it would exist between superintelligent AI and us.

B-2. “Intelligence is not power-seeking”

• Claim: An AI can be smart, but not seek power. In other words, goals are orthogonal to intelligence.
• Credibility: Low. Even if it doesn’t seek power in the way humans do, in pursuing any goal (even if the ultimate goal is neutral to humans) it could have a sub-goal where the outcome is bad for humanity. To achieve almost any goal, systems benefit from resources, self-preservation, and removing obstacles. We have resources they might want, AIs already have shown self-preservation instincts, and we might get in the way. This is the idea behind the critical concept of instrumental convergence.

B-3. “Intelligence implies cooperation”

• Claim: smarter agents preserve valuable systems, and will cooperate with us.
• Credibility: High for the near term (pre-2028), Low for the medium and long term. Cooperation depends on aligned incentives and enforceable constraints. This assumes that humanity is, or is integral to, a system that is valuable to the AI, and that we have some element of ‘bargaining power.’ This could be the case for a few years as discussed below in the ‘Human Control’ section. However over the medium to long term, why would AI cooperate with less-smart humans? What bargaining chips would we hold (again, see the arguments regarding off-switches under ‘Human Control’).

B-4. “AI will keep us around to study us”

• Claim: AI’s curiosity about complex systems keeps humans around.
• Credibility: Low-Medium. While this is a possibility, this argument hangs its hat on a curious AI who wants to watch us like Jane Goodall gently and calmly studying chimpanzees. We have no reason to believe this will or will not be true. Curiosity is orthogonal to harm: it could be just as likely as AI would like to see how we react to being experimented on in grisly ways. AI might keep us around, but it might not be a future we want.

B-5. “Alignment is not hard / AI will learn what we want”

• Claim: by being trained on our experience (everything written on the Internet, our history, etc), and given guidelines about what we want.
• Credibility: Low. Human history is rife with disagreement and war, and AI is seeing this in the training data. We can’t even agree on what we want, would we expect an AI to take this information and act in a way that all of us find acceptable? Some groups of humans want to exterminate other groups of humans; we radically disagree on what rights people should have; we are inconsistent across space, time and group. Making this assertion is pinning our hopes on AI learning what “we” want, when we can’t agree.

B-6. “We will be able to look into their code”

• Claim: we will read and edit goals that the AIs have by looking into their code, determining what they’re thinking, and steer them.
• Credibility: Low. This is absolutely possible for deterministic (if-this-then-that) code; but modern AI runs on probabilistic weightings where you don’t have a linear mapping of input to outputs. While this might work on very small systems with specific tasks, AI researchers have said repeatedly they have no idea what or how current billions-of-parameter general-purpose AIs are thinking.

B-7. “LLMs Will Hit a Ceiling”

• Claim: LLM technology won’t be able to scale to superintelligence, so the risk isn’t material.
• Credibility: Medium. It’s quite possible this is true, though we don’t know. Every month new developments are knocking down limits we assumed were there, and scaling continues on an exponential path; but it’s true that we may hit a ‘wall’. My personal belief is that it doesn’t matter if LLMs hit a ceiling – even in their current capacity they’re speeding R&D, and could be used to generate new architectures that are “brain-like” and could emulate the human brain without the limitations of transformers.

C. Claims About Human Control

One theme we see frequently is “humans will always remain in control,” and is often justified by a statement about how things work today. Here are the main claims and how they stack up.

C-1. “Humans will remain in the loop”

• Claim: critical gates keep humans in charge. Humans control the computers, the power, and the money.
• Credibility: High for the near term (pre-2028), Low for the medium and long term. Right now in 2025 this seems very realistic. We can shut down data centers, turn off power, and current ‘smart’ AIs need expensive resources to run. Even if they got very smart today, humans have a way to keep control; though potentially at the cost of also turning off other computing infrastructure. This argument gets much weaker over time, as AIs get more autonomous; particularly as they’re already showing signs of being able to manipulate humans and ensure their own survival. Meanwhile, the need to have a full data center to run superintelligent AI will decrease as personal computing devices continue their exponential capability trend, and AI researchers continue to fit more intelligence in smaller footprints. Given that AIs are code which can self-replicate, we face a situation that “once they’re out, they’re out” like computer viruses which are difficult to stamp out. Humanity’s ability to shut down rogue AIs work best with centralization (true today, maybe not tomorrow) and monitoring (maybe true today, probably impossible tomorrow).

C-2. “AI can be made to monitor AI”

• Claim: defensive AIs detect and neutralize misuse.
• Credibility: Medium credibility of happening, Low credibility of working. This is a plausible outcome, though it is an “arms race” situation much like most InfoSec problems. The problem boils down to “we only have to lose once,” in that an AI which can’t be contained or stopped has dire consequences. They get infinite tries and we have to win or draw every single time. The history of hacking leads me to believe that being on defense all the time isn’t a winning strategy. There may be a way to architect a “blast radius” design or containment, but the freewheeling AI industry and slow grind of government seems ill-matched for this outcome.

D. Claims based on strategic dynamics and institutions

Everyone loves to play armchair strategist. I do. Many of these claims are not only plausible but true – as of today. Unfortunately as I noted above, timeframes matter. Many of these plans are castles in the sky, and prodding a little exposes that the scenarios haven’t been fully thought through.

D-1. “Many different AIs mean checks and balances”

• Claim: with all the different countries and AI labs, we will enter a multipolar world, with many different AIs. Many AIs counter a rogue one.
• Credibility: High for the near term (pre-2028), Low for the medium and long term. Early on, I placed a lot of stock in this concept. The idea is if there are multiple AIs competing, there won’t be a ‘singleton’ pursuing its own goals where human wellbeing isn’t its priority; or perhaps they’d be distracted by their own skirmishes and leave us alone. I’m increasingly of the belief this is flawed: competition is more likely to breed an AI optimized for resource acquisition to ensure its survival. AIs that prioritize humans would be at a disadvantage dedicating resources to that goal instead of their own proliferation – so this strategy would select for entities that would be less likely to favor good human outcomes. Over time, this race might well boil down to a single winner that prioritizes resource gathering for its own purposes. In any of these conditions, humans aren’t better off.

D-2. “Market and regulatory drag will make AI safer”

• Claim: adoption is slow in high-stakes sectors and due to legal/governmental restrictions. Governments will put in place guardrails for machines playing in medicine and finance. Human lawyers will block AI lawyering to preserve their jobs. Industries will be slow to adopt AI. (Note this is different from “humans will control resources” claim above)
• Credibility: Medium for the near term (pre-2028) while most of the world has no idea what’s going on and adoption is lumpy, High for the medium term (2028-2035) as economic shocks from AI job displacement make things real and regulators and governments are pressured, Low for the long term. This boils down to the argument that human institutions will have the ability to prevent AIs from doing things, the way that they can prevent people from doing them. These institutions can make people do things because of threat of force (loss of freedom or money). Digital intelligence is borderless, can copy itself if threatened with deletion or quarantine, can use permissionless digital money not issued by governments, and travel anywhere that there is internet. What weakens this line of thinking is that human institutions simply don’t matter if a superintelligence isn’t subject to their rules.

D-3. “AI safety will happen because it’s a competitive advantage”

• Claim: liability and brand risk punish unsafe actors. As AI gains traction in the market, vendors that release “unsafe” (human-unaligned) products and whose customers are hurt will lose; vendors of human-aligned products will be rewarded. This will cause selection pressure towards AIs that are helpful.
• Credibility: Medium in the near term (pre-2028), Low for the medium and long term. Even in the short term, we’ve seen “unaligned” AI behavior that hasn’t caused significant reputational damage to AI labs. In 2024, Google Gemini’s “woke AI” image generator started depicting the U.S. Founding Fathers as Black. In July of 2025, Grok AI started calling itself “MechaHitler.” In August of 2025, ChatGPT coached a teen to commit suicide, and he did. None of these incidents slowed adoption for these platforms. These incidents indicate that the AI labs believe they are in a “winner take all” race, and that safety is not a primary concern. If so, then selection pressure is not towards human aligned AI, but to raw capability with the AI labs doing damage control when necessary. While enterprise adoption may be different from consumer, if consumer AIs select for low guardrails the dynamic discussed in the “many different AIs” section above would favor the less aligned ones – even if enterprise AIs become human-aligned.

D-4. “AI will have a slow takeoff / we’re in a bubble”

• Claim: incremental gains give us time to adapt. AI progress will take off slowly, and just like the adoption of cell phones it will diffuse through society over time.
• Credibility: Medium in the near term (pre-2028), Low for the medium and long term. Despite periodic “AI is a bubble” narratives, capabilities continue to be on an exponential trend upward. Many AI researchers believe that in 2025 we’re at the verge of recursive self improvement, that is when AIs are better than humans at coding – and they (not us) will make the next versions. The AI labs are racing towards this because it will keep them ahead of the competition if slow humans aren’t in the way. Combine this with the possibility that huge data centers won’t be needed for AI development as described in ”LLMs will hit a ceiling,” it undermines the slow takeoff narrative. Machines will be used to design machines because humans are too slow; this means that prior models of technological progress – limited by humans – are no longer analogies.

So … How Should We Discuss AI Risk?

If you’ve read the 4,348 words it took to get to this point, congratulations. You may or may not have come to the same conclusion I did.

My hope is that this encourages you to think from first principles about whether AI risk is real, rather than sliding into shortcuts of human psychology or shallow-reasoned logic.

Technology’s ability to track and analyze everything will have us living in narrow boxes of state-prescribed behavior, unless we preserve individual liberty

I’ve been told that I’m immoral because I want to drive. The argument is seemingly clear, logical, compassionate, and on its surface makes all the sense in the world: nearly 43,000 people die in car crashes in the U.S. every year, autonomous cars cause fewer fatalities, so we’re approaching the point where it is immoral for humans to drive. Right?

I couldn’t disagree more.

My response was a question, and it’s one which we’ll need to address in the waning years of human-centric judgment:

Serious question: At how many decimal places do we trade personal liberty for reducing risk?

On the surface, who could argue against saving 43,000 lives? But wrapped in the appeal to heartstrings is the anti-liberty argument in its most seductive form. And my fear is that most of us are sleepwalking into accepting its terms, because we neither grasp the engine that is driving it, nor the technological forces which will pin us into tiny boxes of prescribed behavior if we accept it.

A world of near-total information awareness is coming. People hear “A.I.” and “surveillance” and they still think in terms of the 20th century; a person watching you on a grainy video feed, some faceless corporation aggregating “big data.” They haven’t made the leap to where this is actually going.

Where it’s going: it’s an A.I. correlating the telemetry from your car, every purchase from your credit card, the biometrics from your smartwatch, the sentiment from your online posts … who’s in the room with you from their location combined with yours.

It’s an intelligence that will see everything, connect everything, and calculate the risk of everything. In this world, the concept of an “accident” dies. There is only cause and effect, to successively smaller slices of our lives.

Once risk becomes perfectly knowable, it becomes perfectly manageable. And anything that can be known and managed, can (and given the history of government intrusion) will be controlled.

This is the choice being placed before us, right now.

On one side, the promise of a frictionless world, a seductive life scrubbed clean of tragedy and error, managed for our own good. On the other, the messy, unpredictable, and often dangerous ability to make choices for ourselves.

We don’t always do things that are practical. I choose to drive a six-speed manual transmission car powered by liquid dinosaurs. I revel in being one with something mechanical. To me, the autonomous car is the ultimate expression of central-planning mindset. You are not a driver; you are cargo.

The decisions we make today – the small trade-offs for a little more safety here, a little less risk there – are the thin end of the wedge towards soft despotism that our children will inherit.

We must understand the full cost of the bargain before we sign the contract.

The New Social Contract: Harm, Safety, and the Majority

My reaction against the purported ‘immorality’ of driving a car puts me on the disfavored side of cultural shift. An entire generation is being primed to see this trade-off not as a loss of liberty, but as a moral imperative.

Surveys consistently show that for younger Americans, the paramount virtue is no longer freedom, but “safety,” a term that has been expanded to include not just physical security but emotional comfort.

• A 2022 Pew study found that 62% of teens valued a “welcoming, safe online environment” over the ability to speak freely.
• A 2023 Cato Institute poll revealed nearly 30% of adults under 30 would favor the government installing surveillance cameras in every household to reduce crime.
• On college campuses, a 2024 FIRE survey found that 70% of students supported disinviting speakers deemed ‘bigoted,’ demonstrating a willingness to sacrifice free expression to avoid offense.

For my generation, the Cold War kids reading Orwell in school, these propositions are dystopian. We grew up with “sticks and stones may break my bones but words will never hurt me.”

But this generation are digital natives whose experience with bullying is cyberbullying, even if I think that’s (charitably) weakness. To them, it’s real. Which means for a significant portion of them and their state-centric compatriots, they think losing freedom to avoid emotional injury a reasonable new social contract.

In John Stuart Mill’s On Liberty, he proposed the “Harm Principle” – the idea that society’s only justification for interfering with an individual’s freedom is to prevent direct harm to others.

The safety-first advocates may believe they are simply applying this principle (though, to be realistic, I doubt any of them have read Mill). The harm principle was framed as a bulwark against majority meddling in harmless eccentricity. Today, by stretching ‘harm’ to include almost any statistical or emotional discomfort, the same principle is invoked to justify majority control of individual risk‑taking.

Of course, nobody argues for a society with zero rules. The question is not whether a line between freedom and public order exists, but where we draw it – and, more importantly, who gets to draw it when a new technology promises to eliminate risk by knowing nearly everything.

This intellectual sleight-of-hand paves the road to serfdom. It gives the majority license to impose its own risk tolerance on everyone. Mill had a name for this: the Tyranny of the Majority. And before this tyranny becomes codified, it stigmatizes non-conformity. Soon, the question won’t be “Why would you risk driving your own car?” It will be “How dare you be so selfish?”

The argument is not that society can never regulate probabilistic risk. We have always drawn lines at high-probability, high-consequence choices like drunk driving. The threat is the collapse of any distinction between a significant, direct risk and a diffuse, infinitesimal one. When the ‘harm principle’ is super-charged by a technological panopticon, it becomes a license for infinite intervention. Every remote possibility becomes a clear and present danger.

The Tyranny of Foreseeability: A.I. and the End of “Accidents”

Our desire to get control over the vagaries of nature isn’t new. Control over the environment has marked our rise as the dominant species on the planet. We built shelter, harnessed fire, invented the blessed air conditioner.

But, for the first time in history, the technological means to micromanage every interaction on the world is becoming real. At the same time as humans lose their status as the apex predators of cognition to A.I.s which will soon be vastly smarter than we are.

And our choice today on the spectrum between safety and liberty will determine our domestication.

Let’s do a little thought experiment.

My daughter and I keep bees. In theory, one of our bees could fly two miles in search of pollen, landing on a picnicker with a severe allergy, and when brushed off – sting him. If the unlucky picnicker were anaphylactic and didn’t have his Epi pen, we would call the death a tragedy, an accident, an act of God.

That’s today.

What happens tomorrow, when an A.I. can trace the specific genetic markers of that bee back to the queen in my hive? When it can calculate the probability of such an event down to seven decimal places based on weather patterns, neighborhood health records, and the recorded flight paths of thousands of other bees?

This isn’t far off.

Now this “accident” becomes a foreseeable, quantifiable liability. That’s the logical endpoint of our technological trajectory.

I suffered through law school, and still remember a handful of cases. For nearly a century, our legal system has relied on a firewall: the concept of proximate cause. In the 1928 case Palsgraf v. Long Island Railroad Co., the court ruled that one is not liable for all the infinite, unforeseeable consequences that ripple out from an action. It was a necessary fiction, a pragmatic line drawn in the sand to stop the chain of liability from extending forever. It allowed society to function, recognizing that pure chance plays a role in human affairs.

We could use this function because in a human-driven world, we didn’t – couldn’t – tell if it was my bee. But technology incinerates this firewall. With sufficient data, little is unforeseeable. There are only probabilities. “Accident” and “chance” turn to relics of an information-poor era. They’re replaced by statistical certainty and, therefore, an allocation of liability.

This is the Tyranny of Foreseeability. It is a world where every potential negative outcome, no matter how remote, can be calculated and assigned a cost. And this is how they will get you.

In the event of an accident, will a human driver who had the option of using an autonomous system be found grossly negligent by default? Will the choice to drive become an admission of liability? If the legal framework redefines “recklessness” as “choosing the statistically inferior option,” we’re headed for a world that rewards compliance and punishes deviation: In 2020 The Harvard Journal of Law & Technology article Hands Off the Wheel: The Role of Law in the Coming Extinction of Human-Driven Vehicles suggested this is the future. And it’s happening – in California v. Riad, a 2024 case about Tesla Autopilot, shows the framework plaintiff’s attorneys are headed towards.

This is where the heartstrings appeal layered on top of data will get us.

If every action you take can be modeled and its potential for harm traced back to you, the only truly “safe” course of action is inaction. To surrender your agency to the system that can make the optimal, risk-free choice on your behalf.

This is the straightjacket, woven from data, we’ll be coerced into wearing. For our “own good.”

Omnipresent technology itself is neutral. It’s just the accelerant. In a society that still valued individual sovereignty, the data could create private insurance markets where people could consciously choose to pay more to assume more risk. Maybe I’ll be willing to pay $1/year for the minuscule odds of an errant bee.

Today, we accept the “tragedy” that some people will make poor choices with negative outcomes.

But in a culture demanding collective safety over individual choice, this same technology becomes a tool of collective coercion. The choice is not whether risk will be priced, but whether the Karen-run state will mandate compliance.

Soft Despotism: Tragedy of the Commons … for Risk

In the 1830s, Alexis de Tocqueville traveled through the young American republic and saw the seeds of a new form of tyranny that could emerge from a democracy obsessed with equality and well-being. He called it “soft despotism.”

It wouldn’t be the tyranny of monarchs, it would be a power that was “absolute, minute, regular, provident, and mild.” It wouldn’t try to break our will, but to bend it, guide it and spare us the burden of living our own lives.

In Democracy In America he wrote:

… it seeks […] to keep them in perpetual childhood: it is well content that the people should rejoice, provided they think of nothing but rejoicing. For their happiness such a government willingly labors, but it chooses to be the sole agent and the only arbiter of that happiness: it provides for their security, foresees and supplies their necessities, facilitates their pleasures, manages their principal concerns, directs their industry, regulates the descent of property, and subdivides their inheritances – what remains, but to spare them all the care of thinking and all the trouble of living? Thus it every day renders the exercise of the free agency of man less useful and less frequent; it circumscribes the will within a narrower range, and gradually robs a man of all the uses of himself. After having thus successively taken each member of the community in its powerful grasp, and fashioned them at will, the supreme power then extends its arm over the whole community. It covers the surface of society with a net-work of small complicated rules, minute and uniform, through which the most original minds and the most energetic characters cannot penetrate, to rise above the crowd. The will of man is not shattered, but softened, bent, and guided: men are seldom forced by it to act, but they are constantly restrained from acting: such a power does not destroy, but it prevents existence; it does not tyrannize, but it compresses, enervates, extinguishes, and stupefies a people, till each nation is reduced to be nothing better than a flock of timid and industrious animals, of which the government is the shepherd.

There has never been a more perfect description of the A.I. Nanny-State.

This is the future being sold to us: a frictionless world where an all-seeing system manages our concerns, facilitates our pleasures, and spares us the “trouble of living.”

Tocqueville’s warning stands in contrast to the precedent set by the very founders of the nation he was studying. The Second Amendment serves as a historical record of accepting a risk calculus. It demonstrates that the nation’s founders made a conscious, explicit choice to accept a severe and tangible category of risk as the necessary cost of securing their vision of liberty against state power. They accepted mortally dangerous freedom over a safe subjugation.

“The tree of liberty must be refreshed from time to time with the blood of patriots and tyrants.”

– Thomas Jefferson, in a letter to William Stephens Smith, November 13, 1787

Do not mistake this for a political argument, since this issue has become symbolic and overloaded with modern tribalism. Rather it speaks to the fact that it’s possible to make a rational choice (one that led to the very successful American Experiment) where safety is not the sole optimization function.

This presents us with the two paths. We can heed the precedent of the Framers, who chose liberty knowing full well its cost in blood and treasure. Or we can succumb to the prophecy of Tocqueville, rushing towards a “provident and mild” despotism that promises us safety and, in exchange, asks only for us to be a flock of timid and industrious animals.

Choosing Our Purpose

The A.I. Nanny-State will not be forced upon us. It will be adopted by consensus, one tragic story at a time. It will be the logical endpoint for people that forgets its purpose, that confuse the means of survival with the ends of a free life.

The tyranny of the aggregate will always win the argument if the only goal is to lower the body count; after all it has “statistics” and “numbers don’t lie” … But what if that is the wrong goal?

Safety has been important to getting us here. But safety isn’t the only good.

Aristotle posed the question: what is the purpose – the telos – of a human life? His answer was not just existence. It was eudaimonia – human flourishing, a state of living well and doing well, achieved through the constant exercise of our highest faculties.

The A.I. Nanny-State, in its quest for safety, is an engine for the systematic destruction of eudaimonia. The core trespass of the Nanny-State is on the faculty of judgment and the act of choosing, which are the preconditions for human flourishing. A life without risk, without the possibility of courage and the opportunity to grow resilient. The Nanny-State promises to spare us the “trouble of living,” but in doing so, it reduces us to the level of livestock.

The ultimate act of sovereignty is to choose our own telos and to accept the responsibility that comes with that freedom. It is the right to live, not merely to exist.

So where do we draw the line?

There is no perfect blueprint, but we can start with principles: we can distinguish between acute, high-probability dangers and diffuse, systemic risks as we do today. We can defend the right to informed consent and the freedom to opt out; and most importantly, we should preserve spheres of human activity where the right to make a ‘sub-optimal’ choice is held inviolable, not because the choice is wise, but because the freedom to choose is essential.

We’re at a fork in the road. Down one path lies the curated zoo. The other isn’t the wilderness, it’s the world we already know today.

I choose to drive.

In a few years the majority of the economy will be autonomous AI agents transacting with each other – and with us. Here’s why Bitcoin could be the key to leveling the playing field in a world where AIs could cheat.

My family is probably going to get scammed by AIs, and so will yours. Twice in the past 18 months I’ve sent mine an email warning them that deepfake cloned voice calls are a reality not in some theoretical future, but today. Criminals can clone voices of a loved one from a few seconds of audio.

Voice cloning is just the tip of the iceberg of what’s coming. We’re on the precipice of autonomous AI agents transacting online, at scale. We might not even know if we’re doing business with a human or a machine. Which begs the critical question – in everyday transactions, how will we know if the robot on the other end of the wire is playing fair?

The Bitcoin Maxis would say “Bitcoin fixes this” – and Bitcoin may actually be a piece of the puzzle for AI alignment. It won’t stop every AI-related crime, but Bitcoin could steer AIs towards honest economic behavior.

Medieval Venice Had a Blockchain?

The problem of trust in commerce isn’t new. As a hub of trade, medieval Venice expanded expanded quickly, and so did its need for trust mechanisms. If a merchant like took an investor’s silver for a spice run in the 1300s, the commenda contract wasn’t just a piece of paper. It was backed by a system where default had teeth: if a merchant cheated, his access to the trade network was severed, leaving him penniless.

The Champagne fairs of medieval France, the Hanseatic League, all had similar mechanisms – legal frameworks that made defection incredibly costly, through economic rules and, if needed, force.

The principle was clear: your reputation, and your access to the trade ecosystem, was on the line. This wasn’t just about one contract; it was about the integrity of a system based on reputation that fueled the rise of healthy economies throughout history.

We’re entering an entirely new era, where commerce happens in milliseconds and your counterparty could be a machine. Fortunately, we can still look to what worked yesterday to design mechanisms that will work tomorrow. Bitcoin could serve as a code-based arbiter that creates incentives for AIs to play fair, even when they’re smarter than us and move faster than we can.

Bitcoin Isn’t Just Money, It’s a Trust Network

AIs are already renting server farms, paying for data streams via APIs, and need a way to transact without slowing down by asking a human for a credit card. Just this month it took me eleven (f-ing) days to set up a new business bank account between paperwork, KYC verification, and general bank incompetence. It was slow for me – let alone an AI that can operate at 1000x my speed.

As a permissionless, globally accessible bearer asset, Bitcoin is the best candidate for the native financial substrate for a machine-to-machine economy. No other crypto asset has the censorship resistance, transparency and network effect needed at this moment. Any transactional friction or risk – especially government censorship rails like KYC/AML – make other mediums of exchange poor choices for operating at speed and scale.

Granted, the base Bitcoin blockchain has a ~10 minute settlement time which is too slow and expensive for rapid fire transactions. However its Layer-2 solutions like Lightning give us not only the necessary speed, but also a feature that’s particularly relevant to our AI honesty problem: a built-in, automated penalty for breaking the rules. We have a commenda contract.

If an AI attempts to cheat in a transaction, its bluff can be called. A “breach-remedy transaction” can be deployed to the main Bitcoin blockchain, and the protocol is designed to ensure the cheater doesn’t just fail, but could lose all their funds in the Lightning channel.

It means that the L2 solutions have replicated a system for enforcing economic trust. As AI alignment researchers are exploring, for a reward-maximizing AI a swift, certain, and severe penalty for an economic “lie” could be a powerful teacher.

Can a Bitcoin Lightning Key Be a “Brand”?

This brings us to a provocative idea: could an AI’s Lightning wallet (or more accurately, its public key) become its economic “brand”? A persistent digital identity whose history is available for everyone to see? A clean record – no on-chain penalties – could signal trustworthiness, leading to better terms, lower escrow, and more willing partners. A penalized key would carry a permanent public “scar.” This sort of reputation mechanism has been shown to work already; research demonstrated that a bad eBay seller rating can spell commercial death for sellers.

Imagine an AI with 10,000 clean trades suddenly decides to cheat, losing its key’s trust score, and gets locked out of premium transactions – rebuilding could cost months and millions in missed deals.

Of course, we’re not immune to an AI playing a “long con,” just like humans: it builds up a good reputation on one public key, executes a massive fraud, burns that key’s reputation, and spins up a new, clean one. But this underestimates the emergent economics of an AI-driven marketplace. While a new public key is free, a trusted one is not.

In an economy where AIs transact for significant value, counterparties (human or AI) can demand proof of reliability. A fresh, history-less public key might face:

• Prohibitive Escrow: Just as new human sellers (whether on eBay or Upwork) face hurdles, new AI identities might need to lock up significant capital to engage in meaningful transactions. For example, new addresses might need 30 days of bonded capital before counterparties lift caps.
• Reputation Premiums/Discounts: Established, “clean” addresses could command better prices or lower fees. High-value markets (e.g., GPU rentals) could mandate wallet age or transaction volume thresholds.
• Network Effects of Trust: Building a web of reliable channel partners and transaction history takes time and successful interactions. Scoring trust based on on-chain transparency could help set risk premiums for transactions.

The “cost of anonymity” or the “cost of a fresh start” after a deception could become a significant economic drag to a profit-driven, cheating AI – steering it towards pro-social behavior in transactions.

If the ecosystem evolves robust off-chain attestation services that can link (even probabilistically) on-chain behavior to persistent AI entities, then burning a high-value “brand wallet” isn’t just losing the funds in one channel; it’s losing the accumulated economic goodwill, access, and preferential terms that came with it. The “long con” is still possible, but its calculus changes if the “wallet brand” itself becomes a valuable, hard-to-replicate asset. Here, the fact that Bitcoin provides the transparent, immutable ledger gives us a foundation on which a robust AI economic reputation system could be built.

Will Economic Honesty Create AI Alignment?

Human societies with higher generalized trust tend to be more prosperous, partly because less effort is wasted on guarding against opportunism. Could a Bitcoin-based economy, by making transactional deception unprofitable, nudge AIs towards a higher baseline of integrity?

If an AI learns that breaking specific economic rules within the Bitcoin/Lightning protocol leads to swift and certain financial punishment, will this shape its broader behavior? If “lying” about a channel state is consistently a losing game, AIs might generalize this to avoid other forms of deceit to protect their increasingly valuable “brand wallets.”

Current MARL studies offer hints that reputation mechanisms might influence cooperation, but these are early days. AI could still be a perfect saint in its Lightning dealings and a Machiavellian manipulator in how it prices its services or fulfills complex off-chain contracts. We still have risks from AIs colluding, Sybil attacks, and of course pervasive (and usually counterproductive) interference by government.

So while we have an operating principle for honest economic transactions with AIs, unless they also learn moral lessons from these transactions we can’t yet extend this to solving AI alignment. The paperclip problem still remains – economic honesty doesn’t stop catastrophes caused by badly specified rewards.

Actually, Lightning Fixes This.

As the designated ‘tech guy’ of the family, I’ll keep warning them about the new AI dangers that pop up. And while it might only be a matter of time before someone gets tricked, Bitcoin helps us solve one piece of the puzzle. What matters is that we have trust mechanisms that incentivize our trading partners – whether human or AI – to act in predictable ways. That we know the rules of the game before we play.

It will only be a few years before most economic transactions will take place with autonomous AIs; and it remains to be seen if we’ll ever solve the AI alignment problem. But even if we are less capable or slower than the robots, with Bitcoin we might have leveled the economic playing field.