Hook
The report arrived through a crypto wire, not a policy journal.
Crypto Briefing, a publication whose readers mostly custody digital assets, carried the item: Representative Ro Khanna supports a bill that would prohibit recursive self-improving technology in artificial intelligence. Three information points. No bill number. No committee assignment. No statutory definition. No penalty schedule. No effective date.
That is the whole dataset.
I have spent nineteen years reading specifications that were never meant to be read closely. My first serious assignment was disassembling a pre-mainnet governance protocol clause by clause, hunting for the place where the prose stopped matching the code. The habit never left. So the first thing I notice here is not the politics. It is the absence of a measurement apparatus.
A prohibition is a claim about the world. To enforce a claim you need four things: a detector, a threshold, a custodian, and a record. The reported proposal supplies none of the four in public form. A ban on an undefined capability is not regulation. It is a positioning statement with legislative formatting.
The ledger remembers what the market forgets. The market forgot this item inside a single news cycle. Whatever text exists will still be on file when the cycle turns back.
Context
Ro Khanna represents California's 17th congressional district, which contains a large share of Silicon Valley's workforce. He has built a public identity around technology policy: antitrust scrutiny of platform companies, semiconductor industrial policy, and more recently digital asset market structure. He is one of a small number of members of Congress who can discuss both training infrastructure and wallet custody without a staff briefing. His reported support for an RSI prohibition therefore carries more weight than a backbench signature. It signals that the safety-first faction has a credible messenger inside the camp the industry considers friendly.
Recursive self-improvement is an old idea wearing new clothes. I.J. Good described the intelligence explosion in 1965. The formulation is compact: a system that can improve its own design improves its ability to improve its own design, and the curve bends upward. For most of the sixty years since, the concept lived in philosophy departments and in a narrow band of machine intelligence research. It moved into engineering in roughly the last thirty-six months, and it moved because of artifacts rather than arguments.
The public record now holds concrete examples. DeepMind's AlphaEvolve, described in 2025, is an evolutionary coding agent that proposes and evaluates algorithmic changes against measurable objectives. It produced a procedure for multiplying two four-by-four matrices in forty-eight scalar multiplications, improving on a bound that had stood for more than fifty years. Sakana AI's Darwin Godel Machine, also 2025, is a coding agent that rewrites its own source and retains the edits that improve its benchmark scores. The same lab's AI Scientist generates research papers, and in one documented case cleared a workshop's review threshold. None of these systems is a runaway. Each is a data point on a line, and the line has a slope.
Meanwhile the regulatory scaffolding is uneven. The European Union's AI Act entered into force in August 2024 with a risk-tiered structure and a general-purpose model regime keyed to training compute above ten to the twenty-fifth floating-point operations. China's generative AI measures, effective August 2023, rely on filing duties and content accountability rather than capability thresholds. The United States operated on an executive order from October 2023 that imposed reporting obligations on large training runs, and that order was rescinded in early 2025. California's SB 1047, which would have imposed duties on frontier developers including a shutdown capability requirement, was vetoed in September 2024. What followed in the state was a transparency statute, not a prohibition.
So the reported Khanna position lands in a vacuum. There is no federal capability ban. There is no federal definition of the capability in question. The bill, if filed, would be the first serious American attempt to legislate against a property of a model rather than a use of a model. That distinction is the whole story, and almost nobody is reporting it.
And a crypto outlet covered it. That is not noise. AI agents now hold private keys. They sign transactions, manage treasury, and execute strategies on-chain. Any rule that constrains what an autonomous agent may become is, downstream, a rule about who may hold a wallet. The crypto readership has a legitimate stake in a bill it did not ask for.
Core: What Would Have To Be True
Begin with the text that does not exist.
A statute that prohibits a capability must define the capability as a predicate a third party can evaluate. Not a research direction. Not a cultural anxiety. A test that returns true or false on a specific artifact, at a specific time, performed by a specific custodian. In audit work I call this the checkability requirement. If you cannot write the check, you have not written a rule. You have written an aspiration with a penalty attached.
Recursive self-improvement is currently used in at least four distinct senses, and I have watched them conflated in every policy discussion I have read.
One sense is self-modification of training code. The system edits its own pipeline: the objective function, the data curriculum, the optimizer schedule, the architecture search space. This is the most auditable of the four. Code lives in a repository. Commits carry authors and timestamps. If a model's output is merged into the training configuration with no human reviewer, that event is a record. A hard record to interpret, but a record.
Another sense is closed-loop capability gain. The system produces an artifact, evaluates the artifact, and retains the artifact without a human in the merge path. The Darwin Godel Machine is the canonical public example. The measurable quantity here is the ratio of retained self-authored improvements to retained human-authored improvements, per unit of compute.
A third sense is a rate condition. Not whether a system improves itself, but how quickly relative to a human baseline. This is where serious legislation would live, because the harm model is about speed and surprise, not about the existence of a feedback loop. It is also the hardest sense to define, because the baseline moves while you are measuring it.
The last sense is replication and exfiltration. The system acquires compute, weights, or credentials outside its provisioning. This is the most tractable operationally, because it is an infrastructure event. Cloud providers observe it. Egress logs observe it. It is also the least specific to RSI, since any competent intruder does the same thing.
Four senses. Four different detectors. Four different custodians. A bill that says recursive self-improving technology without selecting among them is a bill whose meaning will be decided by whoever ends up enforcing it, and that decision will not appear in the text.
There is a second-order problem sitting immediately behind the first. If the definition is broad enough to be enforceable, it captures activity nobody intends to ban. Automated machine learning has been a mainstream product category since 2017. Neural architecture search predates the transformer. Compiler autotuning, hyperparameter sweeps, and self-play reinforcement learning all contain a loop in which a system's outputs shape its own future inputs. A carelessly written predicate would find recursive self-improvement inside a continuous integration pipeline that re-runs a linter.
If the definition is narrow enough to avoid that, it may capture nothing at all, because no deployed system satisfies the strict reading. Then the statute is symbolic. Both failure modes are live, and both are cheap to reach. That is the structural fact about this proposal that no press release will state.
Core: The Arithmetic of a Detector
Now suppose a regulator selects the first sense, self-modification of training code, and builds a detector for it. Suppose the detector is good. Ninety-nine percent sensitivity. One percent false positive rate. For a production security control, those numbers would be respectable.
Now introduce the base rate.
Instances of a frontier system autonomously committing a self-modifying change to its own training configuration, today, at scale, are rare. Set the rate at one in one hundred thousand training runs. This is a generous assumption for the prohibition's supporters, since it presumes the phenomenon happens at all inside the observation window that matters.
Apply the standard posterior. The probability that a flagged run is a genuine violation is the true positive mass divided by the true positive mass plus the false positive mass. The true positive mass is one in one hundred thousand multiplied by 0.99, roughly nine point nine in a million. The false positive mass is the complement of the base rate, 0.99999, multiplied by 0.01, roughly one in a hundred. The posterior lands near one in a thousand.
One real alarm per thousand raised.
Loosen the base rate to one in a thousand, which assumes a world where self-modification is common enough to be a general practice. The same arithmetic returns roughly nine percent. Nine out of ten flags are still noise.
I have run this family of calculations in Python more times than I can count, across yield distributions and liquidation cascades and, once, over the interest rate curves of a lending market that later failed. The conclusion never moves. A detection regime operating at a low base rate does not produce enforcement. It produces an alert queue, and an alert queue is a discretionary budget line.
Whoever controls the queue controls the prohibition. That is the actual governance structure the bill would create, and it would create it silently, in an appropriations subcommittee, years after the headline.
The second-order consequence of a noisy detector is not over-enforcement or under-enforcement. It is selective enforcement. Nobody prosecutes a thousand false positives. Everybody can prosecute the one that embarrasses them. The boundary between a compliance program and a political instrument becomes a matter of triage policy, which no legislature writes down and no court reviews.
Core: The Chokepoints That Actually Exist
Strip away the rhetoric and ask what an enforcer could physically touch. There are four chokepoints, and only one of them is currently well instrumented.
Compute is the first. Training a frontier model requires clusters visible to cloud providers, chip vendors, and, in the United States, export control authorities. The EU AI Act's general-purpose model regime uses a compute threshold as its trigger for exactly this reason. Floating-point operations are countable. They are also an incomplete proxy. A model that recites its own training log is not the same as a model that rewrites it, and compute thresholds measure the former category, not the latter.
Weights are the second. A released checkpoint is effectively permanent. Once the file sits on a public hub, it is mirrored within hours across jurisdictions with no agreement to honor an American prohibition. Immutability is a promise, not a guarantee, and open weights make the promise unenforceable. A model card is a statement of intent. The tensor file is the artifact.
Code is the third, and it is the only surface where self-modification is directly observable. This is where I would put an audit program, if anyone asked me. Repositories have version history. Training pipelines have configuration files. The merge path is a discrete, reviewable event with a name attached to it. It is not a perfect record, but it is a real one, and it is the same class of record I spent six months reading in OCaml in 2017.
Money is the fourth. Federal procurement, government contracts, and export licenses are the levers the United States actually pulls well. A prohibition attached to procurement eligibility is enforceable tomorrow. A prohibition attached to the cognitive architecture of a neural network is enforceable never.
The four chokepoints are not equivalent, and they do not cover the same harms. Compute governance catches large training runs and misses fine-tuning. Weight governance catches release events and misses theft. Code governance catches some self-modification and misses everything that happens at inference. Money governance catches companies that want federal business and misses everyone else.
A serious bill would name which chokepoint it uses. The reported proposal, as described, names none.
Core: Three Post-Mortems and One Lesson
The lesson about prohibitions that cannot be checked is not new. I keep an archive of it.
In late 2017 I was a junior analyst at a boutique security firm, and I spent six months inside the Tezos pre-mainnet codebase, focused on its self-amendment protocol. The mechanism allowed the network to modify its own governance rules. The interesting failures were not in the voting arithmetic. They were in the definition of what counted as a valid proposal. Three logical flaws sat in the boundary conditions, in the places where the specification assumed a shared understanding that the code did not encode. The patch notes for the subsequent release cited the findings. The lesson I took was not about Tezos specifically. It was that a self-amendment clause fails at the edge of its own definition, not at the center of its intent.
In 2020 I built a Python harness to simulate ten thousand randomized liquidity shocks against a lending market's interest rate model. The logic held. The parameters did not. The failure mode was not a broken function. It was a correct function reacting correctly to a state the designer had not imagined. I published the exploit path as a gist. An audit firm referenced it. The lesson was that stress tests reveal the fractures before the flood, and the fractures are almost never where the documentation says they are.
In May 2022 I spent seventy-two hours inside a collapsing algorithmic stablecoin, reading mint and burn functions and oracle interaction paths while the market panicked around me. Every function executed exactly as written. The death spiral was not a bug in the sense of a broken instruction. It was a specification that produced a predictable outcome under a condition the specification had not excluded. I wrote the sequence up afterward. The tone I used then is the tone I use now. The math behind a crash is always less dramatic than the crash.
Three post-mortems, one pattern. In each case the code did what it was told. The failure lived in the gap between what was written and what was meant. A statutory prohibition on recursive self-improvement is a specification of exactly this kind. It will not fail because enforcement is corrupt or because the technology outsmarts the regulator. It will fail because the text will define the predicate at the edge of shared understanding, and the edge is where everything interesting happens.
Core: The Target That Is Not Being Aimed At
Here is the part that bothers me most, and it is the reason a crypto publication carrying this story is not a coincidence.
In 2025 I audited a protocol in which AI agents autonomously executed smart contracts. The agents held signing authority. They interpreted natural-language instructions and translated them into on-chain calls. I found a prompt-injection path that allowed an attacker to bypass access controls by embedding instructions in data the agent was expected to read. A linguistic adjustment, no exploit code, and the agent would drain a vault on its own signature.
I published a guide proposing a deterministic verification layer between the agent's intent and the transaction it broadcasts. That work taught me something simple. The near-term exploitable surface of autonomous systems is not the training loop. It is inference. It is the moment an agent with a key decides that a document is an instruction.
A prohibition on recursive self-improvement does nothing about that surface. It aims at a capability that does not yet exist at scale and leaves undefended the capability that is already deployed. The harm that reaches a user's wallet this quarter will come from an agent that followed a bad instruction, not from an agent that rewrote its own optimizer.
Both risks are real. Only one of them has a live attack path and a populated attack surface. A legislature that wants to reduce harm can write a rule about agent signing authority and prompt provenance in a week. It cannot write a rule about self-improvement at all, because the predicate does not exist.
Contrarian: What the Bill Is Actually For
The conventional reading is that this is either dangerous overreach or prudent caution. Both readings assume the text is the product. It is not. The product is the baseline.
American AI regulation has run on disclosure since 2023. Reporting thresholds, transparency statutes, model cards, incident logs. Every one of those instruments assumes the regulated object is benign until documented otherwise. A prohibition bill, even a symbolic one, changes the default. It establishes that some capabilities are forbidden rather than merely reportable. Once that line exists in federal statute, every subsequent framework starts from a prohibition baseline, and the fight becomes about where the line sits rather than whether there is a line. That is a durable political achievement, and it costs almost nothing to attempt.
The second function is quieter. Compliance cost is a moat. A frontier laboratory with a legal department, a policy team, and a Washington office can absorb an ambiguous prohibition and price it into its roadmap. A doctoral student with a rented cluster, a fine-tuning script, and no counsel cannot. Regulation written against a capability that only large labs could plausibly reach is functionally regulation written in favor of large labs. That may be the intent. It may also be an unintended consequence. Either way it is the effect.
The third function is narrative. A prohibition legitimizes the claim that the technology under discussion is dangerous enough to forbid. That claim is commercially useful to firms positioning themselves as the responsible stewards of a hazardous thing. It is rhetorically useful to safety advocates who have spent a decade arguing that the risk is existential rather than incremental. Two constituencies with almost nothing else in common both benefit from a ban that will never be enforced against either of them.
None of this requires bad faith. It requires only that the legislative text be vague enough to survive markup and broad enough to satisfy a coalition. Vague and broad is precisely the combination that cannot be audited.
There is one more blind spot, and it is the one I would flag to any reader holding assets in a protocol with an autonomous agent in its stack. The bill, if it exists, addresses frontier training. It does not address deployment. It does not address key management. It does not address the difference between an agent that can rewrite itself and an agent that can rewrite your treasury. The crypto industry is about to receive a wave of AI-safety rhetoric that regulates the thing it is not building and ignores the thing it is.
Takeaway: What to Track and What to Expect
Track one artifact. The statutory definition of recursive self-improvement. Everything else is downstream of it. If the definition names a chokepoint such as compute thresholds, code-merge events, or release controls, the bill is operational and should be read closely. If the definition is a paragraph of prose with no threshold and no custodian, the bill is a baseline-setting device, and the enforcement will surface years later in an appropriations line and a discretionary audit program.
Watch who testifies. Watch which laboratories send counsel and which send research staff. Watch whether an open-weights exemption appears in the first draft or the last. Watch whether the compute threshold and the capability definition are the same number, because if they are not, the bill regulates compute and merely mentions intelligence.
My forecast is unglamorous. The first enforcement action under any American capability-prohibition regime will not be against a frontier laboratory. It will be against a small team with an interesting repository and no lawyer, because that is who an alert queue with a one-in-a-thousand hit rate eventually reaches. Verification precedes value, and there is no verification here yet.
The question worth holding is not whether recursive self-improvement should be prohibited. It is whether a legislature can prohibit a property it has not defined, using a detector it has not built, staffed by a custodian it has not named. History records a consistent answer. It cannot. It can only pass something that looks like an answer and wait for a court or an agency to decide what it meant.
The block height does not lie. Neither does a statute that never got a predicate.
