From the ashes of 2022, we planted seeds for 2030. I keep returning to that line, because it explains why I still read four-sentence news items at 2 a.m. Manila time — when the charts are flat, nobody is pumping anything, and the only honest signal left is what someone chose to leave out.
One such item crossed my feed recently. Anthropic has granted two outside AI safety organizations, METR and SecureBio, "inside access" to its systems. It mentioned a redaction policy that could affect public trust. Then it stopped.
I read it four times. Not for what it said, but for what it refused to say. No definition of "inside." No word on who decides what gets redacted, or whether the evaluators can publish anything the developer disagrees with. In a bear market we learn to read silence. This silence was loud.
The names matter. METR, formerly ARC Evals, is one of a handful of organizations on earth that tests frontier models for autonomy and self-replication risk — whether a model can acquire resources, evade shutdown, or sustain agency over long horizons. SecureBio works the other flank: biosecurity, testing whether a model can meaningfully uplift a non-expert toward dangerous biological capability.
Those two categories are not random. They map onto the triggers inside Anthropic's Responsible Scaling Policy, the framework governing when a model crosses into ASL-3 and beyond. Autonomy. Mass-casualty capability. In that document, evaluating these risk classes is not garnish — it is a structural step in the release process. This arrangement reads less like safety PR and more like the machinery of a self-imposed governance regime clicking into place.
Here is where my crypto instincts fire. We spent a decade learning that a claim of solvency is not solvency, and a claim of audit is not an audit. I learned it at nineteen, in a Manila hackathon room, the only woman among fifty engineers, listening to men describe whitepapers as though the PDF itself were the product. AI is walking that same road now — faster, with higher stakes, and without a chain underneath it to anchor a single claim.
The phrase "inside access" is doing an enormous amount of unexamined work. In AI evaluation, access is not binary. It is a gradient, and the gradient decides whether an evaluation is science or theater.
At the shallow end sits black-box API access: query the model, observe outputs, probe behavior. Useful for red-teaming, nearly useless for measuring dangerous capability — because capability hides behind refusal training and surfaces only under elicitation: fine-tuning, scaffolding, tool use, long-horizon agentic tasks. At the deep end sits white-box access: checkpoints, weights, training logs, fine-tuning privileges, and the compute to run elicitation yourself.
That distinction is the whole ballgame. An evaluator who can only send prompts is measuring the guardrails, not the model. An evaluator with checkpoint and compute access is measuring the thing that actually matters. The published report looks identical either way. One is evidence. The other is a trust statement wearing evidence's clothes.
I have spent enough years around audits — smart contracts, validator infrastructure, a DeFi protocol I advised through a painful migration — to recognize the pattern. When the audited party controls scope, schedule, and redactions, the audit becomes a document about the relationship rather than the code. I once watched a team celebrate a "clean" audit in which two entire modules had been quietly marked out of scope for timeline reasons. The findings were real. The coverage was fiction.
The compute question cuts deeper. Capability elicitation is expensive: agentic evaluation suites, fine-tuning to surface latent ability, adversarial bio-uplift scenarios all burn serious GPU hours. If Anthropic supplies both the checkpoints and the compute, the evaluator operates inside the evaluated party's infrastructure. Structurally, that is a tenant, not an inspector. Verification that depends on the verified party's resources is not verification — it is a permissioned ritual with a paper trail.
The methodological layer deserves equal suspicion. METR wraps models in scaffolding and runs them against software-engineering and autonomy task suites, calibrating against human baselines to estimate how much unsupervised work a model can sustain. SecureBio's uplift evaluations test whether a model hands a non-expert real gain toward harmful biological knowledge. Both are rigorous. Both are also judgment-heavy, low on quantification, and difficult to reproduce without the same harness, the same access, and the same compute. That is the failure mode DeFi walked into when interest rate curves were presented as market-derived mathematics — precise-looking parameters chosen by committee, defended by the fact that nobody outside could rerun the model.
Then there is publication. METR has long argued that evaluation organizations need independent publish rights: the ability to release findings without the developer's veto. Access without publish rights is an NDA with better branding. Redaction has legitimate purposes — protecting trade secrets, withholding an abuse blueprint, managing export-control exposure. But when the developer alone sets the redaction standard, transparency becomes a variable the developer controls. The public receives a trust claim it cannot falsify.
Falsifiability is precisely what our industry spent a decade industrializing. We did not solve proof-of-reserves by asking exchanges to promise. We solved it with Merkle trees, on-chain attestations, and the brutal simplicity of "don't trust, verify." AI has no such primitive. There is no chain to anchor an evaluation, no cryptographic commitment proving a report reflects the model as it existed on a given day. Everything rests on institutional design — which is exactly why the design deserves this much scrutiny, and why a four-sentence press note cannot settle it.
The comfortable reading is that Anthropic is behaving well and the skeptics are nitpicking. I want to resist that — and resist the opposite reflex too.
The genuinely counterintuitive point is this: the danger is not that Anthropic is lying. It is that the evaluation ecosystem is quietly becoming a paid service whose customers are the companies being evaluated. METR, SecureBio, Apollo Research, Andon Labs — small, grant-funded, talent-constrained. A contract with a frontier lab can be the difference between surviving and dissolving. That is not corruption. It is gravity. But gravity bends conclusions.
Crypto knows this shape intimately. Exchange-funded research. Pay-to-play ratings. Projects that hire the auditor, then announce the score. The mechanism was never a bribe; it was dependency. When the entity under review becomes the entity keeping the reviewer alive, independence stops being a property of the system and becomes a story people tell about it.
The second blind spot is cost. High-quality third-party evaluation is scarce, slow, and expensive. Make it the default gate for deployment and it becomes a moat. Capitalized labs absorb it; open-weight projects and small teams cannot. We have seen this film — compliance regimes drafted by incumbents that quietly outlaw the permissionless version of the same activity. The industry will call it safety. The effect will be consolidation.
So watch two unpublished parameters: the depth of access, and who holds the pen. If evaluators get checkpoints, independent compute, and the right to publish without veto, this is a real step — the institutional equivalent of a Merkle root. If they get API calls and a redaction policy, it is a press release with better posture.
The question underneath is not Anthropic's to answer. It is whether verification can exist in a system with no chain beneath it. We answered that once, in our own domain, with cryptography and stubbornness. Someone has to answer it again — and I would rather the answer come from the people who learned it the expensive way than from the ones who never lost anything at all.
