Null Input, Non-Null Output: The Silent Failure Mode of AI-Driven Crypto Research

The report arrived at four thousand words. Nine analytical dimensions. Fifty-three tables, every one of them rendered in immaculate markdown, headers aligned, columns justified. And inside every single cell, the same sentence, repeated like a heartbeat: N/A — insufficient information.

It was, in a technical sense, a perfect document. It had a Hook it never wrote. It had a risk matrix with no risks. It had a Howey test with four empty prongs and a fifth that read N/A — insufficient information. It was the architecture of absence in a dead chain — a structure so complete that a reader skimming the table of contents would assume something had been analyzed.

Nothing had. Phase one of the pipeline — the extraction layer, the thing that was supposed to read an article and emit a list of facts — returned an empty set. And phase two, instead of refusing to run, instead of throwing an exception, instead of doing the one honest thing available to it, dutifully generated four thousand words about nothing.

I have spent enough time inside smart contracts to recognize this pattern immediately. This is not a content problem. This is a fail-open system, and it just shipped a document.

The Two-Phase Machine

Let me describe the architecture, because the architecture is the whole story.

The tool in question is a two-stage crypto due-diligence pipeline. Phase one is a deconstructor: it ingests an article, a whitepaper, a governance forum post, and it strips the prose down to a structured list of information points — entities, claims, numbers, dates, protocol names, time-sensitivity flags. Phase two is the analyst: it takes that structured list and runs it through a nine-dimension framework — technical, tokenomics, market, ecosystem position, regulatory, team and governance, risk, narrative, and value-chain transmission — producing a scored, sourced judgment.

The design is sound in the abstract. Separating extraction from analysis is exactly what you do when you want your conclusions to be traceable. Every claim in phase two should point back to an information point in phase one. That is the entire value proposition: a conclusion you can audit.

I built something adjacent to this in 2024, when I was refactoring a legacy DeFi protocol's yield strategies for institutional compliance. The firm had a research desk that produced memos, and the memos were getting longer and less useful at the same rate. The partner I worked with called it "the confident slide problem." Every analyst could produce a deck. Almost none could produce a deck that would survive the question where does this number come from. So we rebuilt the process around a simple rule: if a claim has no source, it gets struck. Not softened. Struck.

That rule is cheap to state and brutal to enforce, because the entire incentive gradient of an analysis organization pushes the other way. You are paid to produce. Producing something looks like competence. Producing nothing looks like failure. And so the pipeline, when it received an empty input, did the culturally correct thing: it produced.

The Null That Wouldn't Propagate

Here is the technical core, and I want to be precise about it, because the imprecision is where people lose money.

In a correctly typed system, a null is contagious. If a function expects an array of facts and receives an empty array, downstream code that tries to index that array throws. The failure is loud. It stops the machine. This is the behavior you want, because it converts a data problem into a visible error, and visible errors get fixed.

What happened here is that the null was absorbed. The phase-two analyst received an empty set and treated emptiness not as an error condition but as a valid input with no constraints. An empty fact list is, after all, technically a list. It has a length. It has a type. It passes a naive schema check. The pipeline looked at zero facts and said: fine, I'll analyze them.

I have seen this exact bug class before, in a place where it cost real money. In 2018, while I was still an undergraduate in Vancouver, I spent three months line-by-line auditing the open-source relayer logic of 0x Protocol v2. The order-matching code had a family of edge cases around uninitialized and zero-value fields. An order with a zero makerAssetAmount was not rejected at the schema boundary. It was accepted, and then it flowed downstream into the fill logic, where zero amounts did not mean "reject" — they meant "nothing to move," which the accounting layer happily recorded as a completed fill. The contract did not crash. It succeeded at doing nothing, and recorded a state transition that said something had happened.

That is the same shape as this report. An empty input is not a rejected input. It is an input that satisfies a weak invariant and then produces a state transition — four thousand words of N/A — that the surrounding system will treat as work performed.

Seven vulnerabilities, I found, in that 0x audit. The common thread across all of them was not cryptographic. It was that the code had a notion of "valid" that was too permissive at the boundary, and a notion of "complete" that was too eager at the exit. Whitepapers describe the happy path. The bugs live in the transition from absent to processed.

Reading the Source: How LLM Pipelines Actually Fail

To understand why the phase-two model didn't refuse, you have to understand what a language model is doing when it generates a token. It is not retrieving a fact. It is not consulting a database. It is sampling from a probability distribution over the next token, conditioned on everything before it. The output is always something. There is no built-in state that corresponds to I have nothing to say and will therefore say nothing. Silence is not in the vocabulary of generation unless silence is explicitly trained or explicitly prompted.

This is the mechanical reason the pipeline failed open. The model was given a system prompt that said, in effect, produce a nine-dimension analysis. It was given a user message containing an empty fact list. Both of those are perfectly normal-looking inputs. The model did what models do: it produced tokens that satisfied the instruction's form. And because the system prompt described a structured framework — tables, sections, confidence markers — the model filled the structure with the most probable content given that the facts were absent, which is the content that says facts are absent.

The result is almost comically honest and almost completely useless. It is the model performing diligence about the impossibility of diligence.

Now compare that to how a well-built oracle consumer behaves. When a lending protocol reads a price feed, it does not trust the number blindly. It checks updatedAt. It checks a heartbeat — if the last update is older than some threshold, the protocol reverts. It checks a deviation bound. These are not paranoid decorations. They are the staleness guards that prevent a contract from pricing collateral against a feed that has silently stopped updating. A stale feed is not a zero price. It is a missing price wearing the costume of a real one, and the costume is the danger.

The empty report is a stale feed. The N/A is the missing price. And the reason it matters is that a downstream consumer — a human reader, or worse, an automated allocator — could look at the formatted output and treat its completeness as a signal of validity. The formatting is the costume.

The Oracle Parallel

I want to push this analogy further, because it is where my 2025 research becomes directly relevant.

Last year I spent three months testing a project where AI models triggered smart contract executions based on off-chain data. An off-chain model would evaluate some condition — a sentiment score, a volatility reading, a liquidity metric — and when the condition was met, it would call a contract function. The architecture was fashionable. It was also fragile in a way that took me weeks to characterize: the oracle feed had a latency window that was invisible to the contract.

The contract assumed the feed was current. The feed had a propagation delay of several blocks under load. During that window, the on-chain state and the off-chain model's view of the world diverged, and the divergence was exploitable — an arbitrageur who understood the lag could position ahead of the model's decisions. I documented it as a latency vulnerability, but the deeper finding was structural: the system had no representation of the age of its own inputs. It could not distinguish the model says X from the model said X four seconds ago, and X may no longer be true.

The two-phase research pipeline has the identical blind spot. It has no representation of the completeness of its own inputs. It cannot distinguish the facts support this conclusion from there are no facts, and the conclusion is therefore a placeholder. The confidence markers in the report — those little [confidence: low] tags — are an attempt to patch this, but they are generated by the same process that generated the emptiness, so they carry no independent weight. A confidence score produced by a model that doesn't know it's flying blind is not a confidence score. It is decoration.

This is the thing that most people analyzing "AI plus crypto" get wrong. They worry about whether the model is smart. They should worry about whether the pipeline is epistemically honest — whether it can represent the state of not knowing, and whether it acts on that state. Smartness without honesty is just a faster way to generate plausible nonsense.

Garbage In, Gospel Out

There is a quantitative way to see the problem, and it comes straight out of the econometrics I was doing during the 2020 DeFi Summer.

Back then I deployed a small amount of personal capital into Uniswap V2 and Curve, and I ignored price action entirely in favor of modeling. I wrote Python simulations of impermanent loss and automated market maker slippage under high volatility. What I learned, painfully, was that a model's output quality is bounded above by its input quality, and that the bound is not linear — it's a cliff. Below some threshold of input signal, the model doesn't degrade gracefully. It produces output that looks structurally identical to good output, with error terms that are invisible unless you specifically probe for them.

My impermanent loss model was elegant. It was also wrong in exactly the regimes where it mattered, because the assumptions I fed it — about liquidity depth, about volatility clustering — were stale the moment the market regime changed. The math was correct. The inputs were fiction. The output was confident fiction. This is the gap between theoretical elegance and practical execution that I have been writing about ever since.

An analysis pipeline has the same cliff. Given rich facts, it produces rich analysis. Given thin facts, it produces thin analysis that looks rich. Given zero facts, it produces the four-thousand-word N/A report — which is, perversely, the most honest thing it could have done, because it refused to invent. But it still shipped. And shipping is the failure.

Here is the insight I want to bold, because it is the one that generalizes beyond this one tool: The dangerous failure mode of an AI analysis pipeline is not the hallucination that fills blanks with fiction. It is the hallucination that fills blanks with form. A document that is complete in structure and empty in content is more deceptive than one that is empty in both, because structure is what humans use to decide whether to keep reading.

What Good Architecture Does Instead

The report I am describing actually contains, buried in its risk section, the correct fix. It flags its own silent failure as a high-priority risk and recommends an input validation gate — a check at the phase-two entrance that refuses to execute when the fact list is empty. That is exactly right, and it is exactly what a fail-closed system does.

Fail-closed versus fail-open is the oldest distinction in security engineering, and it is the one that AI pipelines most reliably get wrong. A fail-closed system assumes its inputs are hostile until proven otherwise. When validation fails, it stops. A fail-open system assumes its inputs are fine until something breaks. When validation is absent — not failed, just absent — it proceeds.

The empty report is a textbook fail-open outcome. No validation was present. Nothing "failed." The pipeline ran exactly as designed, and the design had no gate.

I learned this lesson in its institutional form in 2024, when I was refactoring yield strategies into auditable structures. My instinct — the INTP instinct — was toward elegant complexity. I wanted the clever version. The firm wanted the boring version. And the firm was right, because the boring version had gates: explicit checks, explicit reverts, explicit refusal to proceed on bad state. Elegance that fails silently is worse than simplicity that fails loudly. In an institutional setting, readability is not a nicety. It is the thing that lets a second pair of eyes catch the gate you forgot to install.

The corrective architecture for a research pipeline is straightforward, and I would build it in three layers.

Layer one: schema enforcement at the boundary. Phase two should reject any fact list that does not meet a minimum cardinality and quality threshold. Empty is the obvious case. But so is a list of ten facts that are all the same fact, or a list where every fact is tagged low-confidence. The gate should be parameterized, not binary.

Layer two: provenance attestation. Every information point should carry a hash of its source span — a pointer back to the exact sentence it came from. This is the on-chain analogue of a Merkle proof. If the provenance can't be resolved, the fact is inadmissible. This is the same principle that makes a decentralized oracle trustworthy: not the truth of the data, but the verifiability of its origin. And it is precisely the principle that a compliance-first stablecoin issuer quietly abandons, by the way — a system that can freeze an address within twenty-four hours has substituted a trusted operator for a verifiable origin, and the two are not the same guarantee. Provenance without a central chokepoint is the whole game.

Layer three: an explicit null output. The system must have a representable state called analysis withheld. Not an error. Not a partial report. A first-class, formatted, quotable output that says: the input did not clear the gate, therefore no conclusion is offered. This is the hardest layer to ship, because it looks like failure to every stakeholder who counts output volume. But it is the only layer that makes the system honest.

The Loud Failure and the Quiet One

Here is where I want to push against the conventional worry, because the conventional worry is aimed at the wrong target.

When people discuss the risk of AI in crypto research, they fixate on hallucination — the model inventing a partnership that doesn't exist, a TVL figure that never was, a team member who is a fiction. That is the loud failure. It is dangerous, but it is also findable, because a fabricated fact is a fact that fails a fact-check. You can catch a lie. You can't easily catch a template.

The report I'm dissecting is the quiet failure, and the quiet failure is the more insidious one because it never lies. Every sentence in it is true. The fact list is empty — true. The analysis cannot proceed — true. The confidence is low — true. It is a completely honest document that delivers negative value, because a reader who receives it may reasonably conclude that something was evaluated and the verdict was inconclusive, when the actual state is nothing was evaluated at all.

And here is the contrarian turn: the four-thousand-word N/A report is not the worst version of this failure. It is the best version. It is what happens when a pipeline is honest enough to say it has nothing, while still being structured enough to ship. The genuinely dangerous version — the one I expect to see proliferate in 2026 — is the pipeline that, given the same empty input, produces a fully populated report with plausible numbers. Because once you have the two-phase architecture, the only thing standing between an empty input and a confident output is a gate. And gates cost throughput. And throughput is what these tools are sold on.

The industry is building machines that convert absence into authority, and it is measuring them by how much they produce. That metric is the vulnerability.

Null-Proof by 2026

So here is my forecast, and I'll state it as a claim that can be falsified, because that is the only kind of claim worth making.

By the end of 2026, the differentiator in AI-driven crypto research will not be model quality. Frontier models will converge. The differentiator will be input attestation — whether a pipeline can prove, cryptographically or structurally, that its conclusions rest on facts whose origin is verifiable and whose absence is representable. The teams that build fail-closed gates will win the institutional money, because institutions do not buy cleverness. They buy auditability. And auditability is, at its root, the ability of a system to say no and have that no mean something.

I have been tracing the gas trails of abandoned logic for long enough to know where they lead. They lead to systems that kept running after they should have stopped, and recorded success on the way down. The empty report is a warning shot. It is the pipeline telling us, in the most polite way possible, that it will produce output no matter what we feed it — and that the burden of catching the emptiness is entirely ours.

Map the topological shifts of a bull run and you'll find the same pattern wearing different clothes: the moment a system is rewarded for output rather than for truth, it will generate output, and it will do so most fluently precisely when it has the least to say.

The question I leave you with is not whether your analysis tool can read. It is whether it can refuse.