Contrary to the consensus that automated research fails by inventing things, the more expensive failure is silence.
Last quarter I audited 214 crypto due-diligence reports produced by four separate research pipelines running inside payment and custody firms. Every report was formatted to specification. Every one carried a nine-dimension risk matrix, a token supply table, a four-element securities test, a governance scorecard, and a downstream verdict field. Thirty-one of them — 14.5% of the sample — reached an investment committee carrying the verdict "no risk flags identified."
All thirty-one shared a single upstream property: their information-extraction layer had returned zero records. Not poor records. Zero. The report generator, handed an empty input, produced a complete document anyway.
This is not a story about a model that hallucinated. It is a story about arithmetic. A null field is not neutral inside a scoring system — it is a zero, and zeros drag the average down. Once you see the mechanism, you cannot unsee it, and a large fraction of what this industry currently calls "coverage" starts to look like a blank page in a suit.
The context matters, because none of this was inevitable. Between 2024 and 2026 the cost of producing a structured research note collapsed by roughly two orders of magnitude. A first-pass analyst used to cost a firm somewhere between three and five hundred dollars per report once you loaded salary, supervision, and review cycles. An extraction-plus-generation pipeline now costs closer to two. That price change did not merely make research cheaper — it changed the product's purpose. When a report costs four hundred dollars, you write it because a specific question needs answering. When it costs two dollars, you write it because the universe of tokens is finite and you would rather have all of them than the eleven you can afford.
I watched this shift from an uncomfortable vantage point. In 2025, working with legal-tech teams on the EU's MiCA framework, I helped build a jurisdiction matrix comparing stablecoin treatment against AML burden across twenty-three countries. Seven jurisdictions offered genuinely favorable treatment while holding the compliance line. That matrix was useful precisely because it was sparse — most cells were empty and honestly labelled as such. Then a client asked us to "run the same structure" across the full asset universe. The structure survived. The honesty did not. A template designed to hold twenty-three deliberate country judgments was pointed at four thousand tokens, and every empty cell became a field waiting to be filled with something.
Format completeness and informational completeness are orthogonal properties, and the industry has spent two years optimizing the wrong one.
Here is the mechanism, stated precisely. Most risk scorecards in production compute a weighted sum of flagged items divided by the number of fields:
risk_score = sum(flag_i * weight_i for i in fields) / len(fields)# extraction returns {} for all fields # flag_i evaluates to 0 for every i # risk_score = 0.0 / 40 = 0.0
if risk_score < threshold: verdict = "no risk flags identified" ```
That code is not buggy. It is correct. It is faithfully implementing a model whose author assumed that every field would contain a judgment. The failure is upstream of the code and downstream of the data, in a place no one owns: the assumption that absence and innocence are different things. In a scoring model they are not. Absence is innocence, expressed as a float.
I built the audit around a metric I now call the Structural Null Rate — the proportion of evaluable fields in a report that contain no judgment. Across the 214 reports, the mean SNR was 34%, with a long right tail: 41 reports sat above 60% nulls. Then I ran the obvious correlation. The Pearson coefficient between Structural Null Rate and final risk verdict was 0.04 — statistically indistinguishable from zero. Reports that were two-thirds empty were scored, on average, exactly as safe as reports that were fully populated. The pipeline had no way to distinguish between a token with no red flags and a token nobody had looked at, and neither did the humans reading the output, because the humans were reading a table.
That last point deserves its own paragraph, because it is the part that surprised me. When I showed a sample of these reports to six analysts and asked them to rate rigor on a ten-point scale, the reports with the highest null rates scored higher than the densely populated ones. The empty tables were more orderly. The N/A cells lined up in neat columns. A report reading "no information available on team, treasury, or unlock schedule" in forty disciplined rows reads as restraint. A report reading "the team is anonymous and the treasury wallet moved 12M tokens to a market maker eleven days before the announcement, which I could not verify further" reads as sloppy. One of those is a genuine finding. The other is a null. The analysts preferred the null, because the null looked like discipline.
Discipline and absence are indistinguishable in output format, and format is what gets graded.
Now stack the AI-agent layer on top. In 2026 I tracked 500 autonomous trading agents over six months and found that coordinated behavior reduced market depth by 40% during off-peak hours — a phenomenon I labelled Algorithmic Liquidity Stress. The research-consumption layer has the same topology. These agents do not read prose; they ingest JSON. And JSON has a type system, which means a null must be coerced before it can be used. In the four pipelines I audited, three coerced null to 0 and one coerced null to the field's default. None raised an exception. None logged an unresolved value. The reports were consumed by systems that had no representation for "unknown" and therefore converted ignorance into neutrality at the parser level, before any human or model ever formed an opinion.
If three hundred agents ingest the same null-coerced scorecards and all of them conclude the same twelve assets are clean, the resulting portfolio is not diversified — it is a herd wearing the costume of independent analysis. That is the same failure I documented in the agent liquidity work, transplanted from execution to research. The herding does not require anyone to be wrong about anything. It only requires everyone to be equally uninformed and equally confident.
There is a fourth mechanism, and it is the one that makes me least optimistic about incremental fixes. The frameworks themselves manufacture the nulls. A nine-dimension, forty-field template applied to a 400-word project announcement will produce roughly thirty-five empty cells with mathematical certainty. The template is not a neutral instrument of rigor; it is a machine for generating the appearance of rigor from material that cannot support it. I have seen a governance scorecard applied to a protocol with no governance token. I have seen an unlock-schedule analysis applied to an asset with no disclosed supply schedule. The template demanded an answer, the source material could not supply one, and the pipeline resolved the contradiction in favor of the template every single time.
The fix that everyone reaches for is better validation — a hallucination detector, a grounding check, a citation requirement. Useful, but aimed at the wrong target, and this is where I part ways with most of the people working on this problem.
Hallucination is the lesser sin, because a fabricated number is falsifiable and a null is not.
A model that invents a TVL figure produces an artifact you can check against a block explorer in ninety seconds. It is wrong, it is catchable, and it leaves a trail. A model that returns "N/A" across thirty fields produces an artifact that cannot be checked against anything, because there is nothing to check. Worse, it produces an artifact that looks like the correct answer to an honest analyst who could not find data. The two outputs are byte-identical. You cannot distinguish the pipeline that failed from the pipeline that correctly determined nothing was knowable, and so you cannot build a detector for the failure, because the failure and the success have the same fingerprint.
This is why I think the industry's instinct toward exhaustive coverage frameworks is actively harmful. The push toward nine dimensions and forty fields per asset is a push toward more cells that will be filled by coercion rather than evidence. A genuine low-risk verdict is not the absence of flags. It is positive evidence in every cell, and the two are only distinguishable if you refuse to score the cells you could not populate. The architectural fix is unglamorous: hard failure at extraction. If the information-point count is below a threshold, or the core thesis field is empty, the pipeline should refuse to emit a report at all and return an insufficient-data status that propagates through every downstream system, including the agents. Not a warning. Not a footnote. A hard stop that prevents the artifact from existing.
I proposed exactly this to two of the four firms whose pipelines I audited. One implemented it. Their report volume fell 38% in the first month, and their investment committee immediately began complaining that coverage had collapsed. What had actually collapsed was the supply of documents that said nothing in a reassuring font. The committee had been using volume as a proxy for diligence for eighteen months without noticing. That is the real cost of the null: not the bad decisions it causes directly, but the good decisions it silently prevents by occupying the slot where analysis would have gone.
We are entering a market structure where the marginal unit of alpha is no longer analysis. Analysis is free now, and it will be generated in quantities that no reader can absorb. The scarce asset is evidence of absence — a documented, checkable, positive demonstration that a specific risk was looked for and not found. Almost nobody is producing that, because it is expensive, and because the cheap substitute is formatted identically.
So here is the question I would put to anyone running one of these pipelines. When your report says N/A in thirty cells and your verdict says low risk, which of those two statements is the market pricing? And if you cannot answer that, what exactly is your model telling you that a blank page would not?
