The Silent Mutation of Evidence: Tracing Citation Provenance in Agentic Biomedical Hypothesis Generation

Agentic pipelines write research hypotheses and attach citations to each one. They are judged by the hypotheses they produce, and a hypothesis with a plausible bibliography looks the same whether its author read those papers or something attached them afterwards.

In Mendel, it was the second. When the pipeline revised a hypothesis, the step doing the revising had been handed a different list of papers from the step that wrote the original, and the two lists usually shared nothing at all.

We instrumented the pipeline step by step without changing its code, logging every model call and every citation, and recording for each citation which list it came from and whether the model chose it or a heuristic attached it.

Then we tested the diagnosis directly. Before collecting any data we registered the analysis, then revised the same 90 hypotheses twice each, changing only which 25 papers the revising step was shown. Two people independently judged a sample of the resulting citations for relevance, without knowing which list each came from.

The revising step took the first 25 papers in the order retrieval returned them. Retrieval groups papers by source and sub-question, so that order says nothing about relevance.

The fix gives it the same ordering the generator already uses: by relevance, then recency. It applies to all three revision operators.

With only the paper list changed, the evidence a revised hypothesis cites is more closely related to the claim it supports, in every one of the 17 affected questions. On the one question where the two lists already largely overlapped, the change does nothing, as expected.

Two readers judged relevance independently of any automatic measure, and agreed with each other well. Across both, not one citation drawn from the relevance-ordered list was judged off topic.

Agentic systems for scientific hypothesis generation are evaluated on the hypotheses they produce, which cannot show where each hypothesis's evidence came from. We instrument a deployed biomedical pipeline operator by operator, logging every model call and citation without modifying production code. In 17 of 18 research questions the mutation step is shown the first 25 papers in retrieval order, disjoint from the 60 relevance-ranked papers the generator read, so nothing a mutated hypothesis cites of its own accord was available to the step that wrote its parent. A word-overlap heuristic then fills most of the bibliography from the wider corpus and masks the gap. In a pre-registered paired test that changes only that list, relevance-ranked candidates raise the similarity of chosen evidence in every affected question (0.721 to 0.793, difference 0.073 [0.051, 0.102]), with no effect in a negative control. The pipeline's own evaluator, the same model that wrote the hypotheses, rates the better-grounded arm higher (0.453 to 0.690) from four paper titles, a third of them supplied by the linker. Nothing errors.