The case for cross-model evaluation in autonomous research

Most autonomous research systems today are built on a single foundation model. The architecture is straightforward: a planner, a retriever, a synthesizer, and an output formatter — all running through one provider. The system inherits the failure modes of that provider almost completely. If the underlying model has a blind spot, the system has a blind spot. If the model has a stylistic bias, the system has a stylistic bias.

A research agent that consults only one model has not done research. It has consulted a single witness.

The single-model fallacy

Frontier models are remarkable, but they are not interchangeable, and they are not symmetric in their failure modes. Different models disagree on factual claims at non-trivial rates. They disagree more on long-tail topics, more on fast-moving topics, and more on topics with adversarial coverage. Single-model systems present whichever answer the chosen provider settled on, with no indication that other equally capable systems would have produced something different.

This is structurally unsafe in any setting where the consumer of the output cannot independently verify it. Which, in the deployments we actually care about, is most of them.

What cross-model evaluation does

Relvia treats every non-trivial claim as a candidate that is then re-derived against alternate models. The architecture is:

• Synthesize claims using the primary model with the retrieved evidence.
• Re-derive the same claims against alternate models, using the same retrieval context.
• Score agreement per claim, weighting by claim importance and source quality.
• Surface disagreement as part of the output, attached to the specific claim it concerns.

Crucially, models that disagree are not voted away. The disagreement is the signal. A claim where four frontier models substantially disagree under identical retrieval is a claim that should not have been presented as confident in the first place.

Cross-model agreement is the only credible cheap proxy we have for epistemic robustness. It is not perfect. It is much better than nothing.

What this changes for users

In practice, cross-model evaluation produces three concrete benefits. First, single-provider model regressions stop quietly degrading output. Second, claims where the underlying field is genuinely contested are surfaced as such, instead of being collapsed into a false consensus. Third, customers gain provider-independence: a model switch is a parameter change, not an architecture change.

The cost — and why it is worth it

Cross-model evaluation is more expensive than single-model generation. Latency is higher. Compute cost is higher. The infra is more involved. We make this trade deliberately. The thesis behind Relvia is that the right unit cost for AI-native research is not the cheapest output token — it is the cost of producing an output the customer can actually act on. Evaluated outputs are higher unit cost and dramatically lower total cost of ownership.

For high-stakes decision-making, that is not a close call.