Research Note: AI Architecture Decision and Matched Evaluation

Research Note: AI Architecture Decision and Matched Evaluation

Decision principle

Start with the smallest architecture that can pass the representative end-to-end task set at the required risk, quality, latency, and cost levels. Add components to address observed failure modes, not to satisfy an architectural identity.

Anthropic's current guidance recommends beginning with the simplest solution and increasing complexity only when needed. It distinguishes predefined workflows from agents that direct their own process. OpenAI's guide likewise separates single-agent and multi-agent orchestration and recommends establishing an accuracy baseline before optimizing model choice.

Matched evaluation

The single-model and toolchain candidates must receive equivalent tasks, source access, acceptance criteria, and reporting windows. End-to-end success is the primary outcome. Component metrics explain why a candidate passed or failed.

When decomposition helps

Orchestration is a reasonable candidate when the task contains independently verifiable stages, requires external actions or current information, benefits from specialized context, needs approval gates, or repeatedly exceeds one call's reliable planning or state-tracking ability.

It is a weak candidate when the task is short, the acceptance rule is simple, the single-call baseline already passes, or the extra components cannot be observed and tested. More calls create more places for stale context, schema mismatch, authorization error, retry loops, and partial completion.

Decision output

The architecture record should name the selected candidate, the measured improvement, the added operating burden, known residual failures, rollback path, owner, and refresh trigger. "Multi-agent" is not a result. "The orchestrated candidate improved task completion from one measured rate to another while staying inside the named latency, cost, and risk limits" is a result.