Keep agent code model-independent so a model swap doesn't change behaviour

The problem

A multi-agent setup (an intent classifier, a couple of tool-using agents, a fallback path) is easy to build against one specific model version and its particular quirks: how strictly it follows a JSON schema, how verbose its tool-call reasoning is, what temperature it needs to stay deterministic enough. Baking those quirks straight into the orchestration code means upgrading the model later (a newer version, or swapping providers) risks silently changing agent behaviour in ways that are hard to catch in review.

The fix

Keep each agent’s model choice, temperature, and prompt template in config, not in the orchestration code, and write the orchestration logic against a stable interface (structured output validated by a schema) rather than a specific model’s exact response shape.

class AgentConfig:
    model: str
    temperature: float
    prompt_template: str

def run_agent(config: AgentConfig, input: str) -> AgentOutput:
    raw = llm_call(config.model, config.temperature, config.prompt_template.format(input=input))
    return AgentOutput.model_validate_json(raw)  # fails loud on a schema mismatch, not silently

Gotcha

Validating structured output against a schema catches a broken response, but not a plausible one that’s subtly different in style or reasoning depth - re-run the same eval set against a new model version before flipping the config, not just a schema check.