Skip to main content

Roll out an agent version

By the end of this tutorial you will have an agent whose new instructions went live through a staged rollout — served to a share of the traffic beside the old version, measured per run, and promoted in one call.

Five steps:

  1. Write a new version of the agent.
  2. Start the rollout — both versions serve traffic.
  3. Send traffic to the agent.
  4. See which version answered each run.
  5. Promote the new version, then prove it serves everything.

Every step is one API call, shown for all three clients. The ids in the responses are examples — copy the ones your own calls return.

Prerequisites​

export NATURALI_TOKEN=nat_sk_...
export PROJECT=proj_V1StGXR8Z5jdHi6B
export AGENT=agent_V1StGXR8Z5jdHi6B

1. Write a new version​

Any write that changes the configuration archives a new version. Change the instructions so the two versions are easy to tell apart in a reply.

naturali patch-agent \
  --project-id "$PROJECT" \
  --agent-id "$AGENT" \
  --instructions 'Answer in one sentence, then ask: Anything else I can help with?'
{
  "id": "agent_6a0WCYOBLQZOYvYz",
  "name": "refund-explainer",
  "instructions": "Answer in one sentence, then ask: Anything else I can help with?",
  "version": 2,
  "active_release": null
}

Version 1 is not gone. It is in the archive, and GET /v1/projects/{project_id}/agents/{agent_id}/versions lists both, each with the exact config it held:

{
  "data": [
    { "version": 2, "config": { "instructions": "Answer in one sentence, then ask: Anything else I can help with?" } },
    { "version": 1, "config": { "instructions": "Answer in one sentence. If you are unsure, say so." } }
  ],
  "total": 2
}

2. Start the rollout​

PUT /v1/projects/{project_id}/agents/{agent_id}/release serves two archived versions side by side: canary_percent of the traffic gets canary_version, the rest stable_version.

naturali set-agent-release \
  --project-id "$PROJECT" \
  --agent-id "$AGENT" \
  --stable-version 1 \
  --canary-version 2 \
  --canary-percent 50
{
  "id": "agent_6a0WCYOBLQZOYvYz",
  "version": 2,
  "active_release": {
    "stable_version": 1,
    "canary_version": 2,
    "canary_percent": 50,
    "promotion_gate": null
  }
}

50 makes both sides visible in a few runs. In production you would start much lower.

Who gets which version

A request that belongs to a session is assigned by the session's actor, or the session itself — the same end user always gets the same version, so a conversation never switches configuration halfway. A request with neither, like the ones below, is split at random.

3. Send traffic​

Run the same question a few times. ?wait=true holds the call until the run finishes, so the reply is in the response.

naturali create-agent-generation \
  --project-id "$PROJECT" \
  --agent-id "$AGENT" \
  --wait true \
  --messages '[{"role":"user","content":"What is our refund window?"}]'

Two of the four replies from this run, one per version:

{
  "id": "gen_8ADWHOnm2JtQC42C",
  "status": "completed",
  "output": {
    "content": "I am unsure, as I do not have access to your specific account information or terms of service."
  }
}
{
  "id": "gen_44nXn9OGAJaIr2wu",
  "status": "completed",
  "output": {
    "content": "You have 30 days from the date of purchase to request a refund for eligible items. Anything else I can help with?"
  }
}

The closing question gives the second one away, but a real change is rarely that visible. The next step reads the version from the record instead.

4. See which version answered​

Every generation records the agent_version that served it. GET /v1/projects/{project_id}/generations filtered by the agent lists them.

naturali list-generations \
  --project-id "$PROJECT" \
  --agent-id "$AGENT"
{
  "data": [
    { "id": "gen_44nXn9OGAJaIr2wu", "agent_version": 2, "status": "completed" },
    { "id": "gen_8ADWHOnm2JtQC42C", "agent_version": 1, "status": "completed" },
    { "id": "gen_CbbfP9Go13L5gRpP", "agent_version": 1, "status": "completed" },
    { "id": "gen_Cnxs18PPBFSzcYlP", "agent_version": 1, "status": "completed" }
  ],
  "total": 4
}

Both versions served traffic. If yours all landed on one side, run step 3 a few more times — the split is random for these requests. This field is what you group by to compare the versions: their answers, their errors, their cost.

5. Promote the new version​

POST /v1/projects/{project_id}/agents/{agent_id}/release/promote makes the canary the agent's live configuration and ends the rollout.

naturali promote-agent-release \
  --project-id "$PROJECT" \
  --agent-id "$AGENT"
{
  "id": "agent_6a0WCYOBLQZOYvYz",
  "instructions": "Answer in one sentence, then ask: Anything else I can help with?",
  "version": 2,
  "active_release": null
}

active_release is null: the rollout is over. Promoting again answers 409 NO_ACTIVE_RELEASE.

To prove it, run step 3 once more and repeat step 4. Every new run is version 2:

{
  "data": [
    { "id": "gen_Vh6dwiWrfojIvxGb", "agent_version": 2, "status": "completed" }
  ]
}
Rolling back instead

If the new version had done worse, POST /v1/projects/{project_id}/agents/{agent_id}/release/abort ends the rollout the other way: the stable version's configuration goes back live and all traffic returns to it. Neither call writes a new version.

What's next​