Split Testing

Split testing routes an agent’s live calls across variants — weighted configurations that override specific fields (prompt, voice, product, …) while inheriting everything else from the agent’s deployed config. Each call records which variant served it (agent_variant_id / agent_variant_label on the Call object), and the stats endpoint compares outcomes per variant. The control variant (is_control: true) represents the agent’s deployed config unchanged. Enabling split testing auto-creates a control if none exists.

Endpoints

Method	Path	Description
`GET`	`/v1/agents/{agent_id}/variants`	List variants
`POST`	`/v1/agents/{agent_id}/variants`	Create a variant
`PATCH`	`/v1/agents/{agent_id}/variants/{variant_id}`	Update a variant
`DELETE`	`/v1/agents/{agent_id}/variants/{variant_id}`	Delete a variant
`POST`	`/v1/agents/{agent_id}/split-testing`	Enable / disable split testing (staged as a draft)
`GET`	`/v1/agents/{agent_id}/variant-stats`	Per-variant call stats

Variant object

{
  "id": 41,
  "label": "Warmer greeting",
  "weight": 50,
  "is_control": false,
  "prompt": "You are a warm, upbeat support agent…",
  "voice": null,
  "product": null,
  "background_track": null,
  "thinking_level": null,
  "acknowledgement_prompt_mode": null,
  "acknowledgement_prompt": null,
  "inbound_speak_order": null,
  "outbound_speak_order": null,
  "silence_interval_seconds": null,
  "silence_max_checkins": null,
  "created_at": "2026-04-20T18:24:10.113Z",
  "updated_at": "2026-04-20T18:24:10.113Z"
}

Field	Type	Description
`id`	integer	Variant id
`label`	string	≤ 100 chars
`weight`	integer	0–100, default 50. Traffic is distributed proportionally to weights
`is_control`	boolean	At most one control per agent
`prompt`, `voice`, `product`, `background_track`, `thinking_level`, `acknowledgement_prompt_mode`, `acknowledgement_prompt`, `inbound_speak_order`, `outbound_speak_order`, `silence_interval_seconds`, `silence_max_checkins`	field type \| null	Overrides. `null` = inherit the agent’s deployed value; set a value to change just that field for this variant

POST requires label (everything else optional) and returns 201. PATCH accepts any subset; DELETE returns 204.

Enable / disable split testing

curl -X POST https://api.thunderphone.com/v1/agents/12/split-testing \
  -H "Authorization: Bearer sk_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"enabled": true}'

Field	Type	Default	Description
`enabled`	boolean	`true`

Response

{
  "split_testing_enabled": false,
  "draft_split_testing_enabled": true
}

The toggle is staged as a draft on the agent — live traffic starts (or stops) splitting only after you deploy. Enabling with no control variant auto-creates one (label: "Control", weight: 100).

Variant stats

curl https://api.thunderphone.com/v1/agents/12/variant-stats \
  -H "Authorization: Bearer sk_live_YOUR_API_KEY"

Response

[
  {
    "variant_id": 40,
    "label": "Control",
    "is_control": true,
    "calls": 132,
    "success_rate": 84.1,
    "avg_duration_seconds": 163
  },
  {
    "variant_id": 41,
    "label": "Warmer greeting",
    "is_control": false,
    "calls": 129,
    "success_rate": 88.4,
    "avg_duration_seconds": 171
  }
]

Field	Type	Description
`calls`	integer	Total calls served by the variant
`success_rate`	number \| null	% of graded, completed calls with `call_outcome="success"` (ungraded calls excluded); `null` when nothing graded yet
`avg_duration_seconds`	integer \| null	Mean duration of completed calls

Agents

split_testing_enabled lives on the agent (draft/deploy).

Calls

Filter call history by agent_variant_id fields on each call.

Overview

Organizations

Agents

Phone Numbers

Calls

Billing

Developer

Endpoints

Variant object

Enable / disable split testing

Variant stats

Agents

Calls

​Endpoints

​Variant object

​Enable / disable split testing

​Variant stats

​Related

Agents

Calls

Endpoints

Variant object

Enable / disable split testing

Variant stats

Related