Skip to main content
A/B testing answers “is the new prompt actually better?” with data instead of vibes: incoming calls are split between agent variants, each call is graded as usual, and you compare success rates per variant. There are two surfaces:
  • Per agent — tests are configured and tuned in each agent’s builder, on its A/B tab (/dashboard/agents/<id>?tab=variants). That’s where variants are created and the test is switched on.
  • Cross-agent overview — the Experiments page (/dashboard/experiments, under Quality & Testing) aggregates every agent’s test into one table.

​
Set up a test

From the Experiments page, click Set up an A/B test and pick an agent — you land on that agent’s A/B tab, where you create the variant configurations (one is marked control) and enable split testing. See Build an agent for the builder itself.

​
Read the overview

For each agent with a test, the Experiments table shows:
  • Status — Active (split testing on) or Paused.
  • Variants — every variant as a chip, with the control labeled.
  • Leading — the variant with the best success rate so far, with its percentage. Until variants have graded calls, this reads Not enough data yet — don’t crown a winner off a handful of calls.
  • Calls — total calls across the test’s variants.
Agents without a test are listed underneath as one-click nudges to start one. Clicking any row jumps to that agent’s A/B tab to tune or conclude the test.
Which variant handled a call is recorded on the call itself, so you can filter and spot-check each side’s transcripts in call history.
Prefer this built-in feature over hand-rolling A/B assignment in a dynamic config webhook — you get assignment, per-variant stats, and grading for free.

​
Next steps

Build an agent

Create variants on the agent’s A/B tab.

Simulations

Pre-ship testing with AI callers, before real traffic votes.