What’s new at Roark

Grade what actually happened in your backend, not just what the agent said. Every other judge reads the transcript, so an agent that confidently says "your PIN has been updated" after a failed tool call passes them all. API checks ask your own endpoint whether the call really succeeded.

How it works:

• Point it at your endpoint: URL, method, headers and an optional JSON body, with call or simulation values filled in through {{call.properties.cardId}} or {{simulation.variables.newPin}} placeholders.
• Pass or fail with a reason: your endpoint answers { "passed": true | false, "reason": "..." }, and errors on your side (timeouts, 5xx) are recorded as errors, not fails, so your pass rate does not track your endpoint's uptime.
• Runs on live calls and simulations: alongside threshold, pattern, formula and LLM judge metrics, with no extra setup.

Also available in config as code (calculationType: HTTP_REQUEST).

👉 Read the API Checks guide

Each board (Speech-to-Text, Language model, Text-to-Speech, Speech-to-Speech) now shows a ranked chart, a sortable table and trade-off charts, with every figure visible at a glance. The launch post is live at roark.ai/blog/voice-ai-benchmarks.

Also this week on the bench:

• Grok Voice 3.0 added as a speech-to-speech stack beside 2.0.
• GPT-6 Luna and Sol running on the GPT Live stacks at medium effort.
• Smarter tool-use grading: a narrated "checking that now" that ends in an honest "I can't" is a minor issue, not a critical hallucination. Confident false outcomes stay critical.
• Score hover on the board tables shows the full points-per-figure breakdown.

👉 Browse the boards

Define thresholds in code and filter dashboards by any metric's value or by a threshold's pass or fail, same as in the app. Config as code now covers the whole surface.

What you can do:

• Thresholds in config bundles: including thresholds on thresholds, which apply in dependency order.
• Filter dashboards and widgets on a metric's value: by metric slug, for the agent or customer side.
• Filter by a threshold's pass or fail: on dashboards and widgets, matching how reports already work.

Keys: THRESHOLD, METRIC_VALUE, PASSED / FAILED.

👉 Read the config as code guide

Ask Claude about your calls, metrics and simulations, and have it create agents, personas and flows or run a simulation, through Roark's hosted MCP server at https://mcp.api.roark.ai/mcp. Named read and write tools replace the single execute, ready for Claude's connector directory.

What's exposed:

• Read: projects, agents, calls, transcripts, metrics, simulation results, personas, flows and docs.
• Write: create agents, endpoints, personas and customer flows, and run simulations.

list_calls also gained agent, property and sort filters, so an assistant can slice a project's call history the same way the UI does.

👉 docs.roark.ai

You can now author your entire Roark config in TypeScript, with typed references and autocomplete, and ship it with the same CLI you use for YAML.

What's new:

• Author agents, tools, flows and plans in typed TypeScript, with inline prompts and autocomplete across the whole config.
• Reference other entities by variable instead of string names, so a rename or a typo surfaces before you deploy.
• Apply from the CLI the same way you do YAML, with a diff preview before anything changes in your project.

👉 Read the TypeScript authoring guide

Your agents and your CI can now read the same numbers your dashboards show, define dashboards in code, and have a widget's query round-trip cleanly between the API and the platform.

What you can do:

• POST /v1/analytics/query for ad-hoc analytics, with GET /v1/analytics/schema so an agent or script can build a valid query on the first try (metrics, filters, breakdowns, intervals, display types).
• /v1/dashboard CRUD plus /data and per-widget /data to create dashboards, edit single widgets by name, and run every widget server-side.
• kind: dashboard in Config as Code, so a dashboard defined in git shows a Managed by config pill in the platform and stays in sync on every apply.

👉 Read the analytics API guide

Simulation plans can now send each run's Debrief by email (PDF attached) and post it to Slack (PDF uploaded into the thread), using the same shape saved groups already use.

What's new:

• Define recipients once on the plan, in config or on the Report delivery panel in the dashboard, with up to 50 email addresses (no Roark account required) and 10 Slack channels.
• Run a saved plan by its config key, not just by its UUID, so your CI step and your YAML stay name-based end to end.

👉 See the report delivery docs

You can now pull Roark spend into your own dashboards with a project-scoped credential that reads only its project's cost, and break spend down to a single call or a single simulation run.

What's new:

• GET /v1/usage with a project API key that holds the new billing:read grant: scoped to that project, with a 403 (never a silent retarget) if a caller names another project.
• GET /v1/call/{callId}/cost returns a call's billable spend by source and surface, with the cost lines the Cost tab already shows.
• GET /v1/simulation/plan/job/{jobId}/cost returns a run's total spend plus per-simulated-call cost.

👉 Read the usage reporting guide

A new grouped metric that catches voice problems only visible across calls: one sentence of your prompt read in a different voice in most calls, while any single call looks fine.

How it works:

• Add Voice Consistency by Sentence (or its check) to any metric collection job of 6 or more calls, and Roark groups lines by normalised sentence, flags any that break far more often than the rest of the group, and writes a per-turn explanation pointing at the sentence as the likely source.
• Calls are compared only within the same agent and endpoint, so an A/B test on two numbers doesn't pollute the baseline.
• Deterministic, so it's free to run alongside your existing metrics.

The public voice-agent leaderboard now runs every stack through four full industries, scores tool use alongside speech, and ranks on a weighted quality score with a price switch.

What changed:

• Four scenarios, five callers each. Reschedule a healthcare appointment, triage an urgent home repair, file an auto claim, and troubleshoot an internet outage. The same five callers (US quiet, Spanish-accented on a street, Indian English in a café, flustered Southern with a crying baby, fast and interrupting under construction) run every scenario, so industries and voices stay comparable.
• Tool use is a first-class figure. Every stack is graded on whether its agent actually called the right tools, including hallucinated calls and skipped ones. Boards for LLM and speech-to-speech stacks now show tool-call completion and tool hallucination.
• Voice consistency and same-speaker checks. We flag lines where an agent drifts off its usual voice or the wrong speaker lands on the agent track, and the board scores on both.
• Semantic word error rate for transcribers. The raw rate is still shown, but ranking uses a judged rate that forgives differences a listener would read the same way.
• Weighted Roark score with a price switch. Pick quality-only or price-adjusted, filter by industry or caller, and open any model or stack on its own page beside the board.

👉 Browse the live benchmarks

Run several simulation plans together and read them as one report. Keep the whole thing in config as code with scheduled delivery when the group finishes.

What's new:

• Run groups. Group existing runs, or start several plans at once from the dashboard. One combined report with an executive summary, numbered issues, strengths, metric aggregates, and a PDF export.
• Groups in config as code. kind: simulationGroup collects plans by name, schedules them on the same cadence block as plans, and announces each run to any mix of email recipients and up to 10 Slack channels (PDF in the thread).
• Schedule any plan from config. A schedule block on kind: simulationPlan runs it on cadence. The dashboard view goes read-only when the schedule is config-owned.
• Full REST surface. POST /v1/simulation/run-group to start a group, GET to read its full report document, regenerate on demand.

👉 See the simulation groups guide

A collector in kind: collector can now carry each metric's per-collector settings in the file, and you can re-score past calls using exactly those settings over the API.

What you can do:

• Pin settings in YAML. For each metric a collector runs, declare verifiedProperties, excludedPhrases, voicemailScript or screeningBrief inline. The file is the source of truth: an apply resets anything drifted in the app, and misspellings fail validation.
• Re-score past calls with a policy. POST /v1/metric/collection-jobs takes an optional policyId. The job runs each metric at the version and settings that collector runs it at live, so a re-collection against an updated Property Verification setup checks only the properties you now care about.
• Choose which properties a collector verifies. Property Verification's "Properties to verify" is now a first-class collector setting, visible everywhere the collector is edited.

👉 Read the collector config reference

Dashboards now take a filter that applies to every widget, plus 2 to 3 comparison groups (say, agent A vs agent B) that split every widget on the dashboard at once.

Highlights:

• Dashboard-level filters. Set them once, every widget respects them.
• Compare mode. Each group becomes a coloured series on every chart, sharing the same axis instead of sitting in separate cards. Setup is guided: the first filter opens group A, the next populates group B, and the popover closes when every group is filled.
• Lives in the URL. Share a link to a specific comparison without saving it onto the dashboard.

A batch of upgrades for custom metric authors.

• Reference a single property in a prompt. {{properties.partner_outcome}} resolves to that one value and shows up as its own line in Data Sources Used. Mix it with {{properties}} or use it on its own; the Studio picker now lists every custom property on the project.
• Grade per agent reply or per line. Custom LLM metrics can now score each agent response or each transcript line, and failed checks land on the exact turns the judge cited.
• Archive and restore. Hide a custom metric from the library, pickers, collectors and dashboards, and restore it later. The dialog previews everything the archive touches before you confirm.
• Longer prompt cap. Custom metric prompts now accept up to 20,000 characters.

Your agent rarely gets a clean hello anymore. Apple and Google screeners answer first and decide whether to connect, decline, or take a message, and enterprise numbers gate the call with an IVR or an identity check. You can now model any of these as a scripted opening, chain them in front of your improv flows, and run the whole thing end to end.

What's new:

• Screeners as scripted openings: build the front door once (Apple/Google-style "who's calling and why", an IVR menu, an identity check) and link it as the opening for many improv flows. Test the connect path, the decline path, and the take-a-message path as separate branches off the same screener.
• Ask Roark builds the pattern for you: point it at a set of calls that share the same opening and it will split out the scripted screener, link it back to the improv flows, and prompt for the variables at run setup.
• Faithful silence handling: when the screener or IVR expects a pause before its next prompt, the simulator now waits with it instead of speaking early. Fixes the class of run where prompts appeared to play that were never in the test case.
• Clearer step roles in the editor: scripted customer nodes are labeled as Roark's simulated caller, agent nodes as expected responses, and First message is only offered where it belongs.

roark auth login now opens your browser instead of asking for a pasted token, and Config as Code can define alerts alongside your agents, flows, and metrics.

What's new:

• Browser login (OAuth + PKCE): approve in the platform, no more copy-paste tokens. --paste still works, and piped stdin still takes a token so CI is unchanged.
• kind: alert in config: version-control threshold, event, and simulation monitors, with Slack and webhook delivery.
• Cleaner diffs: unchanged config-managed resources show as noop in roark diff, instead of scrolling past a wall of identical lines.
• API-key scope: Config as code is now a first-class permission in the picker.

👉 See the Config as Code overview

Simulate an IVR from the agent's side: script the digits you expect the agent to press, capture them live, and grade them automatically.

What you can do:

• AGENT_DTMF flow steps: add them anywhere in a scripted flow to declare which keys the agent should press at that point.
• Deterministic scoring: the new dtmf_input_correct metric compares position by position and fails wrong-order sequences, which is exactly the wrong-branch case reverse-IVR testing exists to catch.
• Live view: keypresses render on the agent's side of the transcript as [DTMF: 1 2 3], distinct from anything the agent said out loud.

The customer API picked up pagination, variants, and a proper async endpoint for tool invocations, plus a few quality-of-life polish items.

What's new:

• Paginated metric definitions: stop pulling the full list on every fetch.
• Metric variants exposed: list a metric's variants directly, and get a helpful response instead of a 403 when you pass a variant to a metric that doesn't support one.
• Judge rubric + live-only flag returned on GET /metric-definitions, so integrations don't have to look them up separately.
• Async tool-invocation append: attach tool calls to an existing conversation after the fact.
• enrichWithLiveConversation accepted on run creation, and unknown keys no longer get silently dropped.
• OpenAPI served at the standard path, where every code generator already looks.

👉 Browse the API reference

A batch of authoring improvements across simulations and personas that add up to a much smoother workflow.

What's new:

• Knowledge Base Grounding template: pre-attaches faithfulness, ungrounded-claim, and missed-answer metrics; pick which KBs to score against and go.
• Bulk-add run calls to a dataset: promote a whole run's worth of calls in one action.
• Auto-attach Agent Expectations: any run whose flows declare expectations picks them up without a manual step.
• Personas get folders and drag-to-reorder: the picker leads with your recents and the workhorse personas, custom ones sort in the order you set on the hub.
• Ask Roark is folder-aware for flows and personas, and can file things away as it creates them.
• Inline metric config wherever a metric is attached, not only at run creation.

Your flows, metrics, personas, and thresholds can now live as YAML in a directory you own, versioned in git and reconciled against Roark with a single command.

What you can do:

• Author locally: keep every flow, metric, persona, and threshold as YAML alongside your agent code, with file:// inlining for prompts so the long strings stay readable.
• Preview before you push: roark config diff ./roark shows a + ~ - summary of what would change, no writes, safe to run in a PR check.
• Apply with a confirmation: roark config apply ./roark runs the diff, prints the change set, and asks before it writes. --yes skips the prompt for CI.
• Prune by default: anything in Roark that isn't in your bundle gets removed, so your repo is the source of truth. --no-prune opts out.
• JSON output: pipe either command into your own tooling with --json.

👉 Read the config-as-code guide

The Roark command line interface is now a first-class thing you can install in one line and drive your whole config-as-code loop from.

What's new:

• One-line install: curl -fsSL https://roark.ai/install.sh | sh, plus Homebrew and npm paths if you prefer them.
• roark config diff and roark config apply: point them at a directory of flows, metrics, and personas to preview and push changes.
• roark api for anything else: call any endpoint that doesn't have a generated command yet, with shell completion and a real exit code table.
• Docs page live: install, auth, usage, and CI recipes are all documented.

👉 Read the CLI guide

Every simulation run now generates its own executive report the moment it settles, so you can hand a stakeholder one link instead of a spreadsheet.

The summary lives at the top of the run report and includes the verdict, a scorecard, what went well, clustered failure themes with verbatim transcript excerpts, prompt-optimizer recommendations, and a trend against the previous run.

What you get:

• Auto-generated on completion: scheduled and ad-hoc runs both produce a summary without you clicking a button.
• Cued deep-links: every finding jumps to the exact metric pane and cues the affected calls to the moment that decided the verdict.
• PDF export and share link: ship the report to anyone, no login required.
• Email when it's ready: the requester gets a mail the moment the summary lands.
• Free: the summary passes are on us, not metered against your credit.

We shipped four system metrics you can turn on today, aimed at the questions we hear most often from voice teams.

What landed:

• Voice Consistency: catches when a speech-to-speech agent's voice identity flips mid-call (timbre or gender), backed by a new speaker embedding model. Scored per segment so you can pinpoint exactly where it happened.
• Task Completion: did the agent actually finish what the caller asked for, with a matching completion-rate threshold.
• Agent Containment: what share of calls the agent handled end-to-end without a human transfer.
• Human Transfer: a full package covering when and why transfers happen, plus a discovered-classification metric that clusters the reasons from your own transcripts rather than a fixed enum.

Personas got a big upgrade: age is now a first-class dimension, we shipped built-ins across the range, and every persona can carry emotion into the audio itself.

What's new:

• Age as a dimension: child, teenager, adult, and elderly, with voice casting to match.
• Six new built-in personas: Gen Z creator and gamer, a tech-illiterate grandpa on a landline, an overly familiar grandma, and two child accidental callers that a well-behaved agent should refuse to transact with.
• Emotion in the audio: personas now speak with ElevenLabs v3 audio tags, so frustration and warmth land in the voice rather than sitting in a metadata field.
• Accent flags on runs: call rows and debrief chips call out when a persona's accent may have driven the outcome.
• Custom properties everywhere: the properties you attach to a persona show up (and are editable) wherever the persona appears.

Customer flows and dashboards both got easier to organize this week.

What's new:

• Folders for customer flows: group related flows into named, colored buckets ("Verizon failure modes", "Auth edge cases") with the grouped view on by default.
• Drag to reorder flows: move flows within a folder, or across folders, without opening a menu.
• Drag to reorder report cards: dashboards let you rearrange cards the same way.

Alerts got a full rebuild around a first-principles model: WHEN [trigger] THEN [action]. Metric thresholds and platform events both flow through the same unified dialog, and every fire is now traceable end to end.

What's new:

• Event triggers: subscribe an alert to the full platform event catalog (Calls, Evaluations, Simulations, Metrics, Chats, Issues) with human labels and multi-select.
• Webhook actions: add one or more HTTPS URLs to any threshold or event alert. Deliveries go through the same pipeline as customer webhooks, with SSRF protection.
• Alert detail page: every alert now has its own page with config, a "Recent fires" timeline showing per-action outcomes (Slack N/M, webhook status, issue opened), and a "Recent deliveries" list with expandable payloads and responses, filterable by status.
• Slack channel picker on trend alerts, plus batched notification-channel loading so the alerts list stays fast.

If you've attached a knowledge base to your project, you can now measure whether your agent is actually using it correctly, with no per-metric setup and no hand-built variant. Add the Knowledge Base Grounding package to any run, policy, or dashboard, and you get three metrics from one LLM pass.

The three metrics:

• Knowledge Base Grounding (1 to 5): how faithful the agent's factual claims and offers are to your KB. Ships with a >= 3 system threshold.
• Ungrounded Claim (per turn): every agent turn that stated or offered something the KB doesn't support, classified as CONTRADICTS (the KB says otherwise) vs NOT_IN_KB (the KB is silent).
• Missed Answer (per turn): the inverse. Flags when the caller asked something the KB could answer but the agent deflected, said "I don't know", or answered wrongly.

Scope it at run time: in Studio Evaluate mode, pick specific KB documents to ground against, or leave it on "All knowledge bases" to auto-source every READY KB in the project.

The old Review area is now Annotations, and it's grown into a real collaborative workspace for grading calls with your team. Reviewers work from the same queue, label moments straight off the transcript, and disagreements are surfaced instead of buried.

What's new in Annotations:

• Label a moment straight from the transcript. Highlight a turn, tag it against a metric, add a quote, and it lands in the annotation with a jump link back to the exact segment.
• Per-metric rubrics for review sessions. Each metric brings its own scoring guide into the session, so reviewers see the same instructions inline instead of a shared doc nobody reads.
• Inter-annotator agreement across reviewers on the same call, so you can see where your team disagrees before you argue about model accuracy.
• Reviews hub on Library with live auto-score job progress, jumpable review queues, and a lede telling you what needs attention next.
• Alignment detail on follow-ups so a second reviewer coming into a call sees the previous verdicts, the quotes behind them, and where they agreed or split.

Labels aren't just for reporting anymore. When you correct a metric in Studio, that correction becomes ground truth, and your owned LLM metrics fine-tune themselves against it so the next call scores the way you would.

How it works:

• Correct a metric inline. On the call details page and in Studio's EVALUATE view, click a metric result to change the value or verdict. That correction is saved as ground truth against that call.
• Ground-truth panel in Studio shows every example you've labeled for a metric in one place, so you can see the shape of what you've taught it.
• Auto-tune once you have five examples. Owned LLM metrics with five or more ground-truth labels re-score using your examples as few-shots. If agreement with your labels improves, the tuned version sticks. If it regresses, we revert and tell you why.
• Discovery, labeling, and tuning on one surface. The Label calls action opens the picker on the metric card itself, so picking calls, scoring them, and watching alignment move all happen without leaving the metric you're working on.
• Owned LLM metrics only. System metrics and non-LLM metrics don't offer the tune action, so there's no ambiguity about what's actually adjustable.

We shipped three system metric packages that grade parts of a call you couldn't easily grade before, plus a 1-5 Transcription Score that rolls the accuracy family up into a single verdict.

What each package does:

• Property verification: compares the call properties you send us (account IDs, names, dates, CRM context) against what was actually said on the call, then flags mismatches anchored to the turn where they diverged. Nicknames, partial references, and phonetic near-misses are normalized before comparison, so it doesn't fire on "Robert vs Bob".
• Call screening: detects when a receptionist or automated screen picks up an outbound call instead of the person being called, then scores the agent's handling of that interrogation on a 1-5 scale. Ships with an optional brief for what the agent should say when asked who's calling and why.
• Transcription accuracy: grades your customer-supplied realtime transcript against ours, adjudicates every disagreement with an LLM (so we don't flag a case where our post-call transcript was the wrong one), and reports the specific words the transcriber got wrong, split into critical (numbers, names, negations, dates) and non-critical errors.
• Transcription Score: a 1-5 rollup over the accuracy metrics with critical errors as a ceiling, not a weighted term. A call with 0.98 accuracy that got a callback number wrong still can't read as Excellent.

Voicemail simulations no longer live as nodes inside a scripted flow. They're a first-class run template now: pick the Voicemail testing template, tick which greetings you want to test, run.

Why we moved it:

• Eleven greetings out of the box covering the major US carriers and generic voicemail patterns. Multi-select as many as you want per run.
• The old flow-node path silently ran the default carrier greeting regardless of what you configured, on every voicemail step. That's fixed under the new template.
• Existing flows keep working if you have voicemail nodes in production, but new voicemail runs should come through the template.
• Variable interpolation in the expected-message script so a single template can grade dozens of variants.

👉 Read the guide

Two things customers asked for by name: configurable retention on your call data, and a real delete for agents.

What's new:

• Retention windows per project (60 days, 3 months, 6 months, 1 year, or indefinite) in a new Data controls settings tab. Default is indefinite, so nothing changes on your projects until you opt in.
• Soft or hard delete mode. Soft hides the calls, metrics, and traces from the dashboard and is recoverable. Hard permanently purges the underlying records after a 7-day grace period.
• Delete an agent from the platform UI or via DELETE /v1/agent/:agentId. Soft-deleted agents stop accepting new calls immediately across LiveKit and Pipecat, so a deleted agent can't quietly resurrect on the next inbound.

You can now score how human vs robotic your agent actually sounds. Voice Human-Likeness is a new 1-5 metric that listens to the agent's own audio (not the transcript) and judges acoustic delivery against a fixed rubric.

What it gives you:

• Per-agent scores sampled evenly across the call, so scripted monologues never dominate the read.
• Catches smooth robots: TTS voices that pass classic MOS checks but still sound robotic to a real listener.
• Pairs with Voice Naturalness, also new this week. Naturalness (UTMOS-based) grades audio signal quality; Human-Likeness grades whether the delivery reads as human. Together they separate "sounds bad" from "sounds fake."

Frustration and user-effort scores are now folded into a single Customer Reception number. Higher is better, banded 1-5, and it lands on every quality-analysis run without adding another LLM call. It shows up in your metric library automatically, no configuration needed.

Studio's metric page now has a Thresholds section, so you can list, add, rename, archive, and unarchive pass/fail thresholds in one place.

Also new:

• Widget deep-links. "Metric details" on a dashboard threshold widget now jumps straight to the source metric and highlights the specific threshold you clicked from.
• Safe renames. Editing a threshold's name or Pass/Fail labels is never retroactive. Past conversations keep the labels they were graded against.

Studio Evaluate used to expand every picked metric's threshold editor inline, which made a suite of ten metrics feel like fifty. It now renders as a dense checklist where each metric shows a single chip: either the attached threshold or "Set threshold" to add one.

Two side-effects worth calling out:

• Threshold columns appear in the results matrix immediately, no more reload to see them.
• Progress reflects what actually ran, so the bar no longer skews when thresholds are attached.

Environment and Persona background-noise settings now play on calls placed over LiveKit, Telnyx, Daily, and Small WebRTC. If you've configured ambient office chatter or a call-center hum for a persona, it now lands on every transport we support. Loudness is also normalized to a fixed reference level, so a volume: 0.1 setting gives a predictable perceptual result regardless of the source asset.

The call audio player now surfaces where the interesting things happened, not just the transcript. A sentiment ribbon runs under the diarization tracks, and a moments lane above it marks interruptions, agent cutoffs, and any boolean or offset metric that fired during the call.

How it works:

• Hover a marker to see the metric name, value, reasoning, timestamp, and participant.
• Click a marker to seek the audio to that instant and highlight the matching transcript line.
• Toggle legend chips to filter the marker categories you care about.
• Any new segment or turn metric shows up automatically based on its output type. Sentiment paints the ribbon, booleans and offsets drop markers, whole-call scalars stay in the metrics tab.

Every issue in Roark now carries a remediation channel so you don't waste an LLM call generating prompt suggestions for a latency regression. A latency issue points at INFRA or LLM_MODEL, interruption issues point at VOICE_CONFIG, mishearing points at STT, and so on. The suggested-fix panel only auto-runs prompt analysis when PROMPT is actually in the mix; otherwise you get diagnosis-flavored guidance and a link to the right place to fix it.

Opened notifications also went source-agnostic. Trend alerts used to be the only source that pushed a webhook when an issue opened. Now AGENT, MANUAL, API, TRACE_FAILURE, and SINGLE_CALL_ALERT issues all fire the same issue.opened webhook. Existing subscriptions pick this up with no changes on your side.

Also new:

• Sidebar badge with a live count of open and acked issues, hidden at zero.
• Dashboard header chip showing the newest open issue title, or a quiet all-clear.
• "Fix via" chip set on every issue row so you can triage at a glance.

A new 1 to 5 LLM-judged metric in the Conversation Quality package that grades how naturally a call flowed: turn-taking, pacing, responsiveness, and the absence of awkward silences, robotic repetition, or talking over one another. Bands run from Very Choppy to Seamless, with a built-in threshold that fails the two Choppy bands so you can plug it into monitors immediately.

It is transcript-based (it measures conversational naturalness, not audio fidelity) and opt-in per policy, dashboard, or run plan. Add it to a call and you get a holistic "did this flow well?" score in one number, on top of the raw response-time and interruption signals you already have.

A few upgrades that make simulation runs faster to iterate on and more faithful to how your production flows are actually wired.

What's new:

• Preceded-by chained flows. An improv flow can now run a linked scripted flow first at runtime, and the create-run dialog prompts for the preceding flow's variables so the whole chain resolves without hardcoded values.
• Sub-hourly schedules. Run a plan every 15, 30, or 45 minutes (or any custom minute value down to 15). The old floor was hourly.

Roark now answers the simplest question about your voice agent: is it answering right now? A new Health checks page in the Monitor section runs a fixed liveness probe on whatever cadence you pick and renders every monitor as a status-page card.

What you get:

• Status at a glance: Operational / Down / Paused per monitor, a 24-cell probe tick bar, and 30-day uptime pulled from a dedicated ClickHouse fact table (a probe that never connects registers as an outage instead of quietly disappearing).
• Overview widget on the page: 30-day hero plus 7-day and today, with a daily status strip that draws from the same projection the cards do. They can't disagree.
• Uptime on dashboards: a new UPTIME report source auto-provisions an uptime widget on your default dashboard the first time you create a health check.
• Sharper liveness check: the probe's headline is now agent_spoke (did the agent produce any speech at all?) instead of the old who-spoke-last heuristic that false-alarmed on healthy calls where the caller hung up last.
• Cost transparency: the create dialog spells out probe volume for the cadence you pick, so nothing surprising lands on your bill.

Recurring behavioral failures across your production calls now get grouped into Sentry-style issues automatically, no alert configuration required.

How it works:

• Always-on clustering: a scheduled sweep reads the last 24 hours of failing checks and numeric system signals (frustration, sentiment, comprehension failures, conversation loops), asks a model to group them into failure modes, and opens or updates an issue per mode with stable fingerprints and per-call evidence.
• Recent view by default: the issues page now leads with active plus recently-resolved so the trend issues that auto-resolve in an hour aren't hidden the moment you check.
• Issues banner on call detail: when a call is cited as evidence inside an active issue, you see it right there with one click into the full picture.
• Five new default watchers seed on every project (agent going silent, high frustration, negative sentiment trend, comprehension failures, loops), with a one-time backfill for existing projects that never got seeded.

After a simulation run settles, Roark reads the failing checks, the worst calls, and the current agent prompt, then proposes targeted edits with anchors and evidence instead of hand-waving a rewrite. Fix it, re-run the suite, get proof it moved.

What landed:

• Evidence-grounded findings: every suggestion cites the checks and conversations it came from. Anything the model invents is dropped before it reaches your screen.
• Verify loop: focus the analysis on a single failing check, re-run the same suite, and see the movement quantified check by check in the comparison view.
• Prompt versioning: every prompt now carries immutable version history, each conversation records exactly which version ran it, and the optimizer notices when your prompt has drifted since the last analysis.
• Suggestions on issues too: expanding an AI-detected issue runs the same analysis against the calls in evidence and shows the recommended prompt fix inline, so you can go from "here's the pattern" to "here's the edit" without leaving the page.

Ask the in-app assistant to "track how often callers ask about pricing" and it does the whole chain: finds or creates the metric, starts a collector so it runs on every new call, and drops a chart on your dashboard.

How it behaves:

• One confirmation, whole pipeline: metric → collector → report → optional backfill of history, with Roarky narrating what it picked.
• Sensible defaults: auto-places new charts on your default dashboard when you don't name one.
• Honest about empty state: a freshly-charted metric reads empty until data collects. Roarky says so and offers to backfill the history.

Scripted flows now have a branching mode toggle in the editor header, so one flow can either fan out into a simulation per path or collapse into a single call the agent adapts across.

What's new:

• DETERMINISTIC runs one simulation per unique path (the existing behavior).
• ADAPTIVE collapses a flow's own paths into a single call, and links to sub-flows follow whichever path the agent-under-test takes.
• Persona coverage stays orthogonal: five paths across three personas produces three adaptive calls, not one, so you still get real persona variation.
• Simulation tab on call detail: every simulated call now surfaces the flow it ran (adaptive brief or scripted steps), the persona, and the agent endpoint, with deep links out to each.