Chat Scenarios¶

synthkit-chat adds people to a failure story. When an operator activates a SynthKit scenario, a small cast of simulated coworkers talks in Slack around the real alert it raises: someone notices the slowdown, the incident commander asks for an ack, the backend owner is pulled in when asked. The operator is the only human in the room and drives every Grafana IRM button himself. Personas ask him to act; they never claim an IRM action.

It is offline tooling in its own binary. It never links into the synthkit emit binary, and it talks to SynthKit only by polling GET /control/state.

Three files¶

The repository ships a generic example triggered by the bundled acme-ai-platform/backend_latency_storm scenario, a Slack app manifest and a bindings template, all under examples/chat/.

Rehearse in the terminal¶

go run ./cmd/synthkit-chat console -scenario examples/chat/backend-latency.chat.yaml

Console mode needs no Slack, no SynthKit and, with the default -llm fake, no model. You are the operator: plain lines are your messages in the current channel. Type start (or :synthkit on, which honours trigger.delay), then inject what the platform would do:

Add -llm openrouter (with OPENROUTER_API_KEY set) to rehearse against the real model, and -trigger manual to start at once. Every run writes a JSONL journal under chatsim-runs/. Without a control URL, fire and heal inject the SynthKit edge locally, and control-DM notices print with a [dm] tag.

Run it live¶

1. Create a Slack app from examples/chat/slack-app-manifest.yaml ("From a manifest" at api.slack.com), install it, copy the bot token, and create an app-level token with connections:write. Socket Mode needs no public URL.
2. Invite the app to the static channels. New public channels whose name matches a scenario channel's bind.name_match are joined automatically and their recent history is read; a private channel needs the app invited by hand.
3. Copy examples/chat/bindings.example.yaml somewhere private and fill in the ids.
4. Capture IRM's identity: synthkit-chat tap -bindings b.yaml -out tap.jsonl -join-match '^incident-', fire a test alert, and read IRM's bot_id off the captured post. tap posts nothing. Use the same capture to write when.slack.text_match regexes against the real payloads.
5. Check everything offline: synthkit-chat validate -scenario story.chat.yaml -bindings b.yaml.
6. Run: synthkit-chat run -scenario story.chat.yaml -bindings b.yaml. The run arms when the configured trigger.synthkit_scenario goes from inactive to active in /control/state (HTTP Basic control:<token>); a scenario that is already active at launch never fires. -trigger manual starts immediately instead.

In the Helm chart¶

Set chatsim.enabled to run synthkit-chat run as a second container in the emitter pod. It reaches the control plane on the pod's loopback, so set the bindings' control.url to http://127.0.0.1:8088 and leave the control plane closed. Put the scenario, bindings and optional seed in a ConfigMap (chatsim.configMap, keys chatsim.scenarioKey, bindingsKey, seedKey) and the tokens the bindings name in a Secret (chatsim.existingSecret). The files are read at start, so restart the pod after changing them. The chart refuses chatsim with ha: two Socket Mode connections would post every line twice.

The kill switch is /chatsim <command>, accepted only from operator_user_id and answered privately. The same commands work on the terminal's stdin and in the control DM.

The control DM¶

A direct message to the app is the operator's private control channel. Open it in a window the audience cannot see. The app answers there, and posts a notice when the chat starts, the SynthKit scenario is fired or confirmed, a beat opens, someone joins a channel, a line is dropped by lint, a director call fails, the LLM budget runs out, and the run ends. Set control_dm.name and control_dm.emoji in the bindings to give the DM its own voice.

The control panel¶

On startup the app posts a control panel in the DM and pins it, retiring (unpinning and emptying) any panel an earlier run left behind. It is edited in place, at most once every two seconds, so it always shows the run's state: waiting, armed, chatting, paused, winding down or ended; when it started; whether the failure is fired; open beats; who is in which channel; LLM calls against the budget; and the latest notice. Its buttons are the same control verbs and offer only what works now: Start chat, Fire failure, Heal, Pause or Resume, and Stop (which asks first). A dropdown summons anyone not yet in the summon channel, and Stage direction opens a text box. Only the bound operator's clicks count. Notices still arrive as ordinary DM messages below it, so the DM keeps the history and the panel the present. The panel needs the manifest's interactivity enabled and the pins:read and pins:write scopes; without the pin scopes it still works, unpinned.

trigger.start_phrase and trigger.fire_phrase (regexps) do the same from the room: the operator's first message matching start_phrase in a static channel starts the chat, and the first matching fire_phrase after that fires the scenario. Each is optional.

Resetting the workspace¶

For a workspace dedicated to the drill, a seed file (kind: ChatSeed) is the known state to return to between rehearsals: a short history in each channel, written once and posted verbatim (no model). synthkit-chat reset -scenario s.chat.yaml -bindings b.yaml -seed s.seed.yaml deletes every message in every channel the user token's user belongs to (thread replies first), archives channels matching wipe.archive_match (an earlier run's incident rooms; bound channels never), leaves wipe.keep alone, then posts the seed. -dry-run lists what it would do and changes nothing. Seed channels are checked first, so a seed naming a missing channel changes nothing.

Run synthkit-chat run with -seed to add a Reset Slack button (with a confirmation) to the panel, and the bare reset verb in the DM. It ends the current run at once, deactivates the failure scenario if this run fired it, resets, discards the events the reset caused, and arms a fresh run, so the start phrase works again.

A bot token can delete only the bot's own messages, so a reset needs a workspace owner's or admin's user token (slack.user_token_env, default SLACK_USER_TOKEN): add the manifest's user scopes, reinstall, and copy the User OAuth Token. Owners and admins can delete members' and apps' messages in channels they are in; anything Slack still refuses is counted in the report. Seeded lines post now, since Slack cannot backdate a message: pace (default 4s..10s) or a line's after spreads them out. wipe.irm: true first resolves every open Grafana IRM alert group and incident on the bindings' irm.grafana_url (token in irm.token_env, default GRAFANA_IRM_TOKEN, a service account with IRM admin rights). Neither can be deleted for good, so resolved ones remain in IRM's history. A persona's avatar.image_url (public https) shows a picture instead of its emoji. A line is from a persona id, operator (posted as the user token's user) or app, in a logical channel or #name; key and reply_to make a thread. An app line can replay another integration's post: as: {name, icon_url} sets the name and picture it shows, and blocks_file (a JSON array of Slack blocks, relative to the seed file, such as the blocks of a captured conversations.history message) is posted with text as the notification fallback. A blocks file that does not decode stops the reset before anything is deleted.

apiVersion: synthkit.chatsim/v1
kind: ChatSeed
name: monday-morning
wipe: { archive_match: "^incident-" }
pace: 5s..12s
messages:
  - { channel: ops, from: alex, text: "handover: quiet night, nothing open", key: handover }
  - { channel: ops, from: operator, text: "thanks, I have it from here", reply_to: handover }

Freestyle¶

The operator does not have to follow the beats:

• A message that names nobody present and opens no beat opens the floor: exactly one persona in that channel answers it.
• Naming a persona who is not in the channel together with a request word (get, ping, add, invite, need, where, and similar) brings them in: someone present says they are fetching them, the narrator line follows, and the persona's when: { summoned: ... } beat runs. A beat that already summons that persona for the same message takes precedence.
• A line rejected for an unknown number or an uncited completed action is retried once with a hint to rephrase; a line still rejected is dropped and reported in the control DM.

Writing a scenario¶

apiVersion: synthkit.chatsim/v1
kind: ChatScenario
name: my-drill
seed: 7
trigger: { synthkit_scenario: acme-ai-platform/backend_latency_storm, delay: 20s }
llm: { model: deepseek/deepseek-v4.1-flash-1017, zdr: false }
operator: { name: Casey, role: on-call SRE }
channels:
  ops: {}
  incident: { bind: { on: channel_created, name_match: "^incident-" }, fallback: ops }
personas:
  - { id: alex, name: Alex Moreno, role: Incident commander, style: short and calm,
      avatar: { emoji: ":large_blue_circle:" }, present: [ops, incident] }
beats:
  - { id: alert_lands, when: { slack: { from: irm, text_match: "(?i)firing" } },
      speakers: [alex], goal: Alex asks Casey to acknowledge the alert, record: { fact: alert_fired, from: event } }

Decoding is strict: an unknown field is an error. Each beat has exactly one trigger:

A beat opens, gets up to max_turns lines from its speakers, and closes when its turns are used, when the director marks it done, or at expires_after. then: [{ summon: ben, channel: incident, after: 25s..45s }] brings a persona in after the beat's first line: a narrator line in the app's own identity ("... was added to the channel by ...") is posted just before the persona's first message. A final: true beat ends the run once it is done; end.after bounds every run.

Personas post with chat.postMessage plus a per-message username and icon_emoji. They are never edited and never scheduled on Slack's side. A persona with avatar.image_url posts with that picture instead of its emoji.

What keeps it honest¶

One director call per wake-up returns up to max_lines_per_turn lines as strict JSON (response_format json_schema, strict: true, with OpenRouter provider.require_parameters and data_collection: deny; zdr is a knob). The engine validates the answer locally and retries once, then applies deterministic guardrails before anything is posted:

• lines only for an open beat, a persona the operator addressed by name, or the single idle nudge;
• max_turns, per-minute caps (global and per persona), max_llm_calls and end.after;
• a quiet period after each operator message for everyone he did not address;
• no IRM claims: a persona saying it acknowledged, resolved, escalated, silenced, paged or declared anything is rejected outright;
• a completed operational action ("restarted", "rolled back") must cite a ledger entry or a beat may_claim; a number with a unit or a URL must already be in the ledger;
• no @here, @channel, app or user mentions other than the operator, no slash commands, no names outside the cast, no out-of-character AI talk, and max_line_chars.

A rejected line is never posted. Typing delays scale with line length and typing_wpm, and each channel is paced to one post per second.

Current limits¶

Operator intent classification, run resume and reconciliation, persona threads and reactions, and concurrent runs are not implemented yet.