Control Plane¶

synthkit embeds an operator control plane served on port 8088 (configured via JSON_HTTP_ADDR). It has two surfaces:

• Operator UI at /control/ui — a browser dashboard for live runtime management.
• HTTP API — a JSON API used by the UI, the Grafana Infinity datasource, and curl/automation.

All GET endpoints are open (no auth). POST mutation endpoints require HTTP Basic auth when CONTROL_TOKEN is set.

Operator UI¶

Open http://127.0.0.1:8088/control/ui (or the host's address if exposed).

The UI provides:

• Overview — per-blueprint emission status, sink readiness strip, dry-run indicator.
• Scenarios — activate and deactivate named incident scenarios defined in blueprints.
• Scaling — set live workload pod counts within blueprint-declared bounds.
• Failures — ad-hoc failure injection (any mode, any target, any intensity).
• Load — master volume multiplier (scales all synthetic volume coherently across blueprints).
• Blueprints — enable/disable blueprints; manage custom and git-sourced blueprints.
• Constructs / Kinds — toggle individual construct instances or all constructs of a kind.
• Diagnostics — load-time problems (skipped blueprints, dropped config entries).
• Config — redacted runtime config view.

The schema driving the UI (available modes, targets, scenarios, scalable workloads) is derived from the currently loaded blueprints at startup — it is never hardcoded.

Authentication¶

When CONTROL_TOKEN is set:

• All GET routes remain open (unauthenticated). The Grafana Infinity datasource reads GET routes and must not be blocked.
• All POST, DELETE, and other mutation routes require HTTP Basic auth: username control, password = CONTROL_TOKEN.
• A browser hitting a guarded route for the first time triggers Chrome/Firefox's native credential dialog.
• The Grafana Infinity datasource authenticates via basicAuthUser: control + basicAuthPassword: <CONTROL_TOKEN>.

# Example: curl with auth
curl -s -u control:${CONTROL_TOKEN} -X POST http://127.0.0.1:8088/control/load \
  -H "Content-Type: application/json" \
  -d '{"volume_multiplier": 2.0}'

Endpoint reference¶

Read-only (GET)¶

The root path / serves the Infinity JSON host — the full synthetic data schema queryable by Grafana's Infinity datasource.

Mutations (POST / DELETE — guarded by CONTROL_TOKEN when set)¶

Common operations¶

Activate a scenario¶

curl -s -X POST http://127.0.0.1:8088/control/scenarios \
  -H "Content-Type: application/json" \
  -d '{"active_scenarios": ["mine/db-pressure"]}'

Scenarios are identified as blueprint-name/scenario-name. To deactivate all scenarios, pass an empty list.

Scale a workload live¶

curl -s -X POST http://127.0.0.1:8088/control/scaling \
  -H "Content-Type: application/json" \
  -d '{"mine-api": 8}'

Node count cascades: the k8s cluster and EC2 construct both re-derive their node counts via fixture.DeriveNodes so the substrate stays consistent. Scale-down retires old pod and node series automatically (state.DropWhere).

Inject a failure ad-hoc¶

curl -s -X POST http://127.0.0.1:8088/control/failures \
  -H "Content-Type: application/json" \
  -d '{"connection_saturation": {"enabled": true, "intensity": 0.7, "scope": "mine-db"}}'

Check sink readiness¶

curl -s http://127.0.0.1:8088/control/status | jq '.sinks[] | {name, last_success_ms, failures}'

dry_run: true in the response means no live push is happening regardless of sink health — re-check DRY_RUN in your .env.

Boost volume temporarily¶

curl -s -X POST http://127.0.0.1:8088/control/load \
  -H "Content-Type: application/json" \
  -d '{"volume_multiplier": 3.0}'

State persistence¶

Control state (volume multiplier, active scenarios, scaling, disabled blueprints/constructs/kinds) persists across restarts in the snapshot file at CONFIG_SNAPSHOT_PATH. The file is written lazily on the first mutation — it is normal for it not to exist on a fresh deploy.

State is not written on a clean shutdown (it is already in the file from the last mutation). It is written only when a mutation succeeds.

See also¶

• incidents.md — declaring and triggering incident scenarios in blueprints
• custom-blueprints.md — uploading and managing custom blueprints
• configuration.md — CONTROL_TOKEN, JSON_HTTP_ADDR, CONFIG_SNAPSHOT_PATH