Skip to content

CLI & Commands

synthkit ships several binaries under cmd/. All configuration is environment-driven (read from .env or the process environment) unless flags override specific values. Build everything with go build ./....

synthkit — the generator

The main binary. Scans the directory named by BLUEPRINTS (default ./blueprints) and loads only the runtime identities selected by BLUEPRINT_NAMES. Empty/unset starts setup mode; * explicitly loads the complete catalog.

./synthkit [flags]
FlagDefaultDescription
-oncefalseRun one full cycle and exit.
-fleet-unregisterfalseExplicit teardown: unregister exactly the collectors the selected resolved roster would register, including Kubernetes mirrors and persisted scaling. UnregisterCollector only marks each collector inactive; Fleet Management deletes inactive collectors after 30 days. No telemetry or registration is started. Exit zero only on successful teardown; requires DRY_RUN=false and cannot combine with other modes. Lease mode acquires leadership through the existing lifecycle before teardown; interruption before leadership or successful completion exits nonzero.
-fleet-unregister-ownedfalseFull teardown from the persisted Fleet ownership ledger: permanently delete (irreversible, management DeleteCollector; GC_FM_TOKEN needs Fleet Management write access, and a refused delete is reported as rejected) every collector the ledger records for the configured GC_FM_URL and GC_FM_STACK_ID, including historical entries the current selection no longer names, and release each ID only after a confirmed-absence read. Prints one secret-free fleet teardown receipt: line and exits zero only when the status is complete. A missing or unbound ledger reports no_ledger and exits nonzero. Refuses dry-run, a foreign destination, a nonempty unbound legacy ledger, Kubernetes state with HA_MODE=off, and combination with other modes. See fleet-management.md.
-dumpfalseWith -once: print the full series/label inventory to stdout (diff against signals/).
-validatefalseOffline validation of the selected local blueprint set; print JSON cardinality projections and diagnostics, then exit non-zero on errors.
-preflightfalseValidate and probe mandatory live Grafana endpoints, then exit with redacted lane/reason output.
-healthcheckfalseExit successfully only when the local control plane reports delivery readiness; used by Compose.
-startupcheckfalseRead-only local /control/startup probe: boot completion only, never delivery or admission. Uses JSON_HTTP_ADDR directly, not -env; cannot combine with other modes or -one-shot-deadline. Requires a compatible image. See startup versus delivery readiness.
-versionfalsePrint {"version":"X.Y.Z","revision":"<40-hex>"} as JSON, then exit. Local builds report dev/unknown unless stamped.
-env <path>.envPath to the .env file (optional; falls back to process environment).

Verification modes (I32):

# Validate a selected set offline (no credentials needed, no state writes).
BLUEPRINT_NAMES=otlp-native ./synthkit -validate

# Print the full inventory of distinct series names + label keys — push nothing.
SELFOBS_ENABLED=false DRY_RUN=true BLUEPRINT_NAMES=otlp-native ./synthkit -env /dev/null -once -dump

# One live cycle (DRY_RUN=false to push real data).
DRY_RUN=false BLUEPRINT_NAMES=otlp-native ./synthkit -once

# The continuous loop (default).
./synthkit

DRY_RUN defaults to true. You must explicitly set DRY_RUN=false to push synthetic data to Grafana Cloud. See Configuration for the full environment variable reference, and Credentials for how the Grafana Cloud tokens are scoped.

-validate uses BLUEPRINTS, BLUEPRINT_NAMES, BLUEPRINT_DATA_DIR and the source configuration in CONFIG_SNAPSHOT_PATH. It reads built-ins, namespaced uploads and already-fetched git snapshots; it never fetches remote sources, writes a boot manifest, changes file permissions, provisions Synthetic Monitoring, starts self-observability or contacts telemetry endpoints. This remains offline even with DRY_RUN=false. Empty selection validates the empty setup-mode set; BLUEPRINT_NAMES=* selects all available local blueprints. Unknown selected names, strict schema errors, cross-blueprint identity collisions and unavailable projections fail the command.

Stdout is one JSON object with ok, blueprints and diagnostics. Each blueprint has name, provenance, cardinality, estimated and warning diagnostics. Set/load errors carry severity, source, stage and detail in the top-level diagnostics array. Cardinality is the number of distinct metric series observed in one throwaway dry cycle (including native OTLP metrics), not a long-term upper bound; low-volume request lanes and time-dependent families may vary or be absent in that cycle. cardinality: -1 means the projection was unavailable. Runtime control overrides are not applied. Validation takes precedence over other mode flags except -startupcheck, which refuses combinations with any other mode.

What -dump output is deterministic, and what isn't. The authoritative section of a -dump — each sink's <name> {[sorted label/attribute keys]} inventory block — is the structural contract: series names, label/attribute key sets, and (for sigil) the ingest-kind → operation-name mapping. That block is byte-identical across consecutive runs and is what a diff against signals/ should compare.

The [dry-run <sink>] N series e.g. <series> lines underneath it are NOT part of that contract — each one logs a single randomly-sampled exemplar per batch, so which exemplar gets printed varies run to run by design. Never diff two raw -dump outputs line-for-line; it will show noise from this sampling (and, for sigil, from genuinely fresh per-run correlation IDs — see below) and prove nothing about a regression.

For the same reason, the sigil block's final summary line — == sigil: generations=N workflow_steps=N scores=N == — is a live COUNT, not part of the structural contract, and it is expected to vary run to run: internal/ledger mints a fresh, cryptographically-random SessionID/correlation ID per conversation on every run (by design — correlation ids must be unique and unguessable, never derived from a seed unit). Per-conversation turn count is a deterministic hash of that id (internal/workload/aiagent/minter.go's TurnCount), so a fresh id set naturally reshuffles the aggregate turn-derived counts (generations, scores) even though the number of conversations minted per tick is itself fully deterministic (fixed sessions_per_min config × the shape engine's fixed-seed PRNG). Do not treat a generations=/scores= count mismatch across two -once -dump runs as a determinism regression; treat a mismatch in the structural inventory block as one.

The control plane is available at http://<bind>:<port>/control/ (default port 8088). See control-plane.md.

synthkit-deploy.py — deployment identity and rollback helper

The stdlib-only helper emits closed, secret-safe JSON reports. It never prints .env values or private state content.

SubcommandMutation boundary
resolve-imageRead-only selector precedence and mutable-tag report.
set-imageCompare-and-swap only SYNTHKIT_IMAGE_REF; preserves unrelated bytes and aborts on drift. Mutable main/latest needs --allow-mutable.
snapshot-stateRequires the named container stopped; creates an integrity-manifested external private snapshot.
restore-stateRequires the container stopped; verifies the snapshot, atomically replaces the validated state target, and retains displaced state.
write-recordWrites one external mode-0600 closed identity record.
verify-imageRead-only exact index/platform/config/binary/signature/provenance verification.
check-composeRead-only minimum-version and default/profile rendering check with a fake env file.
inspect-runningRead-only cross-check of configured index, platform/config/image ID, health, binary version, and revision against expected values.

verify-image trusts the reusable rknightion/.github/.github/workflows/container-publish.yml workflow path across signer revisions. That trade-off puts control of the workflow repository inside the trust boundary; it does not relax the GitHub Actions OIDC issuer, synthkit source repository/digest/ref, OCI version/revision labels, selected platform, or binary identity checks.

See Deployment for the ordered upgrade and rollback commands.

sm-provision — Synthetic Monitoring provisioner

One-shot snapshot-bound provisioner for Synthetic Monitoring. The published image contains both the emitter and provisioner at the same source version. The tool must match the current snapshot version, but v2 activation survives release-only bumps with unchanged specs/target/schema. The first v2 format upgrade requires explicit apply/restart. Preview is always the default.

docker compose --profile sm-provision run --rm sm-provision
SM_PROVISION_APPLY=true docker compose --profile sm-provision run --rm sm-provision
docker compose restart synthkit

Explicit ownership-bound cleanup is a separate subcommand. Preview it, then apply exactly the previewed plan:

docker compose --profile sm-provision run --rm sm-provision cleanup
SM_PROVISION_APPLY=true docker compose --profile sm-provision run --rm sm-provision cleanup -confirm=<plan>
docker compose --profile sm-provision run --rm sm-provision help

Cleanup deletes only resources the ledger records as owned for the configured target, by recorded ID, after rechecking each one's revision and specification. Every delete first stores a pending intent; a delete counts as done only when an inventory read shows the ID gone. Every run that reaches a plan prints a secret-free receipt line; a refused run prints none. Cleanup also runs after SM is deselected or STARTUP_DISABLED_KINDS=synthetic_monitoring is set: with no snapshot published, it binds to the ledger's recorded target and the stored state instead, and it never re-enables SM emission. The Helm chart runs it through smProvision.cleanup (see Kubernetes). Any other argument is rejected before configuration is read. See synthetic-monitoring.md.

Environment variableRequiredDescription
GC_SM_URLyesSM API base URL
GC_SM_TOKENyesSM API bearer token
CONFIG_SNAPSHOT_PATHnoFile-mode emitter state path; Compose sets /data/control-state.json. Ignored by diskless Kubernetes SM storage.
STATE_BACKENDnofile default or kubernetes; the latter uses in-cluster credentials only.
STATE_SM_CONFIGMAPKubernetesDistinct precreated SM state slot.
SM_PROVISION_LEASE_NAMEKubernetesPrecreated provisioning Lease distinct from HA_LEASE_NAME, shared by all writers to that SM slot.
HA_NAMESPACE, POD_UIDKubernetesInstallation namespace and downward-API process identity.
SM_PROVISION_APPLYnoExact true enables mutations; absent/false previews
SM_PROVISION_ADOPT_LEGACYnoExact true records the exact-match plan during preview and allows only that same plan during apply
SM_PROVISION_MIGRATE_TARGETnoExact true previews or applies a credential/endpoint target migration; apply also requires SM_PROVISION_APPLY=true and a matching preview no older than 15 minutes

DRY_RUN does not control this command. Source checkouts may use go run ./cmd/sm-provision, but the Compose profile is the supported Docker-only deployment route. See synthetic-monitoring.md for ownership and crash-recovery rules. The in-cluster tool acquires its separate Lease before any SM request/state update and reuses the HA request/operation/fence budgets. Acquisition is capped at two minutes; the invocation at ten. There is no environment-selected API endpoint, KUBECONFIG fallback or automatic provisioning. The Kubernetes-state Helm Job supports HA emitters with separate named state and Lease authority. Use matching emitter/tool images at least 1.6.0-rc.176; preview, explicit apply and emitter restart/reacquisition remain required. File mode is still available.

agento11y-provision - manual Agent Observability provisioning

Source-built, same-user local command that creates, disables and tears down the Agent Observability resources declared in a blueprint's agent_observability: block. It is not in the published image or Compose and installs no runtime admission. Build it with `just build-packages ./cmd/agento11y-provision