grafana-cloud-vending-machine¶

A reference architecture for vending Grafana Cloud stacks through a small, declarative API. It supports Grafana Cloud only. Argo CD owns what is in Git, Crossplane continuously reconciles Grafana Cloud, and External Secrets Operator (ESO) moves credentials between Kubernetes and an external secret store.

It is deliberately more than a minimal stack example. The baseline includes rotating administrator and telemetry credentials, deletion protection, stack-local provider configuration, starter content, configurable drift behaviour, OAuth and SAML SSO, reports, plugins, incident relay resources, teams, basic-role ACLs, fixed-role assignments, custom roles, and role assignments.

One stack belongs to one Grafana Cloud organization. One vending-machine installation can serve several registered organizations, each with its own permitted regions and usages. Every namespace that accepts requests carries a same-named organization ProviderConfig and credential Secret for each registry entry because the v2 provider resolves ProviderConfig references in the managed-resource namespace. The request selects an immutable spec.organization; there is no implicit single-organization fallback.

The source and issue tracker live on GitHub.

platform.example.org is a documentation placeholder

Every XRD, Composition, example, and page in this documentation uses the API group platform.example.org. It is not a production API group — it is not resolvable and it is not meant to be used as-is. Replace every occurrence with a domain your organization controls before adopting this API. See Getting started and Configuration for where this matters most.

Who this is for¶

Platform teams that already run Argo CD and Crossplane and want to offer Grafana Cloud stacks as a self-service, GitOps-native product — with rotating credentials, safe defaults, and an explicit Retain-by-default lifecycle with a reviewable Delete path — rather than hand-running Terraform or the Grafana Cloud API per request.

Request-to-stack flow¶

A request is a GrafanaCloudStackRequest custom resource committed to a directory under the repository's top-level enabled/. From there:

Argo CD watches enabled/* through an ApplicationSet and creates one Argo CD Application per directory, applying the request object to the cluster.
Crossplane, through a Composition pipeline backed by a Go composition function, renders the request into a full set of managed resources: a Stack, a rotating administrator service account and token, an optional rotating telemetry access policy, baseline folders and dashboards, and any SSO, plugin, report, incident, team, or access-control resources the request enables.
Crossplane's Grafana provider continuously reconciles those managed resources against Grafana Cloud — creating what is missing and repairing drift on anything the request's reconciliation mode marks as enforced.
External Secrets Operator moves each registered organization's credential from an external secret store into the cluster so the provider can authenticate, and moves the stack's generated administrator and telemetry tokens back out to the external store as structured per-stack documents.

Nothing under examples/catalog is applied by the supplied ApplicationSet, and the top-level enabled/ directory starts empty. A user must copy a catalog example, edit every placeholder, review the rendered output, and commit it before a stack is created. This is a deliberate safety property: cloning or installing this platform cannot, by itself, create a Grafana Cloud stack.

Repository layout¶

The core product is platform/. The other directories are integration examples, inert catalog entries, validation tooling, and the GitHub workflows that publish the composition function.

.
├── platform/
│   ├── apis/                 XRDs and pipeline Compositions
│   ├── function/             Go composition function, tests, package metadata
│   ├── provider/             Grafana provider, activation policy, signature gate
│   ├── rbac/                 minimum extra composition RBAC for ESO resources
│   └── kustomization.yaml
├── examples/
│   ├── README.md             catalog index and safe enablement workflow
│   └── catalog/              inert stack, SSO, Teams, RBAC, and content-ACL examples
├── enabled/                  only path watched by the example ApplicationSet; empty by default
├── deploy/
│   ├── argocd/               controller, platform, and per-request GitOps examples
│   ├── aws/                  SecretStore, ExternalSecrets, ProviderConfig, IAM policy
│   ├── crossplane/           production-oriented Helm values
│   └── external-secrets/     production-oriented Helm values
├── scripts/                  validation and public-release safety scan
└── .github/workflows/        validation and signed multi-platform function publishing

What the baseline includes¶

Every enabled request gets, by default or by opt-in field:

Capability	What it is
Rotating administrator credential	A `StackServiceAccount` with the Admin role and a `StackServiceAccountRotatingToken`, rotated automatically rather than issued once as a static token
Rotating telemetry credential	A stack-realm `AccessPolicy` scoped to `stacks:read`, `metrics:write`, `logs:write`, `traces:write`, with its own rotating token, when `spec.telemetryAccess.enabled` is true
External-resource lifecycle	`spec.lifecycle.externalResources` defaults to `Retain`; a `Delete` intent requires an exact platform authorization for the request namespace, name, UID, and immutable profile, a readiness wait, and three reviewed stages: arm, clear access claims, then remove the request
Deletion protection	Retain mode keeps the `Stack` protected; armed Delete waits for observed `deleteProtection=false` and targets only external state that can outlive the Stack
Stack-local provider configuration	A namespaced Grafana `ProviderConfig` scoped to the one stack, built from the generated administrator credential
Starter content	Three baseline folder/dashboard pairs occupying the billing/usage, telemetry-endpoints, and stack-home slots, when `spec.baselineDashboards.enabled` is true
Configurable drift behaviour	Per-resource `enforced`/`createOnly`/`observeOnly`/`disabled` reconciliation modes that decide whether Crossplane repairs an administrator's UI edit or leaves it alone
OAuth and SAML SSO	Platform-defined SSO profiles a request selects by name — `github`, `gitlab`, `google`, `azuread`, `okta`, `generic_oauth`, or `saml`
Reports	An optional scheduled monthly usage report (PDF/CSV)
Plugins	An optional list of Grafana Cloud plugin installations
Incident relay resources	Optional OnCall outgoing webhooks and Alerting contact points that call a platform-owned relay
Teams	Directory-synchronized or directly-managed Grafana teams
Basic-role ACLs	Folder/dashboard permissions granted to the built-in Viewer/Editor/Admin basic roles
Fixed-role assignments	Existing Grafana-managed fixed roles assigned to a team by UID
Custom roles	Stack-local roles built from explicit action/scope permission pairs
Role assignments	Whole-set or item-level bindings of a role to a team
Explicit modules	Inventory, Fleet pipelines, alerting, Agent Observability, Assistant governance, datasource access, Git provisioning repositories, and product toggles have their own ownership boundaries and prerequisites

What this reference proves, and what it does not¶

This repository is a portable public reference. It never contacts a live Grafana Cloud stack, a live cluster, or any source environment, and it carries no credentials and no live requests. That is a deliberate boundary and not a gap waiting to be filled: the validation gate is designed so it cannot make that contact, and no future change to this repository will produce live behavioural evidence.

What the gate does prove, and the limit of each:

Evidence class	What it establishes	What it does not
Admission against a real API server	Every XRD installs, every catalog example is admitted, and every fail-closed CEL rule actually rejects, with create/update and weaken/admit/restore controls	The server is ephemeral and empty. Nothing reconciles, so no provider behaviour is observed
Renderer tests	What the composition function emits for a given request and observed state, including the refusals and the waits	That the provider then accepts those documents, or that Grafana Cloud accepts what the provider sends
Provider CRD readback	The emitted shapes round-trip through the pinned provider's own CRDs	That a live provider assigns the identities the renderer waits for
Pinned upstream source	Import identities, external-name conventions and management-policy semantics, read from the exact pinned provider and runtime revisions	Runtime behaviour. A source reading is a strong argument, not a reconciliation
Signed package provenance	The published function image matches the source commit and workflow identity that built it	Anything about what that function does once installed

So credential minting, stack adoption, cross-cluster migration and drift correction are described and unit-proven here, and they are not exercised end to end. Rehearse them in an environment you own, against a disposable stack, before running them against one carrying traffic. Treat every procedure in Migration and adoption as requiring that rehearsal.

Reading further¶


Getting started	Prerequisites and the copy-edit-review-commit path to your first stack
Installation	Bootstrapping Crossplane, ESO, and the platform components
Configuration	Platform policy, profiles, and the organization registry
Architecture	The three-controller split, reconciliation, and where state lives
Secrets	ESO wiring and the credential rotation model
SSO	OAuth and SAML configuration
Security	Supply-chain controls, secret handling, and the Retain-by-default lifecycle
Governance	Token ceilings, network restrictions, product modules, retention, and tenancy
Migration and adoption	Breaking schema changes per release boundary, inventory-first adoption, and the non-destructive handover path
Troubleshooting	Common failure modes
FAQ	Short answers to recurring questions

Project¶

grafana-cloud-vending-machine is open source under the Apache 2.0 licence. Apache License 2.0. See LICENSE. Issues and pull requests are welcome on GitHub.