Architecture¶

flowchart LR
    Git[Git request] --> Argo[Argo CD]
    Argo --> XR[GrafanaCloudStackRequest]
    XR --> XP[Crossplane composition]
    XP --> Cloud[Grafana Cloud resources]
    AWSIn[AWS Secrets Manager\norganization and profile credentials] --> ESO[External Secrets Operator]
    ESO --> K8sIn[Kubernetes Secrets]
    K8sIn --> XP
    Cloud --> Generated[Kubernetes connection Secrets]
    Generated --> ESO
    ESO --> AWSOut[AWS Secrets Manager\nper-stack outputs]

There are four separate ownership layers:

Argo CD owns controller installation, platform definitions, environment configuration, and request objects stored in Git.
Crossplane owns the composed Kubernetes managed resources and continuously reconciles their external Grafana objects.
External Secrets Operator (ESO) owns materialization of secret inputs and publication of generated secret outputs.
Grafana Cloud remains the external system of record, observed and corrected by the Grafana provider.

Argo CD must not also declare the managed resources emitted by the Composition. That would give two controllers ownership of the same Kubernetes objects. This is why application.resourceTrackingMethod: annotation and the Crossplane-aware health customizations in deploy/argocd/argocd-values.yaml matter — see Installation.

The vending API¶

The primary API is GrafanaCloudStackRequest. It is namespaced so teams or environments can be separated with Kubernetes namespaces and RBAC. Three access APIs — GrafanaCustomRoleBinding, GrafanaTeamAccess, and GrafanaContentAccessPolicy — sit beside it; see Configuration for every field.

The XRD uses defaultCompositionUpdatePolicy: Automatic and an enforcedCompositionRef. Existing requests therefore move to the latest Composition revision automatically after a platform update. Treat an XRD or function change like a production API release: render it, inspect the desired-resource diff, and roll it through a non-production request first.

How a request becomes managed resources¶

Each of the four kinds is backed by a single-step Crossplane Composition in Pipeline mode that calls the function-grafana-vending composition function (a Go program built from platform/function/). The function reads the request spec plus the platform-owned GrafanaVendingConfig Composition input and renders the complete set of desired Kubernetes managed resources — there is no templating language involved, the rendering logic is Go code covered by platform/function/fn_test.go.

Reconciliation and out-of-band changes¶

Crossplane providers poll the external APIs and compare observed state with desired state. The exact delay is the provider poll interval plus API and controller latency — it is not an immediate webhook response.

An administrator's out-of-band change (through the Grafana UI, say) is repaired only when the affected field's reconciliation mode says so:

Resource or field	Mode	Effect of an out-of-band edit
Stack name, region, labels, readiness flags	managed `forProvider` fields	Crossplane attempts to restore the value in the request
Stack deletion	`deleteProtection` plus no `Delete` policy	Normal controller-driven deletion is refused
Baseline dashboard JSON	`createOnly`	Initial JSON is supplied through `initProvider`; later UI edits are preserved
Baseline dashboard JSON	`enforced`	UI edits are detected and restored from Git-rendered desired state
Folder title and dashboard folder relationship	managed `forProvider` fields	Drift is restored even when dashboard JSON is create-only
Home dashboard UID	`createOnly`	Initial choice is set; a later administrator change is preserved
Home dashboard UID	`enforced`	A later administrator change is restored
SSO settings	`enforced`	Crossplane owns OAuth/SAML settings and repairs UI drift
SSO settings	`createOnly`	Crossplane initializes settings but does not own later changes
SSO settings	`observeOnly`	Crossplane observes the named provider and never creates or updates it
SSO settings	`disabled`	No SSO managed resource is desired
Plugins	listed	Crossplane reconciles listed installations; omitted plugins are not adopted
Custom roles, assignments, team sync	present in a binding	Crossplane repairs drift in the managed fields
Team direct members and preferences	present in `GrafanaTeamAccess`	Crossplane restores the declared direct members and preferences; external sync members are ignored when configured
Fixed/custom `RoleAssignmentItem`	present in `GrafanaTeamAccess`	Crossplane restores the role-to-team assignment without owning other actors assigned to that role
Folder or dashboard ACL	present in `GrafanaContentAccessPolicy`	Crossplane restores the complete declared ACL; omitted entries are removed by Grafana's whole-set permission API
OnCall outgoing-webhook data	`createOnly`	Initial generic payload is set; later UI template edits are preserved
OnCall outgoing-webhook data	`enforced`	Later UI template edits are restored from the Composition
Alerting contact-point payload	enabled	Crossplane always restores the relay payload and Secret-backed authorization contract

Changing SSO from enforced to createOnly moves oauth2Settings/samlSettings from forProvider to initProvider while retaining a stable external name for the provider — the supported handoff from platform ownership to administrator ownership. observeOnly removes write authority. disabled removes the managed-resource object from the Composition; because Delete is not permitted, the external SSO configuration remains but is no longer observed. Changing the identity type itself (e.g. generic_oauth to saml) is an identity-provider migration, not a routine mode toggle — plan a tested login and rollback path.

Argo CD should ignore Crossplane-generated resource churn rather than carrying broad ignore rules for the request itself. If the request in Git changes, Argo CD applies the request; the Composition then computes the resulting managed-resource change.

Baseline and optional resources¶

The baseline creates three portable starting points occupying the same resource slots as a traditional billing/usage, endpoints, and home-dashboard bundle. They are intentionally simple and generic; no source-specific dashboard JSON is copied or implied. Keep service-owned dashboards outside the stack identity API so an ordinary content release cannot disturb stack identity or credentials.

Everything else is a separately owned domain — alerting rule groups and notification policy, data sources, cloud integrations, SLOs, Synthetic Monitoring, OnCall schedules and escalation chains, Fleet Management, k6, ML, Asserts, and Assistant are all out of scope for automatic creation. See the provider-family table in the project README for the complete ownership map across all 111 managed-resource kinds the provider exposes.

Where state lives¶

Desired state lives in Git, as GrafanaCloudStackRequest and access-API objects under enabled/.
Composed managed-resource state lives in Kubernetes, generated by the composition function and never hand-edited — a hand-applied patch to a generated managed resource is overwritten on the next reconciliation and can turn an intended adoption into an attempted create.
External state lives in Grafana Cloud, observed and corrected by the Grafana provider on its poll interval.
Credential material never lives in Git or in a Composition input. It flows through ESO — see Secrets.

Next steps¶

Configuration — every request field.
Secrets — the credential rotation model in detail.
Security — supply-chain verification and the non-destructive lifecycle.