---
title: Operations
description: The camden deployment
---

# Operations

## Where it runs

camden, as a systemd service, reading `/opt/agent-sessions` read-only and exporting OTLP to the
local Alloy on `localhost:4318`. Alloy's Grafana Cloud routing is already owned by Fleet Management;
this service holds no Grafana credential and adds no Alloy configuration.

The transcript trees, their retention timer and the SQLite search index are owned elsewhere. This
service is a reader and is never in the path of archiving, pruning or restore.

## Do not duplicate the existing aggregate collector

`agent-session-metrics.timer` on camden already publishes five-minute metadata and aggregate metrics
about the archive itself: file counts, byte coverage, index freshness, parser errors. That job owns
the state of the archive. This service owns the content of the transcripts: per-turn, per-item and
per-tool detail. Do not re-emit its series.

## Backfill

Native client telemetry starts when it is switched on. The transcripts are retroactive. A ranged
backfill marks every metric point with `codingagent.backfill=true` so a window where both were live
can be excluded rather than double counted. Logs and spans retain their record attributes and
timestamps; `codingagent.backfill` is a metric-only dimension under the field-destination contract.

Run a backfill with both RFC3339 bounds:

```console
codingagent2otel -config /opt/codingagent2otel/config.yaml \
  -backfill-since 2026-01-01T00:00:00Z \
  -backfill-until 2026-02-01T00:00:00Z
```

The lower bound is inclusive and the upper bound is exclusive. A turn is assigned by its completion
time, falling back to its start time when no completion time exists. The command rejects a missing,
empty or reversed bound. With neither flag it runs the normal live tail.

Backfill scans the same trees in the configured bounded batches and uses the normal atomic byte-offset
checkpoint format. Each UTC-normalized window has a deterministic
`checkpoint.backfill-<sha256>.json` beside the configured live checkpoint. It never reads or writes
the live cursor. Repeating the exact window over unchanged input resumes at the end and exports no
duplicate series; appended records are picked up on the next run. Do not run the same window
concurrently because its two processes would share one checkpoint.

### Local throughput measurement

On 2026-09-22, `BenchmarkBoundedBackfill` processed a 6,145-record, approximately 0.94 MB synthetic
Codex fixture through bounded source reads, decoding, range selection, metric aggregation and atomic
checkpoint writes on an Apple M4 Pro (`darwin/arm64`). Three three-second samples were 3.55, 3.75
and 3.95 MB/s; the median was **3.75 MB/s**, or about **24,500 records/s** (251.1 ms per fixture).

```console
go test ./cmd/codingagent2otel -run '^$' \
  -bench '^BenchmarkBoundedBackfill$' -benchtime=3s -count=3
```

The benchmark uses 256 KiB and 512-record source caps and a disabled network exporter, so use the
byte rate to estimate local scan and decode time. Add collector and network headroom separately when
planning a full-history run; the slower of local processing and OTLP export determines elapsed time.

## Health

`health.listen` exposes liveness and the self-observability snapshot. A green health check is not
outcome proof: check `codingagent.selfobs.ingest_lag` and
`codingagent.selfobs.unhandled_record_types` for whether the service is actually keeping up and
actually understanding what it reads.

## When a client updates

A Codex or Claude Code release can change record shapes. `codingagent.selfobs.unhandled_record_types`
and `codingagent.drift_findings` are the alarms. An absent record type is a finding, not a default:
a decoder that quietly produces nothing looks identical to a quiet day.