Observability
Effect already emits traces, metrics, and logs, and every alchemy
runtime ships with the OTLP exporter built in — the exporter is a
Layer, wired to resources through the binding infrastructure. With
alchemy the receiving end is code too: datasets, ingest tokens,
monitors, notifiers, and alarms live in the same Stack as the
Workers and Functions they observe, so a threshold change is a diff
in the PR — reviewable, revertable, answered by git blame.
OTel everywhere
Section titled “OTel everywhere”Every Effect.withSpan opens a span, every Metric update records a
data point, every Effect.logInfo ships a log line — and each fetch,
queue message, cron fire, and invocation gets an http.server-style
root span automatically. Where the signals go is decided by a
telemetry Layer provided on the Function/Worker:
import * as Alchemy from "alchemy";
Effect.provide( Layer.mergeAll( Cloudflare.R2.ReadWriteBucketBinding, Alchemy.Telemetry.layerOtlp({ url: "https://collector.example.com", headers: { "x-api-key": apiKey }, }), ),)Any OTLP endpoint works — Axiom, an OTel collector, any
OTLP-compatible vendor. Swap the layer, ship somewhere else; the code
emitting the signals never changes. Buffered telemetry is flushed
when each request’s scope closes (via waitUntil on Cloudflare, so
export never delays a response). See
Telemetry for the full
mechanics.
Provision the receiving end
Section titled “Provision the receiving end”The endpoint the exporter targets is a resource. An
Axiom Dataset is declared per OTEL signal and exposes its
OTLP endpoints as outputs:
import * as Axiom from "alchemy/Axiom";
export const Traces = Axiom.Dataset("Traces", { name: "app-traces", kind: "otel:traces:v1",});export const Logs = Axiom.Dataset("Logs", { name: "app-logs", kind: "otel:logs:v1",});Each dataset’s otelTracesEndpoint / otelLogsEndpoint /
otelMetricsEndpoint attributes are Outputs —
references you wire into whatever runs the exporter.
Bind the datasets to the runtime
Section titled “Bind the datasets to the runtime”Mint a least-privilege ingest token, then provide Axiom.Telemetry
on the Worker — a binding layer that wires the datasets and token in
one line:
export const Ingest = Axiom.ApiToken("Ingest", { name: "prod-ingest", datasetCapabilities: { "app-traces": { ingest: ["create"] }, "app-logs": { ingest: ["create"] }, },});
// src/worker.ts — in the Worker's init Effect:import { Ingest, Logs, Traces } from "./observability.ts";
Effect.provide( Layer.mergeAll( Cloudflare.R2.ReadWriteBucketBinding, Axiom.Telemetry({ token: Ingest, traces: Traces, logs: Logs }), ),)Building the layer binds each dataset’s OTLP endpoint and the
token’s Authorization header onto the Worker — the bearer travels
as a secret_text binding (see Secrets) and
never appears in plaintext. At runtime the built-in exporter ships
each signal to its dataset — no exporter code in the Worker.
Export from the platform
Section titled “Export from the platform”On Cloudflare you can skip the in-process exporter entirely: an
ObservabilityDestination has Cloudflare push Workers Logs telemetry
(traces, logs, or metrics) to an OTLP collector — no code changes in
the Worker:
// in the Stack's generatorconst traces = yield* Traces;
yield* Cloudflare.Workers.ObservabilityDestination("TracesExport", { url: traces.otelTracesEndpoint, headers: { authorization: `Bearer ${process.env.AXIOM_TOKEN}` }, logpushDataset: "opentelemetry-traces",});One destination per signal (opentelemetry-traces,
opentelemetry-logs, opentelemetry-metrics). The endpoint URL and
headers update in place; changing the dataset triggers a replacement.
Alerts in code
Section titled “Alerts in code”Monitors and notifiers are resources too. A Notifier is an alert
destination; a Monitor is a scheduled APL query that fires it:
const slack = yield* Axiom.Notifier("ops-slack", { name: "ops-channel", properties: { slack: { slackUrl: process.env.SLACK_WEBHOOK_URL! }, },});
yield* Axiom.Monitor("error-rate", { name: "High error rate", type: "Threshold", aplQuery: ` ['app-traces'] | where status >= 500 | summarize count() by bin_auto(_time) `, operator: "Above", threshold: 100, intervalMinutes: 5, rangeMinutes: 5, notifierIds: [slack.id],});Notifiers cover Slack, email, PagerDuty, Opsgenie, Discord, Teams,
and custom webhooks; monitors come in Threshold, MatchEvent, and
AnomalyDetection flavors. Raising the threshold is a one-line diff,
and the same notifier is reused across every monitor in the stack.
CloudWatch alarms and dashboards
Section titled “CloudWatch alarms and dashboards”The same idea on AWS: alarms reference metrics by name and fire SNS topics, and dashboards are structured documents — all declared next to the resources they observe:
import * as AWS from "alchemy/AWS";
const alerts = yield* AWS.SNS.Topic("Alerts");
const errors = yield* AWS.CloudWatch.Alarm("HighErrors", { MetricName: "Errors", Namespace: "AWS/Lambda", Statistic: "Sum", Period: 60, EvaluationPeriods: 1, Threshold: 1, ComparisonOperator: "GreaterThanOrEqualToThreshold", AlarmActions: [alerts.topicArn],});
yield* AWS.CloudWatch.Dashboard("ApiHealth", { DashboardBody: { widgets: [ { type: "metric", width: 12, properties: { title: "Lambda errors", view: "timeSeries", stat: "Sum", metrics: [["AWS/Lambda", "Errors", "FunctionName", "api"]], }, }, { type: "alarm", width: 12, properties: { title: "Alarms", alarms: [errors.alarmArn], }, }, ], },});AWS.CloudWatch.CompositeAlarm ANDs/ORs alarm states into a
higher-level alert (AlarmRule: 'ALARM("HighErrors") OR ALARM("HighLatency")'), and because stacks are per-stage, prod,
staging, and pr-42 each get their own dashboards and alarms.
Where next
Section titled “Where next”- Telemetry — how the built-in exporters work across every runtime, and the override API
- Axiom — datasets, tokens, monitors, notifiers, and dashboards as resources
- Axiom observability guide — end-to-end setup for a Cloudflare Worker
- Tutorial Part 6 — the step-by-step version
- Secrets — how
Redactedvalues like ingest tokens are bound