Skip to content

Observability

Effect already emits traces, metrics, and logs, and every alchemy runtime ships with the OTLP exporter built in — the exporter is a Layer, wired to resources through the binding infrastructure. With alchemy the receiving end is code too: datasets, ingest tokens, monitors, notifiers, and alarms live in the same Stack as the Workers and Functions they observe, so a threshold change is a diff in the PR — reviewable, revertable, answered by git blame.

Every Effect.withSpan opens a span, every Metric update records a data point, every Effect.logInfo ships a log line — and each fetch, queue message, cron fire, and invocation gets an http.server-style root span automatically. Where the signals go is decided by a telemetry Layer provided on the Function/Worker:

import * as Alchemy from "alchemy";
Effect.provide(
Layer.mergeAll(
Cloudflare.R2.ReadWriteBucketBinding,
Alchemy.Telemetry.layerOtlp({
url: "https://collector.example.com",
headers: { "x-api-key": apiKey },
}),
),
)

Any OTLP endpoint works — Axiom, an OTel collector, any OTLP-compatible vendor. Swap the layer, ship somewhere else; the code emitting the signals never changes. Buffered telemetry is flushed when each request’s scope closes (via waitUntil on Cloudflare, so export never delays a response). See Telemetry for the full mechanics.

The endpoint the exporter targets is a resource. An Axiom Dataset is declared per OTEL signal and exposes its OTLP endpoints as outputs:

src/observability.ts
import * as Axiom from "alchemy/Axiom";
export const Traces = Axiom.Dataset("Traces", {
name: "app-traces",
kind: "otel:traces:v1",
});
export const Logs = Axiom.Dataset("Logs", {
name: "app-logs",
kind: "otel:logs:v1",
});

Each dataset’s otelTracesEndpoint / otelLogsEndpoint / otelMetricsEndpoint attributes are Outputs — references you wire into whatever runs the exporter.

Mint a least-privilege ingest token, then provide Axiom.Telemetry on the Worker — a binding layer that wires the datasets and token in one line:

src/observability.ts
export const Ingest = Axiom.ApiToken("Ingest", {
name: "prod-ingest",
datasetCapabilities: {
"app-traces": { ingest: ["create"] },
"app-logs": { ingest: ["create"] },
},
});
// src/worker.ts — in the Worker's init Effect:
import { Ingest, Logs, Traces } from "./observability.ts";
Effect.provide(
Layer.mergeAll(
Cloudflare.R2.ReadWriteBucketBinding,
Axiom.Telemetry({ token: Ingest, traces: Traces, logs: Logs }),
),
)

Building the layer binds each dataset’s OTLP endpoint and the token’s Authorization header onto the Worker — the bearer travels as a secret_text binding (see Secrets) and never appears in plaintext. At runtime the built-in exporter ships each signal to its dataset — no exporter code in the Worker.

On Cloudflare you can skip the in-process exporter entirely: an ObservabilityDestination has Cloudflare push Workers Logs telemetry (traces, logs, or metrics) to an OTLP collector — no code changes in the Worker:

// in the Stack's generator
const traces = yield* Traces;
yield* Cloudflare.Workers.ObservabilityDestination("TracesExport", {
url: traces.otelTracesEndpoint,
headers: { authorization: `Bearer ${process.env.AXIOM_TOKEN}` },
logpushDataset: "opentelemetry-traces",
});

One destination per signal (opentelemetry-traces, opentelemetry-logs, opentelemetry-metrics). The endpoint URL and headers update in place; changing the dataset triggers a replacement.

Monitors and notifiers are resources too. A Notifier is an alert destination; a Monitor is a scheduled APL query that fires it:

const slack = yield* Axiom.Notifier("ops-slack", {
name: "ops-channel",
properties: {
slack: { slackUrl: process.env.SLACK_WEBHOOK_URL! },
},
});
yield* Axiom.Monitor("error-rate", {
name: "High error rate",
type: "Threshold",
aplQuery: `
['app-traces']
| where status >= 500
| summarize count() by bin_auto(_time)
`,
operator: "Above",
threshold: 100,
intervalMinutes: 5,
rangeMinutes: 5,
notifierIds: [slack.id],
});

Notifiers cover Slack, email, PagerDuty, Opsgenie, Discord, Teams, and custom webhooks; monitors come in Threshold, MatchEvent, and AnomalyDetection flavors. Raising the threshold is a one-line diff, and the same notifier is reused across every monitor in the stack.

The same idea on AWS: alarms reference metrics by name and fire SNS topics, and dashboards are structured documents — all declared next to the resources they observe:

import * as AWS from "alchemy/AWS";
const alerts = yield* AWS.SNS.Topic("Alerts");
const errors = yield* AWS.CloudWatch.Alarm("HighErrors", {
MetricName: "Errors",
Namespace: "AWS/Lambda",
Statistic: "Sum",
Period: 60,
EvaluationPeriods: 1,
Threshold: 1,
ComparisonOperator: "GreaterThanOrEqualToThreshold",
AlarmActions: [alerts.topicArn],
});
yield* AWS.CloudWatch.Dashboard("ApiHealth", {
DashboardBody: {
widgets: [
{
type: "metric",
width: 12,
properties: {
title: "Lambda errors",
view: "timeSeries",
stat: "Sum",
metrics: [["AWS/Lambda", "Errors", "FunctionName", "api"]],
},
},
{
type: "alarm",
width: 12,
properties: {
title: "Alarms",
alarms: [errors.alarmArn],
},
},
],
},
});

AWS.CloudWatch.CompositeAlarm ANDs/ORs alarm states into a higher-level alert (AlarmRule: 'ALARM("HighErrors") OR ALARM("HighLatency")'), and because stacks are per-stage, prod, staging, and pr-42 each get their own dashboards and alarms.

  • Telemetry — how the built-in exporters work across every runtime, and the override API
  • Axiom — datasets, tokens, monitors, notifiers, and dashboards as resources
  • Axiom observability guide — end-to-end setup for a Cloudflare Worker
  • Tutorial Part 6 — the step-by-step version
  • Secrets — how Redacted values like ingest tokens are bound