Making behavioral biometrics observable: BehaviorGuard telemetry

The reason most behavioral-biometrics integrations fail in production isn’t the math, the math works. The reason they fail is that the team running the production app doesn’t have the telemetry to triage a single false-positive case, doesn’t have the cohort metrics to detect score drift, doesn’t have the post-incident forensics to answer “did the biometrics flag this account before the fraud team did?”

A risk score on its own is unactionable. A risk score paired with the right per-event metadata, aggregated at the cohort level, surfaced through a structured metric pipeline, is what makes behavioral biometrics deployable beyond a demo.

Good telemetry is also how you confirm the system is catching the attackers it claims to; the BehaviorGuard threat model is the reference for which cases each metric should light up on. Here’s the schema BehaviorGuard emits and the metric surface I’ve found minimal-but-sufficient for an integration to graduate from “we shipped it” to “the fraud team trusts it.”

The per-event record

BehaviorGuard’s RiskEvent is the unit of observability. One emitted per scoring cycle (typically every 5-30 seconds during an active session). Schema:

public struct RiskEvent: Sendable, Codable {
    public let timestamp: Date
    public let sessionID: UUID
    public let userID: String?              // hashed; never PII
    public let scoreValue: Double           // 0.0–1.0
    public let scoreBand: RiskBand          // .normal/.elevated/.suspicious/.critical
    public let confidence: Double           // 0.0–1.0, model's certainty
    public let signalContributions: [SignalContribution]
    public let context: RiskContext
}

public struct SignalContribution: Sendable, Codable {
    public let signalKind: SignalKind       // .touch, .typing, .motion, etc.
    public let weight: Double               // 0.0–1.0
    public let zScore: Double               // distance from baseline
    public let stale: Bool                  // signal hasn't fired in last 30s
}

public struct RiskContext: Sendable, Codable {
    public let intentKind: IntentKind?      // .transfer, .login, .resetPassword
    public let amountBracket: AmountBracket? // .small/.medium/.large/.huge (no actual amounts)
    public let timeOfDay: TimeOfDay
    public let networkContext: NetworkContext
}

The two principles in this schema:

1. No PII. userID is a hash; amountBracket is categorical not numeric; nothing in the record identifies a person to a downstream telemetry consumer.
2. Per-signal attribution. The signalContributions array breaks down WHICH biometric features drove the score in this event. When a fraud analyst asks “why did this user get flagged?”, the answer isn’t “the model said so”; it’s “touch pressure was 3.2 standard deviations from baseline, scroll velocity was 2.1 sigma, but motion features were within baseline.” That story is investigable.

The four metrics every BehaviorGuard integration should publish

Beyond the per-event record, four aggregate metrics turn the data into operational signal:

1. Score distribution per intent kind

persona_risk_score_histogram{intent="transfer"}
persona_risk_score_histogram{intent="login"}
persona_risk_score_histogram{intent="reset_password"}

A scoring engine that produces near-zero scores 99.5% of the time and high scores 0.5% of the time is calibrated. A scoring engine that produces uniformly distributed scores is broken, the score has no information. Watch the shape of these histograms; bimodal is good, uniform is bad.

If you notice the transfer histogram drifting toward higher mean scores week over week, that’s either real population shift (your user base getting more anomalous, e.g. a fraud ring scaling up) or model drift (the baseline aging out). PSI drift detection catches the latter; the histogram comparison surfaces the former.

2. False-positive rate by manual review

persona_step_up_outcomes_total{outcome="user_confirmed"}
persona_step_up_outcomes_total{outcome="user_failed_biometric"}
persona_step_up_outcomes_total{outcome="user_abandoned"}
persona_step_up_outcomes_total{outcome="fraud_team_confirmed_legitimate"}
persona_step_up_outcomes_total{outcome="fraud_team_confirmed_fraud"}

The first three are the user’s reaction to a step-up prompt. The last two are the fraud team’s adjudication. Together they give you:

• False-positive rate = fraud_team_confirmed_legitimate / (fraud_team_confirmed_legitimate + fraud_team_confirmed_fraud)
• Friction rate = (user_abandoned + user_failed_biometric) / step_up_total
• Conversion rate from step-up to user confirmation

A healthy deployment trends toward < 5% false-positive (the fraud team rarely overrides the SDK’s “this user is real, just unusual” call) and a friction rate below 2% (legitimate users complete step-up smoothly).

3. Signal staleness rates per platform

persona_signal_staleness_ratio{signal="motion", platform="ios_17"}
persona_signal_staleness_ratio{signal="motion", platform="ios_18"}
persona_signal_staleness_ratio{signal="motion", platform="ios_19"}

When a signal goes stale, signalContributions[i].stale == true, it means BehaviorGuard couldn’t collect that biometric in the last 30 seconds. Causes:

• iOS deprecated the API in a minor version bump
• A user’s device permissions changed (revoked motion access)
• A bug in a specific iOS version’s collector

If you see one signal’s staleness rate jump on a specific platform version, that’s the leading indicator for “Apple changed something.” You’ll see this signal before the user complaints arrive. The platform-version axis is essential, pooled across platforms, the signal disappears.

4. Score → outcome correlation by amount bracket

persona_risk_outcome_table{
    amount_bracket="huge",
    score_band="critical"
} = { fraud_confirmed: N, legitimate_confirmed: M }

This is the table that fraud teams use to set step-up thresholds. The pattern you want to see: at high amount brackets, even a “elevated” score correlates with non-trivial fraud rates; at small amounts, only “critical” scores warrant intervention. The score-band → outcome correlation should be MORE selective at higher amounts (because the cost of friction is higher) and LESS selective at lower amounts (because the cost of false negatives is higher, but the user-experience cost of step-up is lower).

If the table looks the same across amount brackets, same fraud rate per score band regardless of amount, your model is amount-agnostic, which is probably wrong for real fraud distributions.

The forensic schema for incident response

Beyond steady-state metrics, you need a retention policy for the per-event records that supports post-hoc investigation. Recommendation:

• 30 days at full per-event resolution (every RiskEvent for every session)
• 180 days at session-aggregate resolution (one record per session: max score, mean score, intent kind, outcome)
• 2 years at cohort-aggregate resolution (daily/weekly summary by amount bracket × intent kind)

The 30-day full-resolution window is what lets your fraud team investigate a specific case. The 180-day session-aggregate retention is what lets your security team correlate a known-compromised account back to its first anomalous session. The 2-year cohort-aggregate retention is what lets you tune the score thresholds over time.

GDPR / CCPA considerations:

• userID should be a hash that’s deletable on user request (i.e., the hash function is salt-keyed, and deleting the salt for that user invalidates the link). BehaviorGuard’s auditLog supports this via the purgeUserData() method.
• The RiskEvent itself is biometric data and falls under enhanced protections in some jurisdictions (notably Illinois BIPA). The framework allows you to selectively redact signalContributions from the persisted record while keeping scoreBand for the metric pipeline.

Plumbing the metrics

The minimum-viable pipeline: BehaviorGuard’s TelemetryReporter protocol takes a RiskEvent and dispatches to wherever you collect metrics, Datadog, Honeycomb, an OpenTelemetry collector, your own backend. The framework ships a default OpenTelemetryReporter that emits each of the four metric families above as OTLP traces and metrics.

let telemetry = OpenTelemetryReporter(
    endpoint: URL(string: "https://otel.your-collector.com")!,
    apiKey: ProcessInfo.processInfo.environment["OTEL_API_KEY"]!,
    serviceName: "your-app",
    redactPolicy: .standard            // strip userID hash + amount bracket on egress
)
let persona = try BehaviorGuard(
    configuration: BehaviorGuardConfiguration.default,
    components: .init(telemetry: telemetry)
)

What this gives you in a dashboard the first day after wiring:

• Score histogram per intent kind (so you can see the calibration shape)
• Step-up outcome funnel (so you can see what step-up actually does in the wild)
• Signal staleness ratios (so an iOS update that breaks a sensor is visible the same day)

What it gives you the first time a real incident happens:

• A trace ID per session that connects the model’s score to the user’s outcome
• Per-signal contribution breakdown so the post-mortem isn’t “the model decided”

The goal is the same as any production system: make the model’s decisions inspectable to the team running the app. Behavioral biometrics is just a model that operates over a different feature set than typical fraud rules, it deserves the same observability scaffolding.