Cloud Detection at Scale on a Laptop

How RSigma streams 1.9 million CloudTrail events through a community IR playbook

This is the sixth article in a series on RSigma. The first article introduced RSigma as a CLI tool for evaluating Sigma rules against JSON logs. The second covered running it as a streaming daemon with HTTP and NATS input, stateful correlation, and persistent state. The third showed how to convert Sigma rules into PostgreSQL SQL and run them against TimescaleDB. The fourth wired RSigma into the Grafana LGTM stack for dashboards, metrics, and dynamic alert routing. The fifth plugged live threat intelligence into Sigma rules at runtime via dynamic pipelines.

Five articles, five layers. None of them tested RSigma against a corpus large enough to expose the difference between a fast detection engine and a slow one. This article does. We take 1.9 million real CloudTrail events spanning three and a half years, point a community-curated AWS incident response playbook at them, and watch the v0.11.0 matcher optimizer earn its keep on a laptop. All the rules, configs, scripts, and the Grafana dashboard are in the companion repository.

The corpus you can finally test detection at scale on

Public CloudTrail datasets are surprisingly rare. Synthetic samples from AWS workshops are too small. GuardDuty findings are not raw events. Stratus Red Team and CloudGoat let you generate real CloudTrail by attacking your own lab, but they are not static corpora.

There is one major exception. In October 2020, Scott Piper of Summit Route (now at Wiz) released the CloudTrail logs from flaws.cloud, the first free training site for practicing AWS attacks. The dataset is the closest thing the AWS security community has to a canonical detection benchmark.

$ curl -L https://summitroute.com/downloads/flaws_cloudtrail_logs.tar \
    -o flaws_cloudtrail_logs.tar
$ tar -xf flaws_cloudtrail_logs.tar
$ ls flaws_cloudtrail_logs/ | head -3
flaws_cloudtrail00.json.gz
flaws_cloudtrail01.json.gz
flaws_cloudtrail02.json.gz

Twenty chunks of gzipped JSON. Each chunk decompresses to a single object with a Records[] array, the standard CloudTrail batch format:

$ gzcat flaws_cloudtrail_logs/flaws_cloudtrail00.json.gz | jq '.Records[0]'
{
  "eventVersion": "1.05",
  "userIdentity": {
    "type": "IAMUser",
    "principalId": "AIDAxxxxxxxxxxxxxxxx",
    ...
  },
  "eventTime": "2017-02-12T03:42:01Z",
  "eventSource": "iam.amazonaws.com",
  "eventName": "GetUser",
  "awsRegion": "us-east-1",
  "sourceIPAddress": "x.x.x.x",
  ...
}

The numbers worth knowing:

Anonymization was done with Latacora’s wernicke (no longer available on GitHub), which preserves data shape while randomizing values consistently. IP addresses still look like IP addresses, account IDs are still 12 digits, access keys still start with AKIA, and the same anonymized value appears wherever the original did. This means cross-event correlation still works. It also means the IPs in this dataset are not real attacker IPs. Do not add them to threat intel feeds.

The rule pack: an IR playbook, translated

For a corpus this size, the rule pack matters as much as the engine. We want one that is ATT&CK-mapped from the start, traces every rule back to a published source, and reflects a coherent incident response playbook rather than a grab bag of detections collected by topic. That way every match has a defensible meaning, and every miss points at a real coverage gap rather than at the rule pack’s randomness.

easttimor/aws-incident-response is exactly that. Maintained since 2019 and MIT-licensed, the repo catalogues roughly eighty CloudTrail event names that matter during AWS incident response, organized by service (IAM, S3, EC2, Lambda, GuardDuty, SecurityHub, Config, Macie, IAM Access Analyzer, Inspector, WAF) and grouped into named playbooks: root credential use, MFA removal, IAM privilege escalation, S3 permissions update, CloudTrail disruption, GuardDuty disruption, and so on. Each entry carries explicit MITRE ATT&CK technique and tactic IDs.

The repo expresses these detections as Athena SQL queries against an cloudtrail_xxxxxxxxxxxx table, plus deployable EventBridge rules in Terraform. They are precise, peer-reviewed, and exactly the kind of detection a SOC running CloudTrail through Athena would write by hand. They are also locked to AWS-specific tools.

We translate them to Sigma rules.

From Athena SQL to Sigma rule

Take one of the IR playbook’s most important detections: privilege escalation by adding permissions to a principal. Here is the Athena query as published in the easttimor repo:

SELECT *
FROM cloudtrail_000000000000
WHERE year = '####' AND month = '##' AND day = '##'
  AND eventSource = 'iam.amazonaws.com'
  AND eventName IN (
    'AttachUserPolicy', 'DetachUserPolicy',
    'AttachRolePolicy', 'DetachRolePolicy',
    'PutUserPolicy', 'PutGroupPolicy', 'PutRolePolicy',
    'DeleteUserPolicy', 'DeleteGroupPolicy', 'DeleteRolePolicy',
    'DeleteRolePermissionsBoundary'
  )
ORDER BY eventtime DESC

The same detection as a Sigma rule:

title: AWS IAM Privilege Escalation Through Permissions Update
id: 8c2c1f5a-1abc-4def-9876-aws-easttimor-priv-esc
status: stable
description: >
    Detects IAM API actions that attach, detach, or modify policies on
    users, groups, or roles. These actions are commonly used by attackers
    after credential compromise to expand access for an existing principal
    or to remove explicit deny boundaries.
references:
    - https://github.com/easttimor/aws-incident-response#privilege-escalation-adding-permissions
    - https://rhinosecuritylabs.com/aws/aws-privilege-escalation-methods-mitigation/
tags:
    - attack.persistence
    - attack.privilege_escalation
    - attack.t1098
    - attack.ta0003
    - attack.ta0004
logsource:
    product: aws
    service: cloudtrail
detection:
    selection:
        eventSource: iam.amazonaws.com
        eventName:
            - AttachUserPolicy
            - DetachUserPolicy
            - AttachRolePolicy
            - DetachRolePolicy
            - PutUserPolicy
            - PutGroupPolicy
            - PutRolePolicy
            - DeleteUserPolicy
            - DeleteGroupPolicy
            - DeleteRolePolicy
            - DeleteRolePermissionsBoundary
    condition: selection
falsepositives:
    - Routine IAM administration by trusted automation (cite by `userIdentity.arn`)
    - Infrastructure-as-code pipelines applying policy changes
level: high

The translation is mechanical for most entries. The IN (...) becomes a Sigma list under eventName. The eventSource = 'iam.amazonaws.com' clause becomes a separate selection key. The ATT&CK technique that the easttimor README documents inline becomes Sigma tags. The falsepositives block is the only piece that requires judgment from the rule author.

In the companion repo, every easttimor-derived rule is annotated with a header comment pointing back to the originating section in the upstream README, so the provenance trail is intact:

# Source: easttimor/aws-incident-response#privilege-escalation-adding-permissions
# Original Athena query type: API watchlist
# License: MIT (https://github.com/easttimor/aws-incident-response/blob/master/LICENSE)
title: AWS IAM Privilege Escalation Through Permissions Update
...

For the runs in this article we use the SigmaHQ AWS CloudTrail rule pack on its own: 55 rules covering bucket enumeration, root credential use, IAM persistence, STS misuse, IMDS abuse, GuardDuty/Config disruption, and so on. Each rule already carries ATT&CK tags. The companion repo also contains an in-progress easttimor-derived rule pack (with provenance footers as shown above) which extends coverage in ways the runs below will make explicit.

External validation that this pairing is the right one: Summit Route’s own dataset release post explicitly recommends pairing flaws.cloud with the easttimor queries. We are running the same playbook the dataset author intended, just in a different runtime.

rsigma fields confirms what the pack actually references in CloudTrail JSON:

$ rsigma fields -r rules/ --json | jq '.summary'
{
  "total_rules": 55,
  "total_correlations": 0,
  "total_filters": 0,
  "unique_fields": 21,
  "pipelines_applied": 0
}

Twenty one distinct CloudTrail field names. If any one of them was misspelled (the same class of bug that broke Okta detections in SigmaHQ/sigma#5964 before it was fixed), the affected rules would silently fail to match. Running rsigma fields in CI is the cheapest way to prevent it from ever happening to a pack like this one.

The streaming pipeline

The detection pipeline is the same shape as Article 4, retargeted at CloudTrail:

Vector does three things: read the gz files, decompress them, expand the Records[] envelope so each event becomes a standalone log record, then ship to RSigma over OTLP/HTTP. The Vector configuration:

# vector.toml

[sources.flaws_cloud]
type = "file"
include = ["data/flaws-cloud/*.json.gz"]
read_from = "beginning"
multiline.start_pattern = '^\{'
multiline.mode = "halt_before"
multiline.condition_pattern = '^\Z'
multiline.timeout_ms = 1000

[transforms.parse_cloudtrail]
type = "remap"
inputs = ["flaws_cloud"]
source = '''
    parsed = parse_json!(.message)
    records = parsed.Records
    .events = records
'''

[transforms.fan_out_records]
type = "remap"
inputs = ["parse_cloudtrail"]
source = '''
    . = .events
'''
# Vector's remap can also use the "lua" transform or "log_to_metric" patterns
# to expand arrays. For production volume, prefer the dedicated
# `aws_cloudtrail_logs` source if available in your Vector version.

[sinks.rsigma]
type = "otlp"
inputs = ["fan_out_records"]
endpoint = "http://localhost:9090"
encoding.codec = "json"
compression = "gzip"

The daemon side is one command:

rsigma daemon \
  --rules rules/ \
  --pipeline pipelines/cloudtrail_normalize.yml \
  --input http \
  --api-addr 127.0.0.1:9090

The processing pipeline (pipelines/cloudtrail_normalize.yml) maps a few CloudTrail field-name variations to what the rule pack expects. CloudTrail is mostly consistent (eventName, eventSource, userIdentity.* are standardized), so the pipeline is short.

The baseline run

To measure detection-engine throughput cleanly, separate from network and serialization overhead in the OTLP path, the benchmarks below use rsigma eval directly against the flattened NDJSON corpus. The eval and daemon paths share the same evaluation core, so per-event throughput is the same; eval just removes the variables that would otherwise muddy the numbers.

$ for f in flaws_cloudtrail*.json.gz; do gzcat "$f" | jq -c '.Records[]'; done > flaws_all.ndjson
$ wc -l flaws_all.ndjson
1939207

$ /usr/bin/time -l rsigma eval -r rules/ -e @flaws_all.ndjson > /dev/null
Loaded 55 rules from rules/
Processed 1939207 events, 68576 matches.
       16.76 real        16.04 user         0.60 sys
            14319616  maximum resident set size

Default settings, on an Apple Silicon laptop:

The throughput number is what RSigma achieves with the v0.11.0 matcher optimizer that ships in the default binary (Aho-Corasick batching, RegexSet DFA batching, CaseInsensitiveGroup shared case-folding). The two opt-in layers, bloom prefilter and cross-rule AC index, are still off.

A note on what 115k events per second means in practice. flaws.cloud collected 1.94 million events over 3.5 years. Our laptop chews through the whole archive in 17 seconds. A high-volume production AWS account producing 10 million CloudTrail events per day would replay through this same pipeline in about 90 seconds.

Turning on the optimizer layers

RSigma v0.11.0 added two optional prefilter layers, both off by default because they help in some scenarios and hurt in others. The release notes are explicit that these layers shine on large packs (1,000 rules and up) of substring-heavy detections against high-volume mostly-non-matching events. A 55-rule pack is not that scenario. The benchmarks below show what happens when you turn them on anyway.

Bloom prefilter

The bloom prefilter is a per-field trigram index that runs at the detection-item level. At rule load time, the engine extracts positive substring needles (|contains, |startswith, |endswith) from every rule and inserts each three-byte trigram into a per-field bloom filter. At eval time, for each string field value, the engine slides trigrams over the lowered haystack. If no trigram from any pattern is present, the matcher returns "definitely no match" without running.

$ /usr/bin/time -l rsigma eval -r rules/ -e @flaws_all.ndjson \
    --bloom-prefilter > /dev/null
Processed 1939207 events, 68576 matches.
       16.93 real        16.28 user         0.58 sys
            14254080  maximum resident set size

Match counts are identical, which is the invariant that matters. Differential fuzz tests in crates/rsigma-eval/tests/regression_eval.rs lock down "bloom on equals bloom off" for arbitrary inputs. The wall time is unchanged because the per-event trigram probing cost (about a microsecond) is a wash against the per-rule savings on a 55-rule pack.

Cross-rule Aho-Corasick prefilter

The cross-rule AC prefilter is a whole-rule pruner. At index build time, the engine collects all positive substring needles from every rule and builds one daachorse DoubleArrayAhoCorasick automaton per field. Pattern IDs map back to rule indices. At eval time, one overlapping scan on the lowered haystack marks which rules had at least one pattern hit. Rules that received zero hits are skipped entirely.

$ rsigma --version  # built with --features daachorse-index
rsigma 0.12.0

$ /usr/bin/time -l rsigma eval -r rules/ -e @flaws_all.ndjson \
    --bloom-prefilter --cross-rule-ac > /dev/null
Processed 1939207 events, 68576 matches.
       18.93 real        18.15 user         0.63 sys
            14450688  maximum resident set size

The cross-rule AC index actively slows the run down at this scale. That is not a contradiction with the release notes: the documented 68x to 101x speedups in BENCHMARKS.md are measured at 1,000, 5,000, and 10,000 substring-heavy rules where the per-event amortization dominates. With 55 rules the index build cost and the per-event AC scan cost are not amortized, and the layer is pure overhead.

This is exactly the case the v0.11.0 release notes call out:

“For typical mixed workloads (substring + exact + regex rules, events that hit multiple fields, smaller rule sets), the index adds build-time and lookup overhead with smaller wins or none, and can cause a slowdown.”

Reporting the slowdown honestly is more useful than fabricating a synthetic win.

When each layer earns its keep

The takeaway: the default optimizer is already strong enough that small CloudTrail packs do not benefit from the opt-in layers. Reach for them when your rule pack hits four digits.

Per-rule visibility, by ATT&CK tactic

The aggregate “68,576 matches” headline does not tell you which detections fired. Per-rule labels added in v0.9.0 do, and they group naturally because every SigmaHQ rule already carries attack.tXXXX tags.

Top fires by rule:

Severity breakdown: 54,848 low (80%), 11,151 medium (16%), 2,577 high (4%), 0 critical. By ATT&CK tactic (a single match can carry multiple tactic tags, so the totals overlap):

The companion repo’s Grafana dashboard (grafana/dashboards/cloud-detection.json) reuses the skeleton from Article 4, but every panel is grouped by ATT&CK tactic. Because every rule carries attack.taXXXX and attack.tXXXX tags, the dashboard tells a coverage story: which tactics fired, which never did, which fired so much they need refinement.

The 40 dead-weight rules tell the inverse story. Detections like aws_cloudtrail_guardduty_detector_deleted_or_updated, aws_cloudtrail_pua_trufflehog, aws_ec2_disable_encryption, and aws_efs_fileshare_modified_or_deleted never fire because flaws.cloud is a deliberately small lab without these services or attack patterns. That is a real coverage signal: the rule pack is broader than the corpus. In a real environment, the same dead-weight rules would be the ones to keep watching, because their absence in flaws.cloud reflects the lab's simplicity, not the rules' value.

Findings

The 1.94 million events span 2017–02–12 to 2020–10–07, exactly the 3.5 years cited by Summit Route. Five patterns dominate the matches and each one maps to a known flaws.cloud level or a textbook AWS attack pattern.

Root credential use (10,997 matches). Scott Piper used the Root account heavily while building the lab in early 2017. This is a clean true-positive in mechanism, even though the actor was authorized. The same rule on a real AWS account would be a P1 alert.

Bucket enumeration (12,122 matches). Every s3:ListBuckets call across 3.5 years of attackers running through the early flaws.cloud levels. flaws.cloud was designed around bucket discovery, so this rule fires on exactly the activity it was written for.

STS AssumeRole misuse (42,315 matches). The single largest category, and a known noisy detection. Every action by an IAM role technically uses AssumeRole at some point, so this rule is closer to a high-volume audit feed than an alert. In production, this is a candidate for downgrading to discovery telemetry rather than treating each fire as actionable.

IMDS credential abuse outside AWS (2,573 matches). The cleanest story in the dataset. Every single match traces to one EC2 instance role, arn:aws:sts::811596193553:assumed-role/aws:ec2-instance/i-aa2d3b42e5c6e801a. The matches concentrate in a tight burst on 2020-06-09 between 22:46 and 22:47 UTC, all from a single source IP (248.44.206.35, anonymized), running classic post-credential-theft AWS reconnaissance: DescribeTrails, DescribeStacks, DescribeAlarms, DescribeInstances, DescribeVolumes, DescribeSecurityGroups, DescribeFileSystems. This is exactly the Capital One breach pattern and exactly what flaws.cloud Level 5 was designed to teach: an attacker steals an EC2 IAM role from the metadata service via SSRF, then uses the credentials from outside AWS to enumerate everything they can. The rule caught the technique on the first sweep.

STS GetSessionToken misuse (209 matches). An attacker pattern where stolen long-term keys are exchanged for short-term session tokens to evade key-rotation detection. Lower volume than the IMDS abuse but a high-confidence indicator on its own.

$ jq -r 'select(.rule_title=="Malicious Usage Of IMDS Credentials Outside Of AWS Infrastructure") |
         "\(.event.eventTime) \(.event.sourceIPAddress) \(.event.eventName)"' \
    results/baseline.ndjson \
  | sort | uniq -c | sort -rn | head -5
  14 2020-06-09T22:46:27Z 248.44.206.35 DescribeTrails
  14 2020-06-09T22:46:24Z 248.44.206.35 DescribeStacks
  13 2020-06-09T22:47:02Z 248.44.206.35 DescribeStacks
  12 2020-06-09T22:46:30Z 248.44.206.35 DescribeAlarms
  10 2020-06-09T22:47:18Z 248.44.206.35 DescribeFileSystems

That two-minute window is a model incident. One stolen role, one external attacker, sixty seconds of recon, on a real laptop, surfaced by a community rule pack with no infrastructure beyond a binary and an NDJSON file.

What RSigma got right (and what it revealed)

Speed. 1,939,207 events evaluated against 55 rules on a laptop in 16.76 seconds. No SIEM, no Athena cost, no infrastructure beyond a binary and a flat file. Memory footprint stays under 15 MB the whole time.

The default optimizer is already the right answer for most packs: the opt-in --bloom-prefilter and --cross-rule-ac layers neither help nor hurt the bloom case, and actively slow the cross-rule case down by 12% at 55 rules. This is the documented behavior, and it matters: most teams running Sigma against CloudTrail will never need the opt-in layers. The default binary is fast enough.

Single binary, single rule artifact: the same SigmaHQ rule pack that powers this article also runs in any other Sigma-aware backend through rsigma convert. The detections are not locked to RSigma. They are not locked to AWS either. The IR playbook embedded in the rules is portable in a way Athena queries and EventBridge rules are not.

ATT&CK alignment for free: because every SigmaHQ rule carries attack.tXXXX and attack.taXXXX tags from the start, the per-rule metrics, Grafana dashboards, and detection summaries are all naturally grouped by tactic. There is no separate spreadsheet of "which rule covers which technique" because the rule pack and the metric labels carry the answer.

What the rule pack revealed about the corpus: forty of the 55 rules never fired. That is not a rule-pack problem; it is a coverage report on the dataset. flaws.cloud does not exercise EFS filesystems, traffic mirroring, GuardDuty disruption, or trufflehog reconnaissance, so detections for those patterns sit silent. On a real production CloudTrail those rules would be exactly the ones still worth running.

What CloudTrail cannot tell us: even with a comprehensive rule pack fully evaluated, CloudTrail does not log the contents of S3 GET requests by default, does not capture EC2 SSH sessions, does not record successful console logins from Identity Center without separate configuration, and is anonymized in this dataset. A finding of “no matches” against the data plane does not mean nothing happened on the data plane. The rule pack tells you what the audit log captured. It cannot tell you what was never logged.

The same gap-analysis discipline transfers to any CloudTrail deployment. List what your detection layer can and cannot see, then plan the next data source.

Wrapping up

Five articles into the series, RSigma has been a forensics CLI, a streaming daemon, a SQL compiler, an observability hub, and a dynamic-pipelines runtime. This article uses those layers in service of one question: can a community detection pack run end to end against a real cloud audit log corpus on a laptop, fast enough to be useful?

The answer is yes, in seventeen seconds, and the default optimizer is already strong enough that most operators will never reach for the opt-in flags.

The companion repository contains:

• The bash one-liner that flattens the flaws.cloud tar into NDJSON
• A reference Vector configuration for the production OTLP path
• The 55-rule SigmaHQ pack used for the runs above
• An in-progress easttimor-derived rule pack with provenance footers (worked example in the From Athena SQL to Sigma rule section above)
• A processing pipeline for CloudTrail field normalization
• A Grafana dashboard grouped by ATT&CK tactic
• The replay and benchmark scripts used for this article
• A docs/rule-pack.md provenance table and docs/attack-coverage.md coverage breakdown

If you adopt the rule pack, please open issues against the companion repo for any false positives or missing detections you find. Upstream contributions to easttimor/aws-incident-response and SigmaHQ are welcome too.

RSigma is open source under the MIT license:

cargo install --locked rsigma
# or
docker pull ghcr.io/timescale/rsigma:0.12.0

Credit: dataset by Scott Piper / Summit Route, anonymized with Latacora’s wernicke. IR playbook by easttimor. Detection rules from the SigmaHQ community.

Cloud Detection at Scale on a Laptop was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.