The Open Audio Protocol experienced a consensus halt at ~2:00 AM PT on March 24, 2026, after the validator set dropped below the 2/3 supermajority required by CometBFT — the first of its kind in the history of the protocol.
The outage was caused by a combination of issues: a major operator’s DNS change reducing participation, another operator accidentally taking multiple validators offline, and legacy inconsistencies between consensus and application validator state.
Recovery involved coordinated validator restoration, a database migration to fix the validator set, and a protocol update to accelerate consensus rounds.
The network fully recovered by 4:47 PM PT on March 25, 2026.
Read the full postmortem: https://docs.openaudio.org/blog/postmortem-march-2026-consensus-halt/
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.