Transactional Job Queues and the Two Generals’ Problem

Our worst outage in a decade of running Hypothesis, and how we made sure it never happened again.