S3 Wal Buffer In WalrusDB

WalrusDB uses S3 Express One Zone as the durable persistence layer for the Write-Ahead Log (WAL). Writing to s3 for every transaction write incurs high latency and transactional costs. To solve this, we can implement an in-memory buffer that temporarily stores WAL writes and then syncs them to S3 asynchronously in batches. Overall design The design uses a lock-free, concurrent queue (ArrayQueue)…