Parallel Streaming Pattern in Go: How to Scan Large S3 or GCS Buckets Significantly Faster

I was cleaning up a large GCS bucket when I discovered that listing the files, not deleting them, was the real bottleneck – list APIs are paginated and strictly sequential, 1000 files per request. The fix is to partition the bucket into lexicographic ranges and stream all of them concurrently. The same scan that took hours completes in minutes, at practically the same API cost, and the trick works…