ViPiC-Ru · GitHub

added 2 commits

June 23, 2026 13:15
Add a new global option scanRateLimitKiBps that limits the total disk
read throughput consumed by folder scans across all folders combined.
The limit is enforced by a token-bucket rate limiter shared across all
concurrent scanning goroutines via an atomic pointer stored on the model.
This means that when multiple folders scan simultaneously, they draw
tokens from the same bucket, so the aggregate I/O stays within the
configured limit regardless of the maxFolderConcurrency setting.
The burst size is set to max(scanRateLimitKiBps * 1024, 32 KiB) to
ensure it is never smaller than a single read call, which would cause
WaitN to return an error.
Setting scanRateLimitKiBps to zero (the default) disables throttling
entirely. The limit is updated without restart in CommitConfiguration,
taking effect at the start of the next scan.
Useful on systems with slow mechanical hard drives where unconstrained
scanning saturates disk I/O and degrades overall system responsiveness.
Signed-off-by: Savenkov Ivan <vip-ic@mail.ru>
Rename the scanRateLimitKiBps option to maxDiskIOKbps to reflect its
broadened scope, and extend the rate limiter from folder scans to all
disk I/O performed during sync.
The following operations now draw tokens from the same shared limiter:
- Scan hashing reads (via rateLimitedReader, unchanged)
- Temp-file hashing in reuseBlocks (was passing nil limiter)
- Local block copy reads in copyBlockFromFile (ReadAt)
- Block writes to temp files in limitedWriteAt (WriteAt)
- Kernel-space copies via CopyRange in copyBlockFromFile
A new helper diskIOThrottle on sendReceiveFolder loads the limiter
atomically and consumes tokens in a loop capped at burst size. The loop
is necessary because sync block sizes (128 KiB–16 MiB) can exceed the
token bucket burst, which would cause WaitN to return an error
immediately. The scan path is unaffected as it reads in 32 KiB chunks
that never exceed the burst.
Setting maxDiskIOKbps to zero (the default) disables throttling. The
limit is updated without restart in CommitConfiguration.
Signed-off-by: Savenkov Ivan <vip-ic@mail.ru>
…iting
Adds a new maxDiskIOPS configuration option at both the global (options)
and per-folder level. When set, a token-bucket rate limiter throttles
every filesystem and file operation (open, read, write, stat, rename,
etc.) that causes a disk access.
A global limiter is shared across all folders; a per-folder limiter
applies only to that folder. Both must permit an operation before it
proceeds. The limiters can be updated at runtime via config reload
without restarting the affected folders.
Useful on systems with slow mechanical hard drives to avoid saturating
the disk during scans and syncs. Set to 0 (the default) for unlimited.
Signed-off-by: Savenkov Ivan <vip-ic@mail.ru>

calmh

Read the original on github.com ↗