This module implements three main sets of C implementations that are used within SABnzbd:

• yEnc decoding and encoding using SIMD routines
• CRC32 calculations
• Non-blocking SSL-socket reading
• Marking files as sparse

Of course, they can also be used in any other application.

yEnc decoding and encoding using SIMD routines

yEnc decoding and encoding performed by using rapidyenc from animetosho, which utilizes x86/ARM/RISC-V SIMD optimised routines if such CPU features are available.

CRC32 calculations

Also from rapidyenc, which uses the crcutil library and a PCLMULQDQ/ARMv8-CRC folding approach for very fast CRC calculations.

See src/rapidyenc/VENDOR.md for the vendored version.

Non-blocking SSL-socket reading

When Python reads data from a non-blocking SSL socket, it is limited to receiving 16K data at once. This module implements a patched version that can read as much data is available at once. For more details, see the cpython pull request.

Marking files as sparse

Uses Windows specific system calls to mark files as sparse and set the desired size. On other platforms the same is achieved by calling truncate.

Utility functions

Use sabctools.bytearray_malloc(size) to get an bytearray that is uninitialized (not set to 0's). This is much faster than the built-in bytearray(size) because the data inside the new bytearray will be whatever is present in the memory block.

Use sabctools.rarfile_rar3_s2k as a native replacement for rarfile via rarfile.rar3_s2k = sabctools.rarfile_rar3_s2k.
It provides a significant speed increase for decrypting RAR4 headers when the password length exceeds 28 characters.

Installing

As simple as running:

pip install sabctools --upgrade

When you want to compile from sources, you can run in the sabctools directory:

pip install .

SIMD detection

To see which SIMD set was detected on your system, run:

python -c "import sabctools; print(sabctools.simd);"

The CRC32 routines are selected independently of the yEnc ones, so they can report a different set:

python -c "import sabctools; print(sabctools.crc_simd);"

Either is empty when no accelerated implementation was available for this CPU and a generic one is in use.

OpenSSL detection

To see if we could link to OpenSSL library on your system, run:

python -c "import sabctools; print(sabctools.openssl_linked);"

Testing

For testing we use pytest (install via pip install --group test) and test can simply be executed by browsing to the sabctools directory and running:

pytest

Note that tests can fail if git modified the line endings of data files when checking out the repository!