jcrist · GitHub

@jcrist

A good portion of int encoding time was spent in
`PyLong_AsLongLongAndOverflow`/`PyLong_AsUnsignedLongLong`, and
associated branching on overflow. By poking at the cpython internals we
can:
- Avoid the cost of a black-box function
- Optimize branching for our use case (extract into a uint64_t + sign,
  erroring if the value doesn't fit in an `int64`/`uint64`).
- Avoid doing unnecessary work in the case of values > int64.
This optimization results in a 15-25% speedup in JSON integer encoding,
and a 25-35% speedup in msgpack integer encoding (for a large list of
integers).
Since the internal representation of integers in cpython hasn't changed
in years, and seems unlikely that it will change in the future, I don't
forsee this being a maintenance burden.

Read the original on github.com ↗