serhiy-storchaka · GitHub

Bug report

math.isqrt() returns an incorrect result for an instance of an int
subclass with an overridden comparison operator if the argument is not
less than 2**64.

import math
class MyInt(int):
    def __lt__(self, other):
        return True
print(math.isqrt(MyInt(10**20)))
# 9999999999
# Expected: 10000000000

The arbitrary precision path ends with the check-and-correct comparison
n < a*a, which calls the overridden __lt__, so the result can be
off by one. It is the only place where an overridden operator can be
picked up: all other operations use the value of the argument directly.

The bug exists since math.isqrt() was added in 3.8.

Linked PRs

Read the original on github.com ↗