Previously we added support for tagged unions by generating a lookup
table from tag -> struct type containing all struct types in the union.
However, if the same struct union type was used in multiple places (or
was passed in to `decode(..., type=some_union)` multiple times), the
lookup table would be regenerated each time. This is suboptimal for two
reasons:
- It's doing unnecessary work. While this work ideally would only be
done a fixed number of times on setup (e.g. while creating all `Decoder`
objects used), it's also not uncommon for users to call the top-level
decode functions, which would end up recreating the lookup table on
every call.
- It results in excessive memory usage. While the memory usage itself is
likely negligible compared to other parts of the application, it can
lead to slowdowns in hot loops (especially if the tables are large).
Since `types.UnionType` doesn't have a `__dict__` attribute, we can't
cache the lookup table on the union object itself (like we do for
enums/literal types). Instead, we make use of a global cache `dict` with
a fixed capacity (hardcoded as 64 right now). When initializing a
decoder, the set of found struct types is looked up in the cache, and if
found the existing `StrLookup` object is used instead of building a new
one. Otherwise a new lookup table is constructed and added to the cache.
When the cache reaches capacity, the oldest entry is removed. Note that
this isn't true LRU, since it's not the most recently used element, but
the oldest element added. This is the same strategy used by the stdlib
`re` module for caching compiled regex patterns, and should be
sufficient for our needs.
The following benchmark (with the smallest types possible) was used to
measure the decrease in overhead by using the cache:
```python
import msgspec
from typing import Union
class T1(msgspec.Struct, tag=True):
    a: int
class T2(msgspec.Struct, tag=True):
    a: int
typ = Union[T1, T2]
msg = b'{"type":"T1","a":1}'
%timeit msgspec.json.decode(msg, type=typ)
```
- Master: 484 ns
- This PR: 415 ns
So for the simplest, smallest (2 type) lookup table, creating a new
table takes ~70 ns. This scales linearly with the number of struct types
in the union.