jcrist · GitHub

This adds full support for encoding/decoding generic msgspec.Struct types. Fixes #193.

A quick demo:

from typing import Generic, TypeVar
from msgspec import Struct, json, ValidationError
T = TypeVar("T")
class Paginated(Struct, Generic[T]):
    page: int
    per_page: int
    total_count: int
    items: list[T]
class Order(Struct):
    item: str
    count: int
msg = """
{
    "page": 2,
    "per_page": 5,
    "total_count": 99,
    "items": [
        {"item": "apple", "count": 3},
        {"item": "banana", "count": 2},
        {"item": "carrot", "count": 10},
        {"item": "durian", "count": 1},
        {"item": "eggplant", "count": 4}
    ]
}
"""
# Decoding uses the parametrized type to validate
print(json.decode(msg, type=Paginated[Order]))
#> Paginated(
#>     page=2,
#>     per_page=5,
#>     total_count=99,
#>     items=[
#>         Order(item='apple', count=3),
#>         Order(item='banana', count=2),
#>         Order(item='carrot', count=10),
#>         Order(item='durian', count=1),
#>         Order(item='eggplant', count=4)
#>     ]
#> )
# A validation error is raised if the message doesn't match
bad_msg = """
{
    "page": 1,
    "per_page": 1,
    "total_count": 1,
    "items": [
        {"item": "apple", "count": "oops"}
    ]
}
"""
try:
    json.decode(bad_msg, type=Paginated[Order])
except ValidationError as err:
    print(err)
#> ValidationError("Expected `int`, got `str` - at `$.items[0].count`")

Recursive and complicated parametrizations also should work, including the weird challenge posted on the pydantic twitter:

from __future__ import annotations
from typing import Generic, TypeVar
import msgspec
T = TypeVar("T")
class MyGenericModel(msgspec.Struct, Generic[T]):
    foobar: list[MyGenericModel[T]] | None
    spam: T
MyGenericModelList = MyGenericModel[list[T]]
class MyGenericModelIntList(MyGenericModelList[int]):
    pass
msg = b'{"foobar": [{"foobar": null, "spam": [1]}], "spam": [1, 2, 3]}'
m = msgspec.json.decode(msg, type=MyGenericModelIntList)
print(m)
#> MyGenericModelIntList(
#>   foobar=[MyGenericModel(foobar=None, spam=[1])],
#>   spam=[1, 2, 3]
#> )

In common usage, generics should have no runtime overhead. Since msgspec doesn't validate on __init__ we don't need to generate a new type for each parametrization - type type of Paginated[Order] is typing._GenericAlias. No magic on our side needed!

The scoping rules and behavior for handling complicated parametrized generics are underdocumented in the corresponding PEPs. I believe we've covered all the edge cases here, but would still love for a few users to try this out before it's released. If this passes tests I plan on merging it early, but would hope someone could try this out before it's released.

Read the original on github.com ↗