Macro unsynn

Source

macro_rules! unsynn {
    (enum $name:ident { $($variant:ident),* }) => { ... };
    (struct $name:ident { $($member:ident: $parser:ty),* }) => { ... };
    (struct $name:ident ( $($parser:ty),*);) => { ... };
    (trait $name:ident;) => { ... };
    (trait $name:ident{}) => { ... };
    (impl { $($trait:ident $semicolon_or_block:tt)+ }) => { ... };
    (impl $trait:ident for $type:ty {$body:tt}) => { ... };
    (impl $trait:ident for $($type:ty),+;) => { ... };
    (fn $function:ident($params:tt) {$body:tt}) => { ... };
    (use $($path:path)+ $(as $alias:ident)?) => { ... };
    (keyword $name:ident = keyword_or_group;) => { ... };
    (keyword $name:ident != keyword_or_group;) => { ... };
    (operator $name:ident = "punct";) => { ... };
    (predicatetrait $name:ident $(: $super:ident $(+ $supers:ident)*)?;) => { ... };
    (predicateflag $name:ident = Enable $(for $($trait:ident),+)?;) => { ... };
    (predicateflag $name:ident = Disable $(for $($trait:ident),+)?;) => { ... };
    (predicateflag $name:ident = TokensRemain $(for $($trait:ident),+)?;) => { ... };
    (predicateflag $name:ident $(for $($trait:ident),+)?;) => { ... };
}

Expand description

This macro supports the definition of enums, tuple structs and normal structs and generates Parser and ToTokens implementations for them. It will derive Debug. Generics/Lifetimes are not supported on the primary type. Note: eventually a derive macro for Parser and ToTokens will become supported by a ‘unsynn-derive’ crate to give finer control over the expansion. #[derive(Copy, Clone)] have to be manually defined. Keyword and operator definitions can also be defined, they delegate to the keyword! and operator! macro described below. All entities can be prefixed by pub to make them public. Type aliases, function definitions, macros and use statements are passed through. This makes thing easier readable when you define larger unsynn macro blocks.

The macro definition above is simplified for readability struct, enum and type definitions can include most of the things normal rust definitions can do. This also includes definitions of members of structs and enums:

Any number of attributes (#[...]), including documentation comments. Note that the unsynn macros have limited support for automatically generation documentation. This auto-generated documentation is appended after the user supplied docs.
structs, enums, types and members can exported with the usual pub declarations.
struct, enum, and type definitions support generics and lifetime parameters. These can include trait bounds (simple identifiers, qualified paths like std::fmt::Debug, and lifetime bounds like 'static) and defaults. Multiple bounds can be combined with +. Trait bounds can be defined with where-clauses as well. Lifetime parameters (e.g., <'a>, <'a, T>) are fully supported for structs. Enums and tuple structs support lifetime parameters except when combined with inline impl blocks. HRTB (Higher-Ranked Trait Bounds) are not yet supported. See the COOKBOOK for examples.

Common for enum and struct variants is that entries are tried in order. Disjunctive for enums and conjunctive in structures. This makes the order important, e.g. for enums, in case some entries are subsets of others.

Enum variants without any data will never be parsed and will not generate any tokens. For parsing a enum that is optional one can add a variant like None(Nothing) at the end (at the end is important, because Nothing always matches).

The unsynn!{} macro is still a declarative macro and is somewhat limited in what it can do. We extend it as needed and provide some convenient rust syntax extensions, but some standard rust syntax is just hard to impossible to support.

§Example

// Define some types
unsynn!{
    keyword MyKeyword = "keyword";
    // all items can be declared pub/pub(..) etc
    pub(crate) operator MyOperator = "+++";

    enum MyEnum {
        /// Entries can have attributes/doc comments
        Ident(Ident),
        Braced(BraceGroup),
        Text(LiteralString),
        Number(LiteralInteger),
        Struct{
            keyword: MyKeyword,
            id: Ident,
        },
        // finally if nothing of the above matched, this will match.
        None(Nothing),
        // won't be parsed/matched at all
        Empty,
    }

    // With generics - qualified paths work, no need to import
    struct MyStruct<T: std::fmt::Debug = i32> {
        text: LiteralString,
        number: T,
    }

    // Multiple bounds with lifetime bound
    struct GenericContainer<T: Clone + std::fmt::Display + 'static> {
        value: T,
    }

    struct MyTupleStruct(Ident, LiteralString);

    // Lifetime parameters are also supported:
    //struct WithLifetime<'a> { /* ... */ }
    //struct MixedParams<'a, T> { /* ... */ }

    // type definitions are pass-through.
    pub type Alias = MyStruct<LiteralInteger>;

    // functions are pass though too
    fn testfn() -> bool { true }
}

// Create an iterator over the things we want to parse
let mut token_iter = r#"
    // some enum variants
    ident { within brace } "literal string" 1234 ()
    // MyStruct fields
    "literal string" 1234
    // MyTupleStruct fields
    ident "literal string"
    // MyKeyword and MyOperator
    keyword +++
"#.to_token_iter();

// Use the defined types
let MyEnum::Ident(myenum_ident) = MyEnum::parse(&mut token_iter).unwrap() else { panic!()};
let MyEnum::Braced(myenum_braced) = MyEnum::parse(&mut token_iter).unwrap() else { panic!()};
let MyEnum::Text(myenum_text) = MyEnum::parse(&mut token_iter).unwrap() else { panic!()};
let MyEnum::Number(myenum_number) = MyEnum::parse(&mut token_iter).unwrap() else { panic!()};
// the () will not be consumed by the MyEnum but match None(Nothing)
let myenum_nothing = MyEnum::parse(&mut token_iter).unwrap();
// consume the ()
<ParenthesisGroup>::parse(&mut token_iter).unwrap();

let my_struct =  Alias::parse(&mut token_iter).unwrap();
let my_tuple_struct =  MyTupleStruct::parse(&mut token_iter).unwrap();
let my_keyword =  MyKeyword::parse(&mut token_iter).unwrap();
let my_operator =  MyOperator::parse(&mut token_iter).unwrap();

§Traits within the `unsynn!` macro (EXPERIMENTAL)

The unsynn! macro has limited but ergonomic support for defining and implementing traits. The main purpose for this is to be able to define simple markers and accessors this helps with compile-time validation of grammars.

Trait definitions are passed though. There are simplifications that they are allowed to end with a ; instead a {}. This signifies simple marker traits that have no methods. The form trait {Name; ...} is supported to define a set of simple markers in one go. This syntax not support traits with methods.

Trait implementations are either passed through or are a impl {} block directly following an ADT definition. In the later case most boilerplate is left out and will be auto generated by the macro.

§Generic Types and Trait Bounds

Generic type parameters and lifetime parameters are fully supported:

Lifetime parameters: <'a>, <'a, 'b>, or mixed <'a, T>
Simple trait names: T: Clone
Qualified paths: T: std::fmt::Debug (no need to import the trait)
Lifetime bounds: T: 'static (can appear anywhere in the bound list)
Multiple bounds: T: Clone + std::fmt::Debug + 'static (any combination)
Where clauses: Support the same bound syntax as inline bounds

Examples:

struct WithLifetime<'a> { /* ... */ }
struct MultipleLifetimes<'a, 'b> { /* ... */ }
struct MixedParams<'a, T: Clone> { /* ... */ }

Limitations:

Generic type arguments in bounds (e.g., T: Trait<U>) are not supported
HRTB (Higher-Ranked Trait Bounds like for<'a>) are not yet supported
Lifetime parameters on tuple structs and enums with inline impl blocks are not supported (use separate impl blocks outside the macro instead)

See the COOKBOOK for detailed examples and patterns.

Trait support is very basic and experimental, it is unspecified what features are supported. Try it out, we promise not to break working things (if possible). For trait definitions that are not supported within the unsynn macro it is still possible and advised to define them outside of the macro block.

§Example

unsynn!{
    // Marker traits can be defined in a block
    // with attributes and visibility at the whole block
    #[doc = "This is a marker trait"] // applies to each item
    pub trait {
        TestMarker;
        TestMarker2;
    }
    // or per item
    trait {
        pub MarkerPub;
        MarkerPrivate;
    }

    // ending with a semicolon, supertraits are supported
    trait TestMarker3: TestMarker + TestMarker2;
    // or a brace
    trait TestMarker4 {}

    // Normal rust syntax for accessor methods
    trait SimpleAccessor { fn get(&self) -> bool;}

    // simplified trait impl block following the struct definition
    pub struct SimpleStruct{
        flag: bool
    } impl {
        #[doc = "impl attributes go here"]
        TestMarker;
        TestMarker2 {}
        SimpleAccessor {fn get(&self) -> bool {self.flag}}
    }

    // Marker traits or traits that have blanket implementations can
    // be implemented by a comma separated list
    impl TestMarker for i32, u32, i64, u64;

    // normal rust syntax, passthough trait impl
    impl SimpleAccessor for bool {fn get(&self) -> bool {*self}}
}

// implementing traits outside of the macro works as usual
impl TestMarker4 for SimpleStruct {}

§Extended Syntax Forms

The unsynn! macro extends standard Rust syntax with these conveniences:

§Semicolon Terminators

Traits and tuple structs can end with ; instead of {}:

unsynn! {
    trait MyMarker;  // Instead of: trait MyMarker {}
    struct MyTuple(Ident);  // Semicolon instead of no punctuation
}

§Trait Blocks

Define multiple marker traits at once using trait { Name1; Name2; }:

unsynn! {
    trait {
        MarkerA;
        MarkerB;
        MarkerC;
    }
}

§Inline Impl Blocks

Apply traits directly after struct/enum definitions with impl { Trait; }:

unsynn! {
    // Semicolon is optional before impl
    struct MyStruct(Ident) impl { Marker; }
    // Or with semicolon (both work)
    struct OtherStruct(Ident); impl { Marker; }
}

§Multi-Type Impl

Implement traits for multiple types with impl Trait for Type1, Type2, Type3;:

unsynn! {
    trait MyTrait;
    impl MyTrait for Ident, LiteralInteger, TokenTree;
}

§Custom Parsing with `parse_with`

Transform or validate parsed values using closure syntax. See Parse::parse_with() for the underlying method this uses.

Validation (without from): Verify parsed value meets requirements

unsynn! {
    // Only accept positive integers
    struct PositiveInt(LiteralInteger);
    parse_with |this, tokens| {
        if this.0.value() > 0 {
            Ok(this)
        } else {
            Error::other(None, tokens, "must be positive".into())
        }
    };
}

Transformation (with from Type:): Parse as one type, transform to another

unsynn! {
    // Parse integer as bool (0 = false, non-zero = true)
    struct BoolInt(bool) from LiteralInteger:
    parse_with |value, _tokens| {
        Ok(Self(value.value() != 0))
    };
}

The closure receives:

value - The parsed value (type after from, or Self without from)
tokens - Reference to TokenIter for error reporting

Returns: Result<Self> where errors use Error type

§Custom Token Emission with `to_tokens`

Customize how types are emitted back to tokens. See ToTokens::to_tokens() for the trait method this implements.

unsynn! {
    // Emit booleans as custom keywords
    struct BoolKeyword(bool);
    to_tokens |s, tokens| {
        if s.0 {
            Ident::new("TRUE", Span::call_site()).to_tokens(tokens);
        } else {
            Ident::new("FALSE", Span::call_site()).to_tokens(tokens);
        }
    };
}

The closure receives:

self - Reference to the value being emitted
tokens - Mutable reference to TokenStream to append to

§Combining `parse_with` and `to_tokens`

Both clauses are independent and can be used together:

unsynn! {
    // Parse int as bool, emit bool as int
    struct BoolInt(bool) from LiteralInteger:
    parse_with |value, _tokens| { Ok(Self(value.value() != 0)) }
    to_tokens |s, tokens| {
        Literal::u64_unsuffixed(if s.0 {1} else {0}).to_tokens(tokens);
    };
}

Clause Order: For tuple structs: struct Name(Fields) [from Type:] [parse_with ...] [to_tokens ...] [impl {...}];

The parse_with and to_tokens clauses are independent and optional. The from clause requires parse_with. Impl blocks always come last.

See the COOKBOOK for more examples and patterns.

§Keywords and Operators

Define custom keywords and operators within the macro. See also the standalone keyword! and operator! macros.

Keywords: Match specific identifier strings

unsynn! {
    keyword If = "if";
    keyword While = "while";
    keyword Function = "fn";
}

Operators: Match specific punctuation sequences

unsynn! {
    operator Plus = "+";
    operator Arrow = "->";
    operator DoubleColon = "::";
}

See operator::names for a comprehensive list of predefined operators.

§Parse Predicates

Parse predicates provide zero-cost compile-time control over parser behavior using type-level constraints. The unsynn! macro supports two special forms for defining predicates:

predicatetrait: Defines custom context traits that extend PredicateOp, automatically implementing them for universal predicates (Enable, Disable, TokensRemain) and logical operators (AllOf, AnyOf, OneOf, Not).
predicateflag: Creates zero-sized newtype wrappers around base predicates that implement custom traits, enabling type-safe context validation at compile time.

Example:

unsynn! {
    // Define context traits
    predicatetrait ExpressionContext;
    predicatetrait StatementContext;
     
    // Create context-specific predicates
    predicateflag InExpression = Enable for ExpressionContext;
    predicateflag InStatement = Disable for ExpressionContext;
     
    // Use predicates as type constraints
    pub struct StructLiteral<P: ExpressionContext = InExpression> {
        _guard: P,  // Zero-sized, no runtime cost
        name: Ident,
    }
}

See the COOKBOOK Parse Predicates section and the predicates module for detailed examples and API reference.

unsynn

Macro unsynn Copy item path

§Example

§Traits within the unsynn! macro (EXPERIMENTAL)

§Generic Types and Trait Bounds

§Example

§Extended Syntax Forms

§Semicolon Terminators

§Trait Blocks

§Inline Impl Blocks

§Multi-Type Impl

§Custom Parsing with parse_with

§Custom Token Emission with to_tokens

§Combining parse_with and to_tokens

§Keywords and Operators

§Parse Predicates

Macro unsynn

§Traits within the `unsynn!` macro (EXPERIMENTAL)

§Custom Parsing with `parse_with`

§Custom Token Emission with `to_tokens`

§Combining `parse_with` and `to_tokens`