yf-yang · GitHub

Bug report

Bug description:

>>> value = 123
>>> f"{value:.{'2'}f}" # '123.00'
>>> f"{value:.
{'2f'}}" # '123.00'
# but, shouldn't this one below also be valid?
>>> f"{value:.
... {'2'}f}" # invalid
>>> f"{value:
... {'.2f'}}" # '123.00'

I am checking the code of CPython's lexer and am confused why the lexer resets to normal mode when it encounters line break in format spec

// If we are in a format spec and we found a newline,
// it means that the format spec ends here and we should
// return to the regular mode.
if (in_format_spec && c == '\n') {
tok_backup(tok, c);
TOK_GET_MODE(tok)->kind = TOK_REGULAR_MODE;
current_tok->in_format_spec = 0;
p_start = tok->start;
p_end = tok->cur;
return MAKE_TOKEN(FSTRING_MIDDLE);
}

Go back to the original PR: #110271
Seems this case is not covered by the test cases

The case above could be recognized by tokenizer, so maybe pegen should be modified.
However, since it goes back to normal mode, some corner cases cannot be handled by tokenizer:

>>> value = 123
>>> f"{value:#{'x'}}" # '0x7b'
>>> f"{value:
... #{'x'}}" # breaks the tokenizer

CPython versions tested on:

3.13

Operating systems tested on:

macOS

Linked PRs

Read the original on github.com ↗