bitdancer · GitHub

@bitdancer

The unknown-8bit trick was designed to deal with unknown bytes in an
ASCII message, and it works fine for that.  However, I also tried to
extend it to handle bytes that can't be decoded using the charset
specified in an encoded word, and there it fails because there can be
other non-ASCII characters that were *successfully* decoded.  The fix is
simple: do the unknown-8bit encoding using the utf-8 codec.  This is
especially appropriate since anyone trying to do recovery on an unknown
byte string will probably attempt utf-8 first.

@bitdancer

@bitdancer

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request

Dec 24, 2025
…-142517)
The unknown-8bit trick was designed to deal with unknown bytes in an
ASCII message, and it works fine for that.  However, I also tried to
extend it to handle bytes that can't be decoded using the charset
specified in an encoded word, and there it fails because there can be
other non-ASCII characters that were *successfully* decoded.  The fix is
simple: do the unknown-8bit encoding using the utf-8 codec.  This is
especially appropriate since anyone trying to do recovery on an unknown
byte string will probably attempt utf-8 first.
(cherry picked from commit  1e17ccd )
Co-authored-by: R. David Murray <rdmurray@bitdance.com>

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request

Dec 24, 2025
…-142517)
The unknown-8bit trick was designed to deal with unknown bytes in an
ASCII message, and it works fine for that.  However, I also tried to
extend it to handle bytes that can't be decoded using the charset
specified in an encoded word, and there it fails because there can be
other non-ASCII characters that were *successfully* decoded.  The fix is
simple: do the unknown-8bit encoding using the utf-8 codec.  This is
especially appropriate since anyone trying to do recovery on an unknown
byte string will probably attempt utf-8 first.
(cherry picked from commit  1e17ccd )
Co-authored-by: R. David Murray <rdmurray@bitdance.com>

bitdancer added a commit to bitdancer/cpython that referenced this pull request

Dec 24, 2025

bitdancer added a commit that referenced this pull request

Dec 24, 2025

bitdancer added a commit that referenced this pull request

Dec 24, 2025
…H-142517) (#143147)
The unknown-8bit trick was designed to deal with unknown bytes in an
ASCII message, and it works fine for that.  However, I also tried to
extend it to handle bytes that can't be decoded using the charset
specified in an encoded word, and there it fails because there can be
other non-ASCII characters that were *successfully* decoded.  The fix is
simple: do the unknown-8bit encoding using the utf-8 codec.  This is
especially appropriate since anyone trying to do recovery on an unknown
byte string will probably attempt utf-8 first.
(cherry picked from commit  1e17ccd )
Co-authored-by: R. David Murray <rdmurray@bitdance.com>
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>

bitdancer added a commit that referenced this pull request

Dec 24, 2025
…H-142517) (#143146)
The unknown-8bit trick was designed to deal with unknown bytes in an
ASCII message, and it works fine for that.  However, I also tried to
extend it to handle bytes that can't be decoded using the charset
specified in an encoded word, and there it fails because there can be
other non-ASCII characters that were *successfully* decoded.  The fix is
simple: do the unknown-8bit encoding using the utf-8 codec.  This is
especially appropriate since anyone trying to do recovery on an unknown
byte string will probably attempt utf-8 first.
(cherry picked from commit  1e17ccd )
Co-authored-by: R. David Murray <rdmurray@bitdance.com>
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>

@bitdancer

Closed

Merged

Read the original on github.com ↗