serhiy-storchaka · GitHub

Bug report

The following example:

from email.message import EmailMessage
from email.contentmanager import raw_data_manager
m = EmailMessage()
content = "\u65e5\u672c\u8a9e\n"
raw_data_manager.set_content(m, content, charset='shift_jis')
print(m)

fails with exception

Traceback (most recent call last):
  File "<python-input-0>", line 7, in <module>
    print(m)
    ~~~~~^^^
  File "/home/serhiy/py/cpython/Lib/email/message.py", line 1015, in __str__
    return self.as_string(policy=self.policy.clone(utf8=True))
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/serhiy/py/cpython/Lib/email/message.py", line 1012, in as_string
    return super().as_string(unixfrom, maxheaderlen, policy)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/serhiy/py/cpython/Lib/email/message.py", line 194, in as_string
    g.flatten(self, unixfrom=unixfrom)
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/serhiy/py/cpython/Lib/email/generator.py", line 118, in flatten
    self._write(msg)
    ~~~~~~~~~~~^^^^^
  File "/home/serhiy/py/cpython/Lib/email/generator.py", line 183, in _write
    self._dispatch(msg)
    ~~~~~~~~~~~~~~^^^^^
  File "/home/serhiy/py/cpython/Lib/email/generator.py", line 220, in _dispatch
    meth(msg)
    ~~~~^^^^^
  File "/home/serhiy/py/cpython/Lib/email/generator.py", line 258, in _handle_text
    msg.set_payload(msg._payload, charset)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/serhiy/py/cpython/Lib/email/message.py", line 361, in set_payload
    self.set_charset(charset)
    ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/serhiy/py/cpython/Lib/email/message.py", line 392, in set_charset
    self._payload = charset.body_encode(self._payload)
                    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/home/serhiy/py/cpython/Lib/email/charset.py", line 441, in body_encode
    string = string.encode(self.output_charset).decode('ascii')
             ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'iso2022_jp' codec can't encode character '\udc93' in position 0: illegal multibyte sequence
encoding with 'iso-2022-jp' codec failed

The same for euc-jp. "shift_jis" and "euc-jp" are two charsets in email.charset.CHARSETS for which the output charset is overridden.

This fails in Python from 3.11 to main. It worked in Python 3.10 and older.

Linked PRs

Read the original on github.com ↗