Bug report
The following example:
from email.message import EmailMessage from email.contentmanager import raw_data_manager m = EmailMessage() content = "\u65e5\u672c\u8a9e\n" raw_data_manager.set_content(m, content, charset='shift_jis') print(m)
fails with exception
Traceback (most recent call last): File "<python-input-0>", line 7, in <module> print(m) ~~~~~^^^ File "/home/serhiy/py/cpython/Lib/email/message.py", line 1015, in __str__ return self.as_string(policy=self.policy.clone(utf8=True)) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/serhiy/py/cpython/Lib/email/message.py", line 1012, in as_string return super().as_string(unixfrom, maxheaderlen, policy) ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/serhiy/py/cpython/Lib/email/message.py", line 194, in as_string g.flatten(self, unixfrom=unixfrom) ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/serhiy/py/cpython/Lib/email/generator.py", line 118, in flatten self._write(msg) ~~~~~~~~~~~^^^^^ File "/home/serhiy/py/cpython/Lib/email/generator.py", line 183, in _write self._dispatch(msg) ~~~~~~~~~~~~~~^^^^^ File "/home/serhiy/py/cpython/Lib/email/generator.py", line 220, in _dispatch meth(msg) ~~~~^^^^^ File "/home/serhiy/py/cpython/Lib/email/generator.py", line 258, in _handle_text msg.set_payload(msg._payload, charset) ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/serhiy/py/cpython/Lib/email/message.py", line 361, in set_payload self.set_charset(charset) ~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/serhiy/py/cpython/Lib/email/message.py", line 392, in set_charset self._payload = charset.body_encode(self._payload) ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ File "/home/serhiy/py/cpython/Lib/email/charset.py", line 441, in body_encode string = string.encode(self.output_charset).decode('ascii') ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ UnicodeEncodeError: 'iso2022_jp' codec can't encode character '\udc93' in position 0: illegal multibyte sequence encoding with 'iso-2022-jp' codec failed
The same for euc-jp. "shift_jis" and "euc-jp" are two charsets in email.charset.CHARSETS for which the output charset is overridden.
This fails in Python from 3.11 to main. It worked in Python 3.10 and older.
Linked PRs
- gh-150771: Fix email serialization for shift_jis and euc-jp #151120
- [3.15] gh-150771: Fix email serialization for shift_jis and euc-jp (GH-151120) #151541
- [3.14] gh-150771: Fix email serialization for shift_jis and euc-jp (GH-151120) #152952
- [3.13] gh-150771: Fix email serialization for shift_jis and euc-jp (GH-151120) #152953