ncoghlan · GitHub

added 10 commits

March 5, 2017 16:58
- new PYTHONCOERCECLOCALE config setting
- coerces legacy C locale to C.UTF-8, C.utf8 or UTF-8 by default
TODO:
- configure option to disable locale coercion at build time
- configure option to disable C locale warning at build time
- skip runtime locale warning on Mac OS X
* --with(out)-c-locale-coercion for PY_COERCE_C_LOCALE
* --with(out)-c-locale-warning for PY_WARN_ON_C_LOCALE

@ncoghlan

@ncoghlan

methane

warsaw

- setting PYTHONIOENCODING has unintended side effects on Python 2
  instances run in a subprocess (since Python 2 has no
  `surrogateescape` error handler
- Py_SetStandardStreamEncoding enables surrogateescape for the
  current process without any side effects on subprocesses
Windows doesn't use setenv to set environment variables,
so set PYTHONIOENCODING from test_capi instead of
_testembed when running the forced_io_encoding test.
- move all required logic inside the shared library
- explicitly setting one of the coercion target locales
  now also automatically enables "surrogateescape" on
  sys.stdin and sys.stdout
Locale coercion no longer has any effect if LC_ALL is
explicitly set in the environment.
When locale coercion triggers, it sets either both
LC_CTYPE & LANG (for full locales) or only LC_CTYPE
(for partial locales).
This change also eliminated the need for a custom
test case for the locale coercion warning - instead,
the test suite is able to check for that just by
setting LC_ALL in the child process environment.

@ncoghlan ncoghlan changed the title bpo-28180: Reference implementation for PEP 538 bpo-28180: Implementation for PEP 538

May 28, 2017

methane

ncoghlan

ncoghlan

- avoid unintended side effects on Windows behaviour
- remove a single-use function that made the code harder to follow
- clarify the security considerations around ignoring -E and -I
  when checking PYTHONCOERCECLOCALE
The inline check for "Is this env var exactly zero?" is still more
self-explanatory than factoring out the helper function.
A HAVE_SETLOCALE guard was removed when adding a check for __ANDROID__,
and that may be affecting the default locale reported on Windows.

@ncoghlan

@ncoghlan

ronaldoussoren

@ncoghlan

ncoghlan deleted the pep538-coerce-c-locale branch

March 30, 2018 07:47

Closed

Read the original on github.com ↗