←back to thread

PEP 686 – Make UTF-8 mode default

(peps.python.org)
238 points GalaxySnail | 5 comments | 26 Apr 24 11:55 UTC | HN request time: 0.646s | source
Show context
Affric ◴[26 Apr 24 13:10 UTC] No.40169026[source]▶
Make UTF-8 default on Windows
replies(6): >>40169144 #>>40169474 #>>40170343 #>>40171178 #>>40173598 #>>40175253 #
pjc50 ◴[26 Apr 24 15:07 UTC] No.40170343[source]▶
In addition to ApiFunctionA and ApiFunctionW, introduce ApiFunction8? (times whole API surface)

Introduce a #define UNICODE_NO_REALLY_ALL_UNICODE_WE_MEAN_IT_THIS_TIME ?

replies(1): >>40170814 #
1. cryptonector ◴[26 Apr 24 15:40 UTC] No.40170814[source]▶
ApiFunctionA is UTF-8 capable. Needs a run-time switch too, not just compile-time.
replies(3): >>40170987 #>>40173012 #>>40174417 #
2. garaetjjte ◴[26 Apr 24 15:52 UTC] No.40170987[source]▶
It's now possible, but for years the excuse was that MBCS encodings only supported characters up to 2 bytes.
3. ComputerGuru ◴[26 Apr 24 19:22 UTC] No.40173012[source]▶
Only under windows 11, I believe. And that switch is off by default.
replies(1): >>40174609 #
4. sebazzz ◴[26 Apr 24 21:17 UTC] No.40174417[source]▶
Yes: https://learn.microsoft.com/en-us/windows/win32/sbscs/applic...

> On Windows 10, this element forces a process to use UTF-8 as the process code page. For more information, see Use the UTF-8 code page. On Windows 10, the only valid value for activeCodePage is UTF-8.

> This element was first added in Windows 10 version 1903 (May 2019 Update). You can declare this property and target/run on earlier Windows builds, but you must handle legacy code page detection and conversion as usual. This element has no attributes.

5. int_19h ◴[26 Apr 24 21:38 UTC] No.40174609[source]▶
You're thinking of the global setting that is enabled by the user and applies to all apps that operate in terms of "current code page" - if enabled, that codepage becomes 65001 (UTF-8).

However, on Win10+, apps themselves can explicitly opt into UTF-8 for all non-widechar Win32 APIs regardless of the current locale/codepage.