←back to thread

PEP 686 – Make UTF-8 mode default

(peps.python.org)
238 points GalaxySnail | 1 comments | 26 Apr 24 11:55 UTC | HN request time: 0s | source
Show context
Macha ◴[26 Apr 24 13:02 UTC] No.40168944[source]▶
> And many other popular programming languages, including Node.js, Go, Rust, and Java uses UTF-8 by default.

Oh, I missed Java moving from UTF-16 to UTF-8.

replies(3): >>40169012 #>>40169499 #>>40169530 #
rootext ◴[26 Apr 24 14:00 UTC] No.40169499[source]▶
It seems you are mixing two things: inner string representation and read/write encoding. Java has never used UTF-16 as default for the second.
replies(2): >>40170808 #>>40176650 #
cryptonector ◴[26 Apr 24 15:39 UTC] No.40170808[source]▶
Not even on Windows?
replies(1): >>40171134 #
layer8 ◴[26 Apr 24 16:16 UTC] No.40171134[source]▶
No, file I/O on Windows in general doesn’t use UTF-16, but the regional code page, or nowadays UTF-8 if the application decides so.
replies(1): >>40174570 #
int_19h ◴[26 Apr 24 21:34 UTC] No.40174570[source]▶
Depends on what you define as "file I/O", though. NTFS filenames are UTF-16 (or rather UCS2). As far as file contents, there isn't really a standard, but FWIW for a long time most Windows apps - Notepad being the canonical example when asked to save anything as "Unicode" would save it as UTF-16.
replies(1): >>40174958 #
layer8 ◴[26 Apr 24 22:21 UTC] No.40174958{3}[source]▶
I'm talking about the default behavior of Microsoft's C runtime (MSVCRT.DLL) that everyone is/was using.

UTF-16 text files are rather rare, as is using Notepad's UTF-16 options. The only semi-common use I know of is *.reg files saved from regedit. One issue with UTF-16 is that it has two different serializations (BE and LE), and hence generally requires a BOM to disambiguate.

replies(2): >>40176676 #>>40178858 #
1. TheCycoONE ◴[27 Apr 24 02:43 UTC] No.40176676{4}[source]▶
Powershell use to output utf-16 by default on Windows. It might still but it's been awhile since I needed to try.