←back to thread

Wild – A fast linker for Linux

(github.com)
383 points hkalbasi | 8 comments | 24 Jan 25 16:25 UTC | HN request time: 1.038s | source | bottom
Show context
pzmarzly ◴[24 Jan 25 16:55 UTC] No.42815005[source]▶
Ever since mold relicensed from AGPL to MIT (as part of mold 2.0 release), the worldwide need for making another fast linker has been greatly reduced, so I wasn't expecting a project like this to appear. And definitely wasn't expecting it to already be 2x faster than mold in some cases. Will keep an eye on this project to see how it evolves, best of luck to the author.
replies(5): >>42815102 #>>42815606 #>>42816517 #>>42819089 #>>42819826 #
estebank ◴[24 Jan 25 17:57 UTC] No.42815606[source]▶
Note that Mold has no interest in becoming incremental, so there is a big reason there for another linker to exist. I find it kind of embarrassing that MS' linker has been incremental by default for decades, yet there's no production ready incremental linker on Linux yet.
replies(4): >>42816198 #>>42817110 #>>42817268 #>>42818469 #
jcelerier ◴[25 Jan 25 01:02 UTC] No.42818469[source]▶
OTOH even lld, fast but fairly slower than mold, is already incredibly faster than MS's linker even without the incrmeentality. Like, I'm routinely linking hundreds of megabytes in less than a second anyways, not sure incrementality is that much worth it
replies(1): >>42818743 #
1. cyco130 ◴[25 Jan 25 02:03 UTC] No.42818743[source]▶
Not a rhetorical question: Could it be that part of the speed difference is due to the file system speed? I was shocked when I saw how much modern(ish) Windows file systems were slower than modern(ish) Linux ones.
replies(2): >>42819213 #>>42822889 #
2. leeoniya ◴[25 Jan 25 03:29 UTC] No.42819213[source]▶
it's usually windows defender and virus scanning that causes these massive slowdowns.
replies(2): >>42819849 #>>42820396 #
3. berti ◴[25 Jan 25 06:08 UTC] No.42819849[source]▶
That’s exactly the problem “Dev Drive” is intended to solve I believe. I haven’t tried it myself.

https://learn.microsoft.com/en-us/windows/dev-drive/

replies(1): >>42821567 #
4. miki123211 ◴[25 Jan 25 08:40 UTC] No.42820396[source]▶
And as a result of this, if you want to optimize your program to be fast on Windows, you will make very different optimization decisions than on other platforms.

For example, Windows runs virus scans on close(), which makes it very slow. This means that sometimes it makes sense to have one or more background threads exclusively dedicated to closing files.

There's a good talk on this at https://www.youtube.com/watch?v=qbKGw8MQ0i8

replies(1): >>42821414 #
5. leeoniya ◴[25 Jan 25 13:06 UTC] No.42821414{3}[source]▶
might have been a problem in Bun that was fixed by: https://github.com/oven-sh/bun/pull/16747
6. p_ing ◴[25 Jan 25 13:39 UTC] No.42821567{3}[source]▶
It helps. Install your games on it, too.
7. p_ing ◴[25 Jan 25 17:08 UTC] No.42822889[source]▶
NTFS and ReFS are high performance file systems. But yes, it is due to file system filters, which OOTB means Windows Defender, though that can be extended by 3rd parties, including Sysinternals utils such as procmon.

Microsoft built a very extensible I/O stack and prior to Defender/prior to SSDs it really wasn't very noticable... back when it was originally designed well through the 90s and early 00s.

Unfortunately it is now noticable despite being an otherwise smart design. Which means Windows and/or NTFS are blamed as being slow, neither of which has any basis in fact when we look at the overall design of Windows' subsystems/VMM in comparison to macOS/Linux.

It sucks. You've got great plumbing in Windows with a shit shell on top.

replies(1): >>42825740 #
8. rerdavies ◴[25 Jan 25 22:51 UTC] No.42825740[source]▶
Disabling Defender on directories involved in compiling and linking speeds up compile and link times by a factor of 5x or 6x. It's not a subtle difference.

I agonize over the insanity of removing Defender protection on precisely the files that are most vulnerable on my computer each time I do it. But I do it anyway.

Interestingly, Android Studio offers to turn off Defender protection for development directories for you, and does so each time you load a project, if you haven't already done so. So I no longer feel like I'm alone in my insanity.