Why HN was slow and how Rtm fixed it

1 thread per connection??? Not doing continual GC in a separate thread and instead taking 7 seconds and blocking everything?

What is this the 1990s?

Feel free to fork MzScheme and replace the garbage collector with a new one that runs continuously.

There is no such thing as a free lunch

Surely there's a way to target the JVM?

I'd trust my life on the JVM, it's pretty battle tested, and the GC is simply awesome.

Do you ever question the choice of MzScheme?

> I'd trust my life on the JVM

You sure you want to do that? :-)

> Java technology is not fault tolerant and is not designed, manufactured, or intended for use or resale as on-line control equipment in hazardous environments requiring fail-safe performance, such as in the operation of nuclear facilities, aircraft navigation or communication systems, air traffic control, direct life support machines, or weapons systems, in which the failure of Java technology could lead directly to death, personal injury, or severe physical or environmental damage

http://technet.microsoft.com/en-us/library/cc976720.aspx

It had to be some dialect of Lisp with continuations, which meant Scheme, and MzScheme seemed the best. Our first version ran on Scheme48, but that was such a pain that we switched.

Yeah I'd trust my life on it. I've had java processes running for several months without issue. The hardware fails before the jvm does.

I wouldn't trust anything from microsoft though.

I love this response. Would be really interested in a follow-up from axod (or by pg about axod's version).

It's easy to criticize, but let's see what happens when the pedal hits the metal.

The JVM really is great and the garbage collecting outstanding. Anyone who has written a real high performance app can tell you that.

Let me add that one "thread" per connection works in Erlang for up to 80k connections easily. And since each "thread" is actually a process and each with their own GC, long pause-times are never a problem.

It is not the year, but the naivety that is the problem here, if any.

My response TBH, would be that there is no point spending months tweaking the engine of a ford to try and get it to perform like a porche. Just get a porche in the first place.

I don't value "being able to write it in my favorite language" at all. From what I've read, pg does. To the extent that the product suffers.

There would be absolutely no point me trying to improve mzScheme when you can do exactly the same job in other languages/platforms, and the user doesn't care/know the difference. HN could be rewritten in a weekend, in PHP/python/whatever and we wouldn't be sitting here waiting for pages to load.

(I run Mibbit, which handles a few thousand HTTP requests a second, on VPS level hardware. In Java).

Neither Python nor PHP run garbage collection continuously in a separate thread. Python is reference counted, and as far as I know PHP is too.

But I'd bet both of them perform far better than MzScheme.

In any event, it's a "solved problem". The sheer amount of time and effort going into this is silly. Just pick a better language/platform and use it.

I guess the point is, axod, if you aren't willing to backup talk with action, it might be best to think about what you say before you say it.

Don't slam something saying I could do better, and then have your bluff called.

Makes you look silly. This applies to even if the person calling your bluff was NOT pg.

It depends on what you mean by "perform better". Any reference-counted GC mechanism will have memory leaks, which is a serious problem in a long-running process. Python gets around this by also occasionally running a traditional GC, which, btw, is a stop-the-world GC.

I've helped build rail control systems that run on Java, so you're probably trusting your life to Java daily without knowing it.

Of course you can write a link aggregator in a weekend, but you can't write HN in a weekend - there's a lot of complexity in the HN source around controlling voting rings, spam, etc. When you're dealing with complex issues, it's a net win to use a language that enables you to think at the highest level of abstraction possible.