Serving 200M requests per day with a CGI-bin

I think you might have found that CGI scripts deployed as statically-linked C binaries, with some attention given to size, you might've not been so disappointed.

The "performance hit of starting a new process" is bigger if the process is a dynamically-linked php interpreter with gobs of shared libraries to load, and some source file, reading parsing compiling whatever, and not just by a little bit, always has been, so what the author is doing using go, I think, would still have been competitive 25 years ago if go had been around 25 years ago.

Opening an SQLite database is probably (surprisingly?) competitive to passing a few sockets through a context switch, across all server(ish) CPUS of this era and that, but both are much faster than opening a socket and authenticating to a remote mysql process, and programs that are not guestbook.cgi often have many more resource acquisitions which is why I think FastCGI is still pretty good for new applications today.

That's likely true - but C is a scary language to write web-facing applications in because it's so easy to have things like buffer overflows or memory leaks.

Can use rust, go, or whatever compiled language you want and it'll probably be much more performant starting up than any interpreted language.

CGI basically is a protocol between the webserver and some process. It didn't go out of fashion because of C (many/most CGI scripting for a time was perl and php or even bash), it went out of fashion because people wanted (or were told that smart people did this) to use languages that had runtimes which were expensive to run in a fork()+exec() execution model like Java.

You can use it with any language that can read stdin and write stdout. Yes, printing and reading.

We didn't have so many options in the 90s!

> Can use rust, go, or whatever compiled language you want and it'll probably be much more performant starting up than any interpreted language.

One additional bit of context is the person you’re replying to, simonw, is also the creator of Django, which at the time was the world’s defacto standard Python web framework, and was created at a time (2005) that long predates either Go (2009) or Rust (2012).

Neither of these things came up in the early days of web apps development in the mid-90s when I was doing this.

I had no real debugging environment. I was probably writing all my code in vi and then just compiling and deploying. I guarantee there were buffer overflows and off-by-ones etc.

Web app code was so simple back then, though. The most complex one I wrote was a webmail app, which I was so pleased with, and then HoTMaiL was released three weeks later, with this awesome logo:

https://tenor.com/view/hotmail-outlook-microsoft-outlookcom-...

I wrote my first web app upon realizing that cgi-bin stuff could just be .bas files I compiled in QB 4.5. I pretty quickly switched to Perl for the ecosystem of helper libs, but at the time Basic was all I knew. I think I was 10 or 11 years old.

Don't be afraid.

Look at qmail, which has the best track record of any piece of software I am aware of in wide distribution, and it was written in C.

Also: Memory leaks go away when you exit(), so they are actually more common in dynamic languages in my experience, although they manifest as fragmentation that the interpreter simply lacks the ability to do anything out.

Buffer overflows seem pretty common to people who do a lot of dynamic memory allocation: I would recommend not doing that in response to user-input.

The result is that your C-based guestbook CGI is probably written very differently than a PHP-based guestbook. Mine basically just wrote to a logfile because since 2.6.35 we have been able to easily make a 1mb PIPE_BUF and get lock-free stores with no synchronisation and trivial recovery, and thus know exactly where each post began and end. I'm not sure I want more than 1mb of user input back in those days, but the design made me very confident there were no memory leaks or buffer overflows in what was like 5 system calls. No libraries.

You could do this.

You can do this.

But you want more? That C-based guestbook also only ever needs to write to one file, so permissions could be (carefully) arranged to make that the only file it can write to. A PHP-based guestbook needs read (and possibly write-access) to lots of files. Some of those things can be shared objects. It is so much easier to secure a single static binary than a dynamic language with dynamic loading that if you actually care about security, you could focus on how to make those static binaries easier.

That didn’t bother me so much (less than it should have), but string manipulation in C is tedious, man! And there was soo much string manipulation (none of it done by a helpful framework)…

I used Perl instead, which worked way better in that regard (+ taint based security was welcome in handling untrusted user input), and an enormous (for the time) CPAN ecosystem, but had other problems.

Python web ecosystem was a mess, so PHP3 it was (ah the “good” ol days of mysql_real_escape_string()) … until some enterprising individuals wrote Django and I happily switched. Thank you :)

I made fork run a chroot and if it crashes, it doesn't really matter as it's just a fork. At that time, people weren't really generally good breaking out of a chroot using exploits.

If you use secure string functions, you can generally avoid buffer overflows in C. The problem is that not everyone does. That said, exploiting buffer overflows in programs whose source code and binaries are not public is very difficult. It likely can be done, but most people would likely go after easier targets. If you deploy AddressSanitizer in production, you can get the program to terminate whenever a buffer overflows occurs, at the expense of additional overhead when there is no buffer overflow.

Memory leaks are considered a feature in short lived programs, since not freeing memory in favor of relying on the kernel to free it at program exit lets them run faster.