How I wrote a self-hosting C compiler in 40 days

Thought this was going to be an inspiration to me to continue with my pet project of writing my own little programming language. But it starts off on day 8 with him already having written a basic compiler, with no explanation of how he did any of the basics. Still interesting, just not what I thought it was.

Luckily you can see the history on Git.

Same here. I have been trying the same project as you... and get stuck on the grammar, every.single.time.

Are you done with that part yet?

I think the best inspiration to take here is that the best way to write a compiler in 40 days is to first write one in 400 days.

From the first entry:

> Implementing these features is easy because this is the second time for me.

What do you mean "stuck on the grammar"? Can't figure out how to write a grammar, can't get all the fiddly implications correct so OOO fails, can't figure out how to write the parser, can't muster up enough enthusiasm to finish the work...?

One option is to punt for a bit and use what I sometimes call the "Lisp non-grammar", even if you don't intend to be writing a Lisp; use parentheses to directly encode a parse tree, and whatever other hacky symbols you need for now to set up atom types or whatever. You might still be able to explore enough what you're interested in to figure out what you want in the grammar. Whatever the core idea of your toy language is, you should try to get to that as quickly as possible, punting on everything else you possibly can, so if the point of your language isn't to have an innovative new grammar, you might want to try to defer that work.

On which parts exactly are you stuck? I've rolled with Ragel as lexer, and Lemon as parser, and it was relatively simple.

[1] https://github.com/castel/libcastel/blob/master/parse/source...

[2] https://github.com/castel/libcastel/blob/master/parse/source...

I can't recommend the Coursera compilers course highly enough. It was enough to get me from zero to a fully working compiler. Albeit for a simplified language, and a really bad compiler :-)

If you did it in one of the languages for which scaffolding already exists, it'll be even easier (I did it in a different language, so had to build everything from scratch).

And thus, the 10X programmer.

It's fine. "Compilers" topic is too wide. There are lot of sub-topics. I personally find lexing/parsing rather boring. It's after I have AST is when the fun begins for me (and right before actual/real code gen, where you have to know intricacies of particular CPU) :)

"If you want to be a Millionaire, start with a billion dollars and launch a new airline ." -- Richard Branson

The C grammar is printed in BNF in the back of K&R and available in machine-readable form in the standards.

This is quite well said. It never occurred to me that 10x programmer is just the programmer who has already written code that grappled with the ideas that matter most. By your fifth compiler, all the basics seem easy.

I haven't finished any part of it yet. My goal is to write a small language entirely in C. It would have a lexer, parser, bytecode compiler, stack based bytecode VM, and GC. It would only have numbers, strings, and functions to start with. That would be my "hello world" that I can build on to add new features. And since it would all be written in C, it could be easily embeddable. So my goal is to make it similar to Lua, except with a nicer API for embedding. I've got some vacation days coming up and hope to make some progress on this during that time.

Let me guess, you're stuck fighting yacc and shift-reduce conflicts?

I find writing your own parser to be much more instructive and a better way to understand why parser generators work the way they do.

Alternatively you can use a PEG generator such as peg/leg[1] and your grammar will more or less work (until it resolves an ambiguity in some way you didn't expect, anyway).

1: http://piumarta.com/software/peg/