The magic is +LEVEL and -LEVEL (instead of FIG-FORTH's traditional STATE), which when you start a top level loop or conditional it transitions from level 0 to 1, and switches the outer interpreter into temporary compilation mode that compiles headerless code into into a compile-buffer, then when you -LEVEL back to zero it executes the code you just compiled, without it actually being part of a permanent word.
So your temporary top level code is out of the way of HERE and can compile permanent stuff into the dictionary or whatever it needs to do. Then you can do stuff like "10 0 do i , loop" and the numbers you're ,'ing won't get mixed up with the code of the loop that's ,'ing them.
This post has a bunch of links to the OpenFirmware metacompiler's implementation and also the CForth implementation:
https://news.ycombinator.com/item?id=38689282
Here are the control structures in kernel.fth (this is some beautiful FORTH code to read for pleasure):
kernel.fth: https://github.com/MitchBradley/openfirmware/blob/master/for...
Here is the same approach in CForth, the low level C kernel code (necessarily ugly C macrology) and the higher level FORTH control structure definitions (more beautiful FORTH code):
forth.c: https://github.com/MitchBradley/cforth/blob/master/src/cfort...
control.fth: https://github.com/MitchBradley/cforth/blob/master/src/cfort...
Here is another paper about refactoring the FORTH compiler/interpreter with deferred words that Mitch wrote called "Yet Another Interpreter Organization":
https://groups.google.com/g/comp.lang.forth/c/lKQjcJL_o54/m/...
>There has been a mild controversy in the Forth community about how to
implement the text interpreter. The particular problem is how the
distinction between compiling and interpreting should be coded. At least
three distinct solutions have been advocated over the years. I propose a
fourth one, and claim that it is the best solution yet.
[describes FIG-FORTH's solution with STATE, PolyForth's solution with two separate loops for compiling and interpreting, Bob Berkey's coroutines approach]
>What is Wrong with all this
>These different schemes do not at all address what I consider to be the
fundamental problems with the interpreter/compiler.
>Fundamental Problem #1:
>The compiler/interpreter has a built-in infinite loop. This means that you
can't tell it to just compile one word; once you start it, off it goes, and
it won't stop until it gets to the end of the line or screen.
>Fundamental Problem #2:
>The reading of the next word from the input stream is buried inside this
loop. This means that you can't hand a string representing a word to the
interpreter/compiler and have it interpret or compile it for you.
>Fundamental Problem #3:
>The behavior of the interpreter/compiler is hard to change because all the
behavior is hard-wired into one or two relatively large words. Changing
this behavior can be extremely useful for a number of applications, for
example meta-compiling.
>Fundamental Problem #4:
>If the interpreter/compiler can't figure out what to do with a word (it's
not defined and it's not a number), it aborts. Worse yet, the aborting is
not done directly from within the loop, but inside NUMBER. This severly
limits the usefulness of NUMBER because if the string that NUMBER gets is
not recognizable as a number, it will abort on you. (The 83 standard punts
this issue by not specifying NUMBER, except as an uncontrolled reference
word).
[describes Mitch's solution of making DO-DEFINED, DO-LITERAL, and DO-UNDEFINED a deferred word]
>So what?
>This may seem to be more complicated than the schemes it replaces. It
certainly does have more words. On the other hand, each word is
individually easy to understand, and each word does a very specific job, in
contrast to the old style, which bundles up a lot of different things in one
big word. The more explicit factoring gives you a great deal of control
over the interpreter.
[describes cool examples of what you can do with it]
>Finally, a really neat way to write keyword-driven translators. Suppose you
have some kind of a file that contains a bunch of text. Interspersed
throughout the text are keywords that you would like to recognize, and the
program should do something special when it sees a keyword. For things that
aren't keywords, it just writes them out unchanged. Suppose that the
keywords are ".PARAGRAPH", ".SECTION", and ".END". [...]
>I have used this technique very successfully to extract specific information
from data base files produced by a CAD system. Instead of outputting
unrecognized words, I actually just ignored them in this application, but
the technique is the same in either case.
Mitch Bradley
Bradley Forthware
P.O. Box 4444
Mountain View, CA 94040
wmb@forthware.com
Mitch had the coolest P.O. Box address for his Forthware company in Mountain View!
This deferred word approach is actually what I used for the HyperTIES markup language interpreter/formatter for NeWS I wrote in Forth and C and PostScript, using Mitch's Sun Forth / Forthmacs (predecessor to OpenFirmware that ran on the Sun):
https://donhopkins.com/home/ties/doc/formatter.st0
https://donhopkins.com/home/ties/fmt.f
https://donhopkins.com/home/ties/fmt.c
https://donhopkins.com/home/ties/fmt.cps
https://donhopkins.com/home/ties/fmt.ps