This is a really useful document - the explanations are very clear and it covers a lot of ground.
Anyone know who wrote it? It's not credited and it's pubished on a free Markdown pastebin.
The section on DRY - "repetition penalties" - was interesting to me. I often want LLMs to deliberately output exact copies of their input. When summarizing a long conversation for example I tend to ask for exact quotes that are most illustrative of the points being made. These are easy to fact check later by searching for them in the source material.
The DRY penalty seems to me that it would run counter to my goal there.
replies(1):