It's a vibe-coded project and it definitely feels like a vibe-coded fun hack idea (which there's nothing wrong with having a little nerdfun!).
Some more critical thoughts, in case you are motivated to push this forward:
Biggest critical point: I think there may also be a misunderstanding of llms.txt vs. llms-full.txt proposals, and a conflation of the former with the latter. A "minified" version is likely redundant since the intention of llms.txt (as opposed to llms-full.txt) is:
> `llms.txt` is an index file containing links with brief descriptions of the content. An LLM or agent must follow these links to access detailed information.
> `llms-full.txt` includes all the detailed content directly in a single file, eliminating the need for additional navigation.
> A key consideration when using `llms-full.txt` is its size. For extensive documentation, this file may become too large to fit into an LLM's context window. (https://langchain-ai.github.io/langgraph/llms-txt-overview)
There's also lots of folks working together trying to figure this stuff out. Reach out to them and join their communities!
And if you're up for some more feedback:
- Curated knowledge < stochastic summarization -- One-shot minification for core domain knowledge of a product seems likely less effective and inherently risky compared to a domain expert human "minifying" it (and most professional/technical writing already does a kind of minification via progressive disclosure with introductions, quick start guides, navigation, etc.). If you want to convince people your way is the way, write evals that show it's better than a human expert at achieving agentic outcomes more efficiently and effectively (more reliably leads to successful desired outcomes).
- You introduce a new standard (which feels very vibe-coded...), and it's very specific, rigidly structured, verbose, nested, introduces additional semantic nodes where none existed before (e.g. instead of using the actual names of things, it creates new names and nests the old underneath), and it's generally complex and nested. This could lead to fairly long compute run-times and misses for a back-and-forth trying to understand the larger semantic surface area.
- confusing file types -- .txt AND .md files? Isn't llms.txt already markdown? And in your new llms-min.txt file, the text is actually more like code (highly structured data) and its own DSL.
- unintended (?) prompt injections - e.g., "You are an expert AI Software Engineer." in your examples (https://github.com/marv1nnnnn/llm-min.txt/blob/main/sample/c...)
- Your token reduction just counts the tokens, it's not actually informative about token (and compute) costs for an LLM actually try to use the files. If an LLM is going to have to reason extensively about the file to grok it, token count of the static file is largely meaningless.
- Production-ready agents that would need to have a shorter version of texts are likely already equipped with chunking, text search, semantic grouping, and other tools that they can call when encountering a large corpus without flooding their context window with tokens. These will typically be setup by an agent developer to deliver maximally efficient & effective tokens for their task and capabilities.
Anyway, none of the above was vibe-commented. But since you are vibe-coding this, maybe throw it all at your RooCode agent to develop a plan to address this critical feedback? :D Happy vibing!