Our goal was to build a tool that allowed us to test a range of "personal contexts" on a very focused everyday use case for us, reading HN!
We are exploring use of personal context with LLMs, specifically what kind of data, how much, and with how much additional effort on the user’s part was needed to get decent results. The test tool was a bit of fun on its own so we re-skinned it and decided to post it here.
First time posting anything on HN but folks at work encouraged me to drop a link. Keen on feedback or other interesting projects thinking about bootstrapping personal context for LLM workflows!
Other than quality of life stuff (multiple pages for example), I'd like to see it continually learn.
A few things got miscategorized and I'd love for it to naturally correct that with additional input from me.
The idea of having some kind of thumbs up/down on what you see after getting recs, that gets added to your preferences, or being able to do another round of preferences (rather than just re-doing them like we have now) is for sure on our next steps if we continue with. Were not quite sure what the feedback loops will be yet (we did look at adding whole webhistory for example but that felt like a bit much and pretty invasive).
For the miscats, on a meta level what we are generally interested in is whether they come from compression of the preferences into your user profile (essentially if more or better data is the path to better context for such a specific usecase, or whether there is more bang for buck optimizing the various prompts. Keen to hear if its obvious from looking at your profile what was the case.
If we get serious with this evals are a must next step. We are only 2 days in at the moment :)