I'm also working on learning about building software with LLMs, specifically I am building a small personal project that will allow me to experiment with them using measurable hypotheses and theories, rather than just tweaking a prompt a bunch and guessing when it is working the best. I know others have done this, but I am building it from the ground up because I'm using it as a learning experience.
I plan to take my experimentation platform and build a small "personal agent" software package to run on my own computer, again building from scratch for my own learning process, that will do small things for me like researching something and writing a report. I don't expect anything too useful to come out of it, since I am using 1.7B/4B models on a MacBook Air M2 (later I might use my 3080 but that won't be much improvement), but it will be interesting to build the architectural stuff even if the agents are effectively just useless cycle-wasters.