←back to thread

Using LLMs at Oxide

(rfd.shared.oxide.computer)
694 points steveklabnik | 3 comments | 07 Dec 25 01:17 UTC | HN request time: 0s | source
Show context
thundergolfer ◴[07 Dec 25 01:46 UTC] No.46178458[source]▶
A measured, comprehensive, and sensible take. Not surprising from Bryan. This was a nice line:

> it’s just embarrassing — it’s as if the writer is walking around with their intellectual fly open.

I think Oxide didn't include this in the RFD because they exclusively hire senior engineers, but in an organization that contains junior engineers I'd add something specific to help junior engineers understand how they should approach LLM use.

Bryan has 30+ years of challenging software (and now hardware) engineering experience. He memorably said that he's worked on and completed a "hard program" (an OS), which he defines as a program you doubt you can actually get working.

The way Bryan approaches an LLM is super different to how a 2025 junior engineer does so. That junior engineer possibly hasn't programmed without the tantalizing, even desperately tempting option to be assisted by an LLM.

replies(9): >>46178592 #>>46178622 #>>46178776 #>>46179419 #>>46180863 #>>46180957 #>>46180987 #>>46181685 #>>46184735 #
pests ◴[07 Dec 25 02:15 UTC] No.46178592[source]▶
> That junior engineer possibly hasn't programmed without the tantalizing, even desperately tempting option to be assisted by an LLM.

Years ago I had to spend many months building nothing but Models (as in MVC) for a huge data import / ingest the company I worked on was rewriting. It was just messy enough that it couldn't be automated. I almost lost my mind from the dull monotony and started even having attendance issues. I know today that could have been done with an LLM in minutes. Almost crazy how much time I put into that project compared to if I did it today.

replies(2): >>46179312 #>>46179750 #
aatd86 ◴[07 Dec 25 05:08 UTC] No.46179312[source]▶
The issue is that it might look good but an LLM often inserts weird mistakes. Or ellipses. Or overindex on the training data. If someone is not careful it is easy to completely wreck the codebase by piling on seemingly innocuous commits. So far I have developed a good sense for when I need to push the llm to avoid sloppy code. It is all in the details.

But a junior engineer would never find/anticipate those issues.

I am a bit concerned. Because the kind of software I am making, a llm would never prompt on its own. A junior cannot make it, it requires research and programming experience that they do not have. But I know that if I were a junior today, I would probably try to use llms as much as possible and would probably know less programming over time.

So it seems to me that we are likely to have worse software over time. Perhaps a boon for senior engineers but how do we train junior devs in that environment? Force them to build slowly, without llms? Is it aligned with business incentives?

Do we create APIs expecting the code to be generated by LLMs or written by hand? Because the impact of verbosity is not necessarily the same. LLMs don't get tired as fast as humans.

replies(3): >>46179431 #>>46179492 #>>46183982 #
1. ambicapter ◴[07 Dec 25 18:46 UTC] No.46183982[source]▶
If it's such a mind numbing problem it's easy to check it though, and the checking you do after the LLM will be much smaller than you writing every field (implicitly "checking" it when you write it).

Obviously if it's anything even minorly complex you can't trust the LLM hasn't found a new way to fool you.

replies(2): >>46185615 #>>46187044 #
2. pests ◴[07 Dec 25 21:59 UTC] No.46185615[source]▶
This is exactly it. There wasn't any complex logic. Just making sure the right fields were mapped, some renaming, and sometimes some more complex joins depending on the incoming data source and how it was represented (say multiple duplicate rows or a single field with comma delimited id's from somewhere else). I would have much rather scanned the LLM output line by line (and most would be simple, not very indented) then hand writing from scratch. I do admit it would take some time to review and cross reference, but I have no doubt it would have been a fraction of the time and effort.
3. aatd86 ◴[08 Dec 25 00:45 UTC] No.46187044[source]▶
True. The counterpoint being that back in the days, they could have decided to write a parser if the data was structured and they would have then learnt things that they will never learn by relying on AI.

For a junior in the learning phase that can be useful time spent. Then again, I agree that at times certain menial code tasks are not worth doing and llms are helpful.

It's a bit like a kid not spending time memorizing their time tables since they can use a calculator. They are less likely to become a great mathematician.