←back to thread

214 points Brajeshwar | 1 comments | | HN request time: 0s | source
Show context
marcyb5st ◴[] No.45087065[source]
In terms of LOCs maybe, in terms of importance I think is much less. At least that's how I use LLMs.

While I understand that <Enter model here> might produce the meaty bits as well, I believe that having a truck factor of basically 0 (since no-one REALLY understands the code) is a recipe for a disaster and I dare say long term maintainability of a code base.

I feel that you need to have someone in any team that needs to have that level of understanding to fix non trivial issues.

However, by all means, I use the LLM to create all the scaffolding, test fixtures, ... because that is mental energy that I can use elsewhere.

replies(2): >>45087181 #>>45090543 #
epicureanideal ◴[] No.45087181[source]
Agreed. If I use an LLM to generate fairly exhaustive unit tests of a trivial function just because I can, that doesn’t mean those lines are as useful as core complex business logic that it would almost certainly make subtle mistakes in.
replies(1): >>45087230 #
andsoitis ◴[] No.45087230[source]
> If I … generate fairly exhaustive unit tests of a trivial function

… then you are not a senior software engineer

replies(1): >>45087252 #
triyambakam ◴[] No.45087252[source]
Neither are you if that's your understanding of a senior engineer
replies(2): >>45087398 #>>45087504 #
mgh95 ◴[] No.45087398[source]
I think the parent commentors point was that it is nearly trivial to generate variations on unit tests in most (if not all) unit test frameworks. For example:

Java: https://docs.parasoft.com/display/JTEST20232/Creating+a+Para...

C# (nunit, but xunit has this too): https://docs.nunit.org/articles/nunit/technical-notes/usage/...

Python: https://docs.pytest.org/en/stable/example/parametrize.html

cpp: https://google.github.io/googletest/advanced.html

A belief that the ability of LLMs to generate parameterizations is intrinsically helpful to a degree which cannot be trivially achieved in most mainstream programming languages/test frameworks may be an indicator that an individual has not achieved a substantial depth of experience.

replies(3): >>45087613 #>>45087958 #>>45091265 #
com2kid ◴[] No.45087613[source]
The useful part is generating the mocks. The various auto mocking frameworks are so hit or miss I end up having to manually make mocks which is time consuming and boring. LLMs help out dramatically and save literally hours of boring error prone work.
replies(1): >>45088016 #
mgh95 ◴[] No.45088016{3}[source]
Why mock at all? Spend the time making integration tests fast. There is little reason a database, queue, etc. can't be set up in a per-test group basis and be made fast. Reliable software is built upon (mostly) reliable foundations.
replies(3): >>45088108 #>>45088169 #>>45090246 #
lanstin ◴[] No.45088108{4}[source]
hmmmm. I do like integration tests, but I often tell people the art of modern software is to make reliable systems on top of unreliable components. And the integration tests should 100% include times when the network flakes out and drops 1/2 of replies and corrupts msgs and the like.
replies(2): >>45088136 #>>45090518 #
1. mgh95 ◴[] No.45088136{5}[source]
> I do like integration tests, but I often tell people the art of modern software is to make reliable systems on top of unreliable components.

There is a dramatic difference between unreliable in the sense of S3 or other services and unreliable as in "we get different sets of logical outputs when we provide the same input to a LLM". In the first, you can prepare for what are logical outcomes -- network failures, durability loss, etc. In the latter, unless you know the total space of outputs for a LLM you cannot prepare. In the operational sense, LLMs are not a system component, they are a system builder. And a rather poor one, at that.

> And the integration tests should 100% include times when the network flakes out and drops 1/2 of replies and corrupts msgs and the like.

Yeah, it's not that hard to include that in modern testing.