←back to thread

214 points Brajeshwar | 4 comments | | HN request time: 0s | source
Show context
marcyb5st ◴[] No.45087065[source]
In terms of LOCs maybe, in terms of importance I think is much less. At least that's how I use LLMs.

While I understand that <Enter model here> might produce the meaty bits as well, I believe that having a truck factor of basically 0 (since no-one REALLY understands the code) is a recipe for a disaster and I dare say long term maintainability of a code base.

I feel that you need to have someone in any team that needs to have that level of understanding to fix non trivial issues.

However, by all means, I use the LLM to create all the scaffolding, test fixtures, ... because that is mental energy that I can use elsewhere.

replies(2): >>45087181 #>>45090543 #
epicureanideal ◴[] No.45087181[source]
Agreed. If I use an LLM to generate fairly exhaustive unit tests of a trivial function just because I can, that doesn’t mean those lines are as useful as core complex business logic that it would almost certainly make subtle mistakes in.
replies(1): >>45087230 #
andsoitis ◴[] No.45087230[source]
> If I … generate fairly exhaustive unit tests of a trivial function

… then you are not a senior software engineer

replies(1): >>45087252 #
triyambakam ◴[] No.45087252[source]
Neither are you if that's your understanding of a senior engineer
replies(2): >>45087398 #>>45087504 #
mgh95 ◴[] No.45087398[source]
I think the parent commentors point was that it is nearly trivial to generate variations on unit tests in most (if not all) unit test frameworks. For example:

Java: https://docs.parasoft.com/display/JTEST20232/Creating+a+Para...

C# (nunit, but xunit has this too): https://docs.nunit.org/articles/nunit/technical-notes/usage/...

Python: https://docs.pytest.org/en/stable/example/parametrize.html

cpp: https://google.github.io/googletest/advanced.html

A belief that the ability of LLMs to generate parameterizations is intrinsically helpful to a degree which cannot be trivially achieved in most mainstream programming languages/test frameworks may be an indicator that an individual has not achieved a substantial depth of experience.

replies(3): >>45087613 #>>45087958 #>>45091265 #
VectorLock ◴[] No.45087958[source]
Parameterized tests are good, but I think he might be talking about exercising all the corner cases in the logic of your function, which to my knowledge almost no languages can auto-generate for but LLMs can sorta-ish figure it out.
replies(1): >>45088000 #
mgh95 ◴[] No.45088000[source]
We are talking about basic computing for CRUD apps. When you start needing to rely upon "sorta-ish" to describe the efficacy or a tool for such a straightforward and deterministic use case, it may be an indicator you need to rethink your approach.
replies(1): >>45088558 #
VectorLock ◴[] No.45088558[source]
If you want to discount a tool that may save you an immense amount of time because you might have to help it along the fast few feet, thats up to you.

If you can share a tool that can analyze a function and create a test for all corner cases in a popular language, I'm sure some people would be interested in that.

replies(1): >>45088623 #
1. mgh95 ◴[] No.45088623{3}[source]
You should look up intellitest and reshaper test generator. Products exist for this.
replies(1): >>45091321 #
2. imtringued ◴[] No.45091321[source]
Maybe you should have brought that up earlier instead of acting smug and burying the lede? It's also pretty telling that you didn't elaborate this further and kept your comment short.

What you should have said is that some parameterized test generators do automated white box testing where they look at your code similar to a fuzzer and try to find the test cases automatically. Your first link is literally just setting up an array with test cases, which basically means you'd have to use an LLM to quickly produce the test cases anyway, which makes parameterized testing sound exceedingly pathetic.

https://learn.microsoft.com/en-us/visualstudio/test/generate...

>IntelliTest explores your .NET code to generate test data and a suite of unit tests. For every statement in the code, a test input is generated that will execute that statement. A case analysis is performed for every conditional branch in the code. For example, if statements, assertions, and all operations that can throw exceptions are analyzed. This analysis is used to generate test data for a parameterized unit test for each of your methods, creating unit tests with high code coverage. Think of it as smart fuzz testing that trims down the inputs and test cases to what executes all your logic branches and checks for exceptions.

replies(1): >>45094961 #
3. mgh95 ◴[] No.45094961[source]
> Maybe you should have brought that up earlier instead of acting smug and burying the lede? It's also pretty telling that you didn't elaborate this further and kept your comment short.

I thought people were generally competent within the areas they discuss and are aware of the tooling within their preferred ecosystem. I apologize if that is not the case.

replies(1): >>45098485 #
4. VectorLock ◴[] No.45098485{3}[source]
Reshaper huh...

"With AI Assistant, you can generate unit tests for C# methods."

https://www.jetbrains.com/help/resharper/Generate_tests.html