←back to thread

504 points Terretta | 3 comments | | HN request time: 0s | source
Show context
cendyne ◴[] No.45064774[source]
My experience with 'sonic' during the stealth phase had it do stuff plenty fast, but the quality was slightly off target for some things. It did create tests and then iterate on those tests. The tests it wrote don't actually verify intended behavior. It only verified that mocks were called with the intended inputs while missing the larger picture of how it is used.
replies(1): >>45067443 #
1. miohtama ◴[] No.45067443[source]
Sounds like it excels at tasks like generating boilerplate.
replies(2): >>45067949 #>>45076133 #
2. bpavuk ◴[] No.45067949[source]
something GPT-4.1 and Gemini 1.5 Flash also did very well!
3. seunosewa ◴[] No.45076133[source]
I wonder how it compares the GPT 5.1 Mini and Gemini 2.5 Flash.