←back to thread

593 points Gricha | 1 comments | | HN request time: 0.193s | source
Show context
xnorswap ◴[] No.46233056[source]
Claude is really good at specific analysis, but really terrible at open-ended problems.

"Hey claude, I get this error message: <X>", and it'll often find the root cause quicker than I could.

"Hey claude, anything I could do to improve Y?", and it'll struggle beyond the basics that a linter might suggest.

It suggested enthusiastically a library for <work domain> and it was all "Recommended" about it, but when I pointed out that the library had been considered and rejected because <issue>, it understood and wrote up why that library suffered from that issue and why it was therefore unsuitable.

There's a significant blind-spot in current LLMs related to blue-sky thinking and creative problem solving. It can do structured problems very well, and it can transform unstructured data very well, but it can't deal with unstructured problems very well.

That may well change, so I don't want to embed that thought too deeply into my own priors, because the LLM space seems to evolve rapidly. I wouldn't want to find myself blind to the progress because I write it off from a class of problems.

But right now, the best way to help an LLM is have a deep understanding of the problem domain yourself, and just leverage it to do the grunt-work that you'd find boring.

replies(23): >>46233156 #>>46233163 #>>46233206 #>>46233362 #>>46233365 #>>46233406 #>>46233506 #>>46233529 #>>46233686 #>>46233981 #>>46234313 #>>46234696 #>>46234916 #>>46235210 #>>46235385 #>>46236239 #>>46236306 #>>46236829 #>>46238500 #>>46238819 #>>46240191 #>>46243246 #>>46243719 #
pdntspa ◴[] No.46233365[source]
That's why you treat it like a junior dev. You do the fun stuff of supervising the product, overseeing design and implementation, breaking up the work, and reviewing the outputs. It does the boring stuff of actually writing the code.

I am phenomenally productive this way, I am happier at my job, and its quality of work is extremely high as long as I occasionally have it stop and self-review it's progress against the style principles articulated in its AGENTS.md file. (As it tends to forget a lot of rules like DRY)

replies(13): >>46233446 #>>46233448 #>>46233642 #>>46233652 #>>46233782 #>>46234010 #>>46234898 #>>46235480 #>>46238997 #>>46241434 #>>46241981 #>>46242791 #>>46244543 #
n4r9 ◴[] No.46233446[source]
I think we have different opinions on what's fun and what's boring!
replies(4): >>46233963 #>>46234282 #>>46234342 #>>46234495 #
Nemi ◴[] No.46234495[source]
You've really hit the crux of the problem and why so many people have differing opinions about AI coding. I also find coding more fun with AI. The reason is that my main goal is to solve a problem, or someone else's problem, in a way that is satisfying. I don't much care about the code itself anymore. I care about the thing that it does when it's done.

Having said that I used to be deep into coding and back then I am quite sure that I would hate AI coding for me. I think for me it comes down to – when I was learning about coding and stretching my personal knowledge in the area, the coding part was the fun part because I was learning. Now that I am past that part I really just want to solve problems, and coding is the means to that end. AI is now freeing because where I would have been reluctant to start a project, I am more likely to give it a go.

I think it is similar to when I used to play games a lot. When I would play a game where you would discover new items regularly, I would go at it hard and heavy up until the point where I determined there was either no new items to be found or it was just "more of the same". When I got to that point it was like a switch would flip and I would lose interest in the game almost immediately.

replies(8): >>46234628 #>>46235992 #>>46236149 #>>46236930 #>>46237085 #>>46239535 #>>46242499 #>>46244116 #
altmanaltman ◴[] No.46239535[source]
A few counterpoints:

1. If you don't care about code and only care about the "thing that it does when it's done", how do you solve problems in a way that is satisfying? Because you are not really solving any problem but just using the AI to do it. Is prompting more satisfying than actually solving?

2. You claim you're done "learning about coding and stretching my personal knowledge in the area" but don't you think that's super dangerous? Like how can you just be done with learning when tech is constantly changing and new things come up everyday. In that sense, don't you think AI use is actually making you learn less and you're just justifying it with the whole "I love solving problems, not code" thing?

3. If you don't care about the code, do the people who hire you for it do? And if they do, then how can you claim you don't care about the code when you'll have to go through a review process and at least check the code meaning you have to care about the code itself, right?

replies(3): >>46239867 #>>46239996 #>>46240026 #
keeda ◴[] No.46239996[source]
Note I'm not saying one is better than the other, but my takes:

1. The problem solving is in figuring out what to prompt, which includes correctly defining the problem, identifying a potential solution, designing an architecture, decomposing it into smaller tasks, and so on.

Giving it a generic prompt like "build a fitness tracker" will result in a fully working product but it will be bland as it would be the average of everything in its training data, and won't provide any new value. Instead, you probably want to build something that nobody else has, because that's where the value is. This will require you to get pretty deep into the problem domain, even if the code itself is abstracted away from you.

Personally, once the shape of the solution and the code is crystallized in my head typing it out is a chore. I'd rather get it out ASAP, get the dopamine hit from seeing it work, and move on to the next task. These days I spend most of my time exploring the problem domain rather than writing code.

2. Learning still exists but at a different level; in fact it will be the only thing we will eventually be doing. E.g. I'm doing stuff today that I had negligible prior background in when I began. Without AI, I would probably require an advanced course to just get upto speed. But now I'm learning by doing while solving new problems, which is a brand new way of learning! Only I'm learning the problem domain rather than the intricacies of code.

3. Statistically speaking, the people who hire us don't really care about the code, they just want business results. (See: the difficulty of funding tech debt cleanup projects!)

Personally, I still care about the code and review everything, whether written by me or the AI. But I can see how even that is rapidly becoming optional.

I will say this: AI is rapidly revolutionizing our field and we need to adapt just as quickly.

replies(2): >>46240289 #>>46240376 #
altmanaltman ◴[] No.46240289[source]
Honestly, I fundamentally disagree with this. Figuring out "what to prompt" is not problem-solving in a true sense imo. And if you're really going too deep into the problem domain, what is the point of having the code abstracted?

My comment was based on you saying you don't care about the code and only what it does. But now you're saying you care about the code and review everything so I'm not sure what to make out of it. And again, I fundamentally disagree that reviewing code will become optional or rather should become optional. But that's my personal take.

replies(2): >>46240624 #>>46241825 #
keeda ◴[] No.46240624[source]
> My comment was based on you saying you don't care about the code and only what it does. But now you're saying you care about the code and review everything so I'm not sure what to make out of it.

I'm not the person you originally replied to, so my take is different, which explains your confusion :-)

However I do increasingly get the niggling sense I'm reviewing code out of habit rather than any specific benefit because I so rarely find something to change...

> And if you're really going too deep into the problem domain, what is the point of having the code abstracted?

Let's take my current work as an example: I'm doing stuff with computer vision (good old-fashioned OpenCV, because ML would be overkill for my case.) So the problem domain is now images and perception and retrieval, which is what I am learning and exploring. The actual code itself does not matter as much the high-level approach and the component algorithms and data structures -- none of which are individually novel BTW, but I believe I'm the only one combining them this way.

As an example, I give a high-level prompt like "Write a method that accepts a list of bounding boxes, find all overlapping ones, choose the ones with substantial overlap and consolidate them into a single box, and return all consolidated boxes. Write tests for this method." The AI runs off and generates dozens of lines of code -- including a tunable parameter to control "substantial overlap", set to a reasonable default -- the tests pass, and when I plug in the method, 99.9% of the times the code works as expected. And because this is vision-based I can immediately verify by sight if the approach works!

To me, the valuable part was coming up with that whole approach based on bounding boxes, which led to that prompt. The actual code in itself is not interesting because it is not a difficult problem, just a cumbersome one to handcode.

To solve the overall problem I have to combine a large number of such sub-problems, so the leverage that AI gives me is enormous.

replies(1): >>46243641 #
1. skydhash ◴[] No.46243641[source]
What people are wary of is not solving the problem in the first pass. They are wary of technical debt and unmaintainable code. The cost of change can be enormous. Software engineering is mostly about solving current problems and laying the foundation to adapt for future ones at the same time. Your approach's only focus is current problems which is pretty much the same as people that copypaste from StackOverflow without understanding.