←back to thread

576 points Gricha | 1 comments | | HN request time: 0s | source
Show context
xnorswap ◴[] No.46233056[source]
Claude is really good at specific analysis, but really terrible at open-ended problems.

"Hey claude, I get this error message: <X>", and it'll often find the root cause quicker than I could.

"Hey claude, anything I could do to improve Y?", and it'll struggle beyond the basics that a linter might suggest.

It suggested enthusiastically a library for <work domain> and it was all "Recommended" about it, but when I pointed out that the library had been considered and rejected because <issue>, it understood and wrote up why that library suffered from that issue and why it was therefore unsuitable.

There's a significant blind-spot in current LLMs related to blue-sky thinking and creative problem solving. It can do structured problems very well, and it can transform unstructured data very well, but it can't deal with unstructured problems very well.

That may well change, so I don't want to embed that thought too deeply into my own priors, because the LLM space seems to evolve rapidly. I wouldn't want to find myself blind to the progress because I write it off from a class of problems.

But right now, the best way to help an LLM is have a deep understanding of the problem domain yourself, and just leverage it to do the grunt-work that you'd find boring.

replies(23): >>46233156 #>>46233163 #>>46233206 #>>46233362 #>>46233365 #>>46233406 #>>46233506 #>>46233529 #>>46233686 #>>46233981 #>>46234313 #>>46234696 #>>46234916 #>>46235210 #>>46235385 #>>46236239 #>>46236306 #>>46236829 #>>46238500 #>>46238819 #>>46240191 #>>46243246 #>>46243719 #
james_marks ◴[] No.46233206[source]
This is a key part of the AI love/hate flame war.

Very easy to write it off when it spins out on the open-ended problems, without seeing just how effective it can be once you zoom in.

Of course, zooming in that far gives back some of the promised gains.

Edit: typo

replies(2): >>46233357 #>>46233380 #
thewebguyd ◴[] No.46233357[source]
> without seeing just how effective it can be once you zoom in.

The love/hate flame war continues because the LLM companies aren't selling you on this. The hype is all about "this tech will enable non-experts to do things they couldn't do before" not "this tech will help already existing experts with their specific niche," hence the disconnect between the sales hype and reality.

If OpenAI, Anthropic, Google, etc. were all honest and tempered their own hype and misleading marketing, I doubt there would even be a flame war. The marketing hype is "this will replace employees" without the required fine print of "this tool still needs to be operated by an expert in the field and not your average non technical manager."

replies(1): >>46233533 #
hombre_fatal ◴[] No.46233533[source]
The amount of GUIs I've vibe-coded works against your claim.

As we speak, my macOS menubar has an iStat Menus replacement, a Wispr Flow replacement (global hotkey for speech-to-text), and a logs visualizer for the `blocky` dns filtering program -- all of which I built without reading code aside from where I was curious.

It was so vibe-coded that there was no reason to use SwiftUI nor set them up in Xcode -- just AppKit Swift files compiled into macOS apps when I nix rebuild.

The only effort it required was the energy to QA the LLM's progress and tell it where to improve, maybe click and drag a screenshot into claude code chat if I'm feeling excessive.

Where do my 20 years of software dev experience fit into this except beyond imparting my aesthetic preferences?

In fact, insisting that you write code yourself is becoming a liability in an interesting way: you're going to make trade-offs for DX that the LLM doesn't have to make, like when you use Python or Electron when the LLM can bypass those abstractions that only exist for human brains.

replies(2): >>46233670 #>>46233772 #
onethought ◴[] No.46233670[source]
Love that you are disagreeing with parent by saying you built software all on your own, and you only had 20 years software experience.

Isn't that the point they are making?

replies(1): >>46233681 #
hombre_fatal ◴[] No.46233681[source]
Maybe I didn't make it clear, but I didn't build the software in my comment. A clanker did.

Vibe-coding is a claude code <-> QA loop on the end result that anyone can do (the non-experts in his claim).

An example of a cycle looks like "now add an Options tab that let's me customize the global hotkey" where I'm only an end-user.

Once again, where do my 20 years of software experience come up in a process where I don't even read code?

replies(2): >>46233729 #>>46234303 #
onethought ◴[] No.46233729[source]
But anyone didn't do it... you an expert in software development did it.

I would hazard a guess that your knowledge lead to better prompts, better approach... heck even understanding how to build a status bar menu on Mac OS is slightly expert knowledge.

You are illustrating the GP's point, not negating it.

replies(1): >>46233827 #
hombre_fatal ◴[] No.46233827{3}[source]
> I would hazard a guess that your knowledge lead to better prompts, better approach... heck even understanding how to build a status bar menu on Mac OS is slightly expert knowledge.

You're imagining that I'm giving Claude technical advice, but that is the point I'm trying to make: I am not.

This is what "vibe-coding" tries to specify.

I am only giving Claude UX feedback from using the app it makes. "Add a dropdown that lets me change the girth".

Now, I do have a natural taste for UX as a software user, and through that I can drive Claude to make a pretty good app. But my software engineering skills are not utilized... except for that one time I told Claude to use an AGDT because I fancy them.

replies(1): >>46234297 #
ModernMech ◴[] No.46234297{4}[source]
My mother wouldn't be able to do what you did. She wouldn't even know where to start despite using LLMs all the time. Half of my CS students wouldn't know where to start either. None of my freshman would. My grad students can do this but not all of them.

Your 20 years is assisting you in ways you don't know; you're so experienced you don't know what it means to be inexperienced anymore. Now, it's true you probably don't need 20 years to do what you did, but you need some experience. Its not that the task you posed to the LLM is trivial for everyone due to the LLM, its that its trivial for you because you have 20 years experience. For people with experience, the LLM makes moderate tasks trivial, hard tasks moderate, and impossible tasks technically doable.

For example, my MS students can vibe code a UI, but they can't vibe code a complete bytecode compiler. They can use AI to assist them, but it's not a trivial task at all, they will have to spend a lot of time on it, and if they don't have the background knowledge they will end up mired.

replies(1): >>46234943 #
hombre_fatal ◴[] No.46234943{5}[source]
The person at the top of the thread only made a claim about "non-experts".

Your mom wouldn't vibe-code software that she wants not because she's not a software engineer, but because she doesn't engage with software as a user at the level where she cares to do that.

Consider these two vibe-coded examples of waybar apps in r/omarchy where the OP admits he has zero software experience:

- Weather app: https://www.reddit.com/r/waybar/comments/1p6rv12/an_update_t...

- Activity monitor app: https://www.reddit.com/r/omarchy/comments/1p3hpfq/another_on...

That is a direct refutation of OP's claim. LLM enabled a non-expert to build something they couldn't before.

Unless you too think there exists a necessary expertise in coming up with these prompts:

- "I want a menubar app that shows me the current weather"

- "Now make it show weather in my current location"

- "Color the temperatures based on hot vs cold"

- "It's broken please find out why"

Is "menubar" too much expertise for you? I just asked claude "what is that bar at the top of my screen with all the icons" and it told me that it's macOS' menubar.

replies(2): >>46235449 #>>46236384 #
1. ModernMech ◴[] No.46236384{6}[source]
I didn't make clear I was responding to your question:

"Where do my 20 years of software dev experience fit into this except beyond imparting my aesthetic preferences?"

Anyway, I think you kind of unintentionally proved my point. These two examples are pretty trivial as far as software goes, and it enabled someone with a little technical experience to implement them where before they couldn't have.

They work well because:

a) the full implementation for these apps don't even fill up the AI context window. It's easy to keep the LLM on task.

b) it's a tutorial style-app that people often write as "babby's first UI widget", so there are thousands of examples of exactly this kind of thing online; therefore the LLM has little trouble summoning the correct code in its entirety.

But still, someone with zero technical experience is going to be immediately thwarted by the prompts you provided.

Take the first one "I want a menubar app that shows me the current weather".

https://chatgpt.com/share/693b20ac-dcec-8001-8ca8-50c612b074...

ChatGPT response: "Nice — here's a ready-to-run macOS menubar app you can drop into Xcode..."

She's already out of her depth by word 11. You expect your mom to use Xcode? Mine certainly can't. Even I have trouble with Xcode and I use it for work. Almost every single word in that response would need to be explained to her, it might as well be a foreign language.

Now, the LLM could help explain it to her, and that's what's great about them. But by the time she knows enough to actually find the original response actionable, she would have gained... knowledge and experience enough to operate it just to the level of writing that particular weather app. Though having done that, it's still unreasonable to now believe she could then use the LLM to write a bytecode compiler, because other people who have a Ph.D. in CS can. The LLM doesn't level the playing field, it's still lopsided toward the Ph.D.s / senior devs with 20 years exp.