Most active commenters
  • yarekt(3)

←back to thread

Chat is a bad UI pattern for development tools

(danieldelaney.net)
858 points cryptophreak | 13 comments | 04 Feb 25 16:06 UTC | HN request time: 0.879s | source | bottom
Show context
wiremine ◴[04 Feb 25 18:13 UTC] No.42936346[source]▶
I'm going to take a contrarian view and say it's actually a good UI, but it's all about how you approach it.

I just finished a small project where I used o3-mini and o3-mini-high to generate most of the code. I averaged around 200 lines of code an hour, including the business logic and unit tests. Total was around 2200 lines. So, not a big project, but not a throw away script. The code was perfectly fine for what we needed. This is the third time I've done this, and each time I get faster and better at it.

1. I find a "pair programming" mentality is key. I focus on the high-level code, and let the model focus on the lower level code. I code review all the code, and provide feedback. Blindly accepting the code is a terrible approach.

2. Generating unit tests is critical. After I like the gist of some code, I ask for some smoke tests. Again, peer review the code and adjust as needed.

3. Be liberal with starting a new chat: the models can get easily confused with longer context windows. If you start to see things go sideways, start over.

4. Give it code examples. Don't prompt with English only.

FWIW, o3-mini was the best model I've seen so far; Sonnet 3.5 New is a close second.

replies(27): >>42936382 #>>42936605 #>>42936709 #>>42936731 #>>42936768 #>>42936787 #>>42936868 #>>42937019 #>>42937109 #>>42937172 #>>42937188 #>>42937209 #>>42937341 #>>42937346 #>>42937397 #>>42937402 #>>42937520 #>>42938042 #>>42938163 #>>42939246 #>>42940381 #>>42941403 #>>42942698 #>>42942765 #>>42946138 #>>42946146 #>>42947001 #
ryandrake ◴[04 Feb 25 18:39 UTC] No.42936709[source]▶
I guess the things I don't like about Chat are the same things I don't like about pair (or team) programming. I've always thought of programming as a solitary activity. You visualize the data structures, algorithms, data paths, calling flow and stack, and so on, in your mind, with very high throughput "discussions" happening entirely in your brain. Your brain is high bandwidth, low latency. Effortlessly and instantly move things around and visualize them. Figure everything out. Finally, when it's correct, you send it to the slow output device (your fingers).

The minute you have to discuss those things with someone else, your bandwidth decreases by orders of magnitude and now you have to put words to these things and describe them, and physically type them in or vocalize them. Then your counterpart has to input them through his eyes and ears, process that, and re-output his thoughts to you. Slow, slow, slow, and prone to error and specificity problems as you translate technical concepts to English and back.

Chat as a UX interface is similarly slow and poorly specific. It has all the shortcomings of discussing your idea with a human and really no upside besides the dictionary-like recall.

replies(9): >>42936954 #>>42937127 #>>42938119 #>>42938564 #>>42939410 #>>42943038 #>>42944645 #>>42946579 #>>42946796 #
1. yarekt ◴[04 Feb 25 20:27 UTC] No.42938119[source]▶
That's such a mechanical way of describing pair programming. I'm guessing you don't do it often (understandable if its not working for you).

For me pair programming accelerates development to much more than 2x. Over time the two of you figure out how to use each other's strengths, and as both of you immerse yourself in the same context you begin to understand what's needed without speaking every bit of syntax between each other.

In best cases as a driver you end up producing high quality on the first pass, because you know that your partner will immediately catch anything that doesn't look right. You also go fast because you can sometimes skim over complexities letting your partner think ahead and share that context load.

I'll leave readers to find all the caveats here

Edit: I should probably mention why I think Chat Interface for AI is not working like Pair programming: As much as it may fake it, AI isn't learning anything while you're chatting to it. Its pointless to argue your case or discuss architectural approaches. An approach that yields better results with Chat AI is to just edit/expand your original prompt. It also feels less like a waste of time.

With Pair programming, you may chat upfront, but you won't reach that shared understanding until you start trying to implement something. For now Chat AI has no shared understanding, just "what I asked you to do" thing, and that's not good enough.

replies(7): >>42938201 #>>42939091 #>>42939986 #>>42942735 #>>42944074 #>>42947003 #>>42954463 #
2. RHSeeger ◴[04 Feb 25 20:34 UTC] No.42938201[source]▶
I think it depends heavily on the people. I've done pair programming at a previous job and I hated it. It wound up being a lot slower overall.

For me, there's

- Time when I want to discuss the approach and/or code to something (someone being there is a requirement)

- Time when I want to rubber duck, and put things to words (someone being there doesn't hurt, but it doesn't help)

- Time when I want to write code that implements things, which may be based on the output of one of the above

That last bucket of time is generally greatly hampered by having someone else there and needing to interact with them. Being able to separate them (having people there for the first one or two, but not the third) is, for me, optimal.

replies(1): >>42945860 #
3. ionwake ◴[04 Feb 25 21:32 UTC] No.42939091[source]▶
this is so far removed from anything I have ever heard or experienced. But I know not everyone is the same and it is refreshing to view this comment.
4. freehorse ◴[04 Feb 25 22:32 UTC] No.42939986[source]▶
Pair programming is imo great when there is some sort of complementarity between the programmers. It may or may not accelerate output, but it can definitely accelerate learning which is often harder. But as you say, this is not what working with llms is about.
5. taneq ◴[05 Feb 25 02:16 UTC] No.42942735[source]▶
> As much as it may fake it, AI isn't learning anything while you're chatting to it.

What's your definition of 'learn'? An LLM absolutely does extract and store information from its context. Sure, it's only short term memory and it's gone the next session, but within the session it's still learning.

I like your suggestion to update your original prompt instead of continuing the conversation.

replies(1): >>43069485 #
6. skue ◴[05 Feb 25 05:09 UTC] No.42944074[source]▶
> I'm guessing you don't do it often (understandable if its not working for you). For me pair programming accelerates development to much more than 2x.

The value of pair programming is inversely proportional to the expertise of the participant. Junior devs who pair with senior devs get a lot out of it, senior devs not so much.

GP is probably a more experienced dev, whereas you are the type of dev who says things like “I’m guessing that you…”.

replies(2): >>42948827 #>>42950219 #
7. Tempest1981 ◴[05 Feb 25 08:56 UTC] No.42945860[source]▶
You could try setting some quiet hours. Or headphones.

Maybe collaborate the first hour each morning, then the first hour after lunch.

replies(1): >>42946723 #
8. andreasmetsala ◴[05 Feb 25 10:49 UTC] No.42946723{3}[source]▶
I think you missed the point. AI chat is not compatible with the solitary focused programming session.
9. viraptor ◴[05 Feb 25 11:25 UTC] No.42947003[source]▶
> but you won't reach that shared understanding until you start trying to implement something.

That's very much not my experience. Pairing on design and diagrams is as or more useful than on the code itself. Once you have a good design, the code is pretty simple.

10. balp ◴[05 Feb 25 14:22 UTC] No.42948827[source]▶
As a senior dev, when pairing with junious I get a more skilled team. Then I can continue to give new teams the skill and we all grow as people and companies.
11. kybernetikos ◴[05 Feb 25 15:49 UTC] No.42950219[source]▶
I don't agree with this at all. Pairing where there's a big skill gap isn't proper pairing, it's more like mentoring or hands on training.

In pair programming as I learned it and as I have occasionally experienced it, two individuals challenge each other to be their best selves while also handing off tasks that break flow so that the pair as a whole is in constant flow. When it works this is a fantastic, productive, intense experience. I would agree that it is more than 2x as productive. I don't believe it's possible to achieve this state at all with a mismatched pair.

If this is the experience people have in mind, then it's not surprising that they think that those who think it's only for training juniors haven't actually tried it very much.

replies(1): >>43068220 #
12. yarekt ◴[16 Feb 25 14:19 UTC] No.43068220{3}[source]▶
Exactly this. And I did mean equal skill when i said “more than 2x” implying that you get more done together than if you were separate.

One interesting thing is skill levels aren’t really comparable or equatable. I find pairing is productive where skills only partially overlap, meaning the authority on parts of implementation flows between participants, depending on the area you’re in.

I have some examples where I recently paired with a colleague for about 2-3 weeks nearly every day. i’m confident that what took us a month would be near impossible just working on our own

13. yarekt ◴[16 Feb 25 16:58 UTC] No.43069485[source]▶
When working together with a colleague, after some weeks each of you understand more how the other works, what they need to be productive, how to communicate better and achieve good results. Same with the domain context you’re working on, as your mental model grows more detailed, your performance goes up.

with LLMs that leaning is only context, and in case of “chat ai” is the chat backlog. It’s easy for the LLM to get stuck, and there aren’t many tools yet that help you change and morph that context to be more like shared understanding between two colleagues.

There is research going on in this area, eg “chain of thought”, so we’ll see, maybe things will get better