Nobody knows how to build with AI yet

> Search has just become Chat

I think chat-like LLM interfacing is not the most efficient way. There has to be a smarter way.

There’s an efficient way to serve the results, and there’s an efficient way for a human to consume them, and I find LLMs to be much more efficient in terms of cognitive work done to explore and understand something than a google search. The next thing will have to beat that level of personal mental effort, and I can’t imagine what that next step would look like yet.

I think Photoshop is a good guide here.

Famously complicated interface with a million buttons and menus.

Now there's more buttons for the AI tools.

Because at the end of the day, using a "brush" tool to paint over the area containing the thing you want it to remove or change in an image is MUCH simpler than trying to tell it that through chat. Some sort of prompt like "please remove the fifth person from the left standing on the brick path under the bus stop" vs "just explicitly select something with the GUI." The former could have a lot of value for casual amateur use; it's not going to replace the precise, high-functionality tool for professional use.

In software - would you rather chat with an LLM to see the contents of a proposed code change, or use a visual diff tool? "Let the agent run and then treat it's stuff as a PR from a junior dev" has been said so many times recently - which is not suggesting just chatting with it to do the PR instead of using the GUI. I would imagine that this would get extended to something like the input not just being less of a free-form chat, but more of a submission of a Figma mockup + a link to a ticket with specs.

I find a well-written human article or guide to be far more efficient when it exists. But if AI rehash them... then the market for those may disappear, and in the process, the AI won't be very good either without the source to summarise.

There is certainly much innovation to come in this area.

I'm thinking about Personal Knowledge Systems and their innovative ideas regarding visual representations of data (mind maps, website of interconnected notes, things like that). That could be useful for AI search. What elements are doing in a sense is building concept web, which would naturally fit quite well into visualization.

The ChatBot paradigm is quite centered around short easily digestible narratives, and will humans are certainly narrative generating and absorbing creatures to a large degree, things like having a visually mapped out counter argument can also be surprisingly useful. It's just not something that humans naturally do without effort outside of, say, a philosophy degree.

There is still the specter of the megacorp feed algo monster lurking though, in that there is a tendency to reduce the consumer facing tools to black-box algorithms that are optimized to boost engagement. Many of the more innovative approaches may involve giving users more control, like dynamic sliders for results, that sort of thing.

English and other languages come with lots of ambiguity and assumptions. A significant benefit of programming languages is they have explicit rules for how they will be converted into a running program. An LLM can take many paths from the same starting prompt and deliver vastly different output.

> Famously complicated interface with a million buttons and menus.

Photoshop is quite nice for an expert tool. Blender is the complicated one where you have to get a full-sized keyboard and know a handful of shortcuts to have a normal pace.

> The former could have a lot of value for casual amateur use; it's not going to replace the precise, high-functionality tool for professional use.

I was just discussing that in another thread. Most expert works are routine, and they will build workflows, checklists, and processes to get them to be done with the minimum cognitive load. And for that you need reliability. Their focus are on the high leverage decision points. Take any digital artist's photoshop settings, They will have a specific layout, a few document templates, their tweaked brushes. And most importantly, they know the shortcuts because clicking on the tiny icons takes too much times.

The trick is not about being able to compute, it's knowing the formula and just give the parameters to a computer that will do the menial work. It's also not about generating a formula that may or may not be what we want.

Search wasn't just "search". It was "put a prompt in a form and then spend minutes/hours going through various websites until I get my answer". LLMs change that. I don't have to go through 20 different people's blog posts on "Which 12v 100Ah LifePO4 battery tests for the highest watt hours", the LLM simply just gives me answer that is most relevant across those 20 blog posts. It just distilled what I would have taken an hour to do down to seconds or 2 minutes.

I do agree… perhaps the thing to do is write fragments of the program, like the start and end, asking it to complete the middle. If you have precisely described how the output will be printed, for example, then you have essentially formally specified how the data should be organized…

> LLMs change that

Yup. Now you get a quick reply and have to then do the same job as before to validate it. Except all websites are deploying crawler countermeasures so it takes even longer now.

I don’t disagree with that at all, but that’s not what I’m talking about. The market for serving information goes where the people want to consume it. The old portals of the 90s gave way to search because it was easier for people to find what they wanted. LLMs give people an even easier way to find information. The downstream effects don’t factor into most people’s decision to use an LLM over source material.

A nice article on this was in the January issue of the Communications of the ACM [1]. With a reference to a piece by Dijkstra predicting that this is never going to be effective, back in 1979.

Being able to write code in a programming language is a feature, not a flaw. If we had always had to program in natural language, the precision and unambiguity of programming languages would be an eagerly welcomed revolution.

[1] https://cacm.acm.org/opinion/on-program-synthesis-and-large-...