A lot of companies are just "deploying a chatbot" and some of the results from this study show that this doesn't work very well. My experience is similar: deploying simple chatbots to the enterprise doesn't do a lot.
For things to get better, two things are required, neither of which are easy:
- Integration into existing systems. You have to build data lakes or similar system that allow the AI to use data and information broadly across an enterprise. For example, for an AI tool to be useful in accounting, it's going to need high quality data access to the company's POs, issued invoices, receivers, GL data, vendor invoices, and so on. But many systems are old, have dodgy or nonexistent APIs, and data is held in various bureaucratic fiefdoms. This work is hard and doesn't scale that well.
- Knowledge of specific workflows. It's better when these tools are built with specific workflows in mind that are designed around specific peoples' jobs. This can start looking less like pure AI and more like a mix of traditional software with some AI capabilities. My experience is that I sell software as "AI solutions," but often I feel a lot of the value created is because it's replacing bad processes (either terrible older software, or attempting to do collaborative work via spreadsheet), and the AI tastefully sprinkled throughout may not be the primary value driver.
Knowledge of specific workflows also requires really good product design. High empathy, ability to understand what's not being said, ability to understand how to create an overall process value stream from many different peoples' narrower viewpoints, etc. This is also hard.
Moreover, this is deceiving because for some types of work (coding, ideating around marketing copy) you really don't need that much scaffolding at all because the capabilities are latent in the AI, and layering stuff on top mostly gets in the way.
My experience is that this type of work is a narrow slice of the total amount of work to be done, though, which is why I'd agree with the overall direction this study is suggesting that creating actual measurable major economic value with AI is going to be a long-term slog, and that we'll probably gradually stop calling it AI in the process as we attenuate to it and it starts being used as a tool within software processes.