←back to thread

440 points pseudolus | 1 comments | | HN request time: 0.208s | source
Show context
Havoc ◴[] No.45063050[source]
Not sure what these guys are studying but can tell you in the real world - essentially zero AI rollout in accounting world for anything serious.

We've got access to some fancy enterprise copilot version, deep research, MS office integration and all that jazz. I use it diligently every day...to make me a summary of today's global news.

When I try to apply it to actual accounting work. It hallucinates left, right & center on stuff that can't be wrong. Millions and millions off. That's how you get the taxman to kick down your door. Even simple "are these two numbers the same" get false positives so often that it's impossible to trust. So now I've got a review tool that I can't trust the output of? It's like a programming language where the equality (==) symbol has a built in 20% random number generator and you're supposed to write mission critical code with it.

replies(14): >>45063417 #>>45063575 #>>45063964 #>>45064042 #>>45064413 #>>45064732 #>>45065017 #>>45065089 #>>45065569 #>>45065576 #>>45068813 #>>45069627 #>>45076092 #>>45093899 #
1vuio0pswjnm7 ◴[] No.45065569[source]
"...can tell you in the real world - essentially zero AI rollout in accounitng world for anything serious."

The jobs the reseearchers concluded were affected were "unregulated" ones where there are no college education or professional certification requirements, e.g.,

   receptionists
   translators
   software "engineers"
"Not sure what these guys are studying..."

Apparently, they studied payroll data from ADP on age, job title and headcount together with, who would have guessed, data from an AI company (Anthropic)

https://digitaleconomy.stanford.edu/publications/canaries-in...

This study has not been peer-reviewed

replies(2): >>45065727 #>>45067350 #
1. 0xdde ◴[] No.45065727[source]
It should also be noted that there are some pretty big flaws in the analysis. They mention "the distribution of firms using ADP services does not exactly match the distribution of firms across the broader US economy," but make no attempt to adjust their analysis for it. They also drop 30% of the data for which there is no job title recorded. With such a skewed sample, it's hard to tell how the analysis is supposed to generalize.