490 points by Betelbuddy 1 day ago | 259 comments | View on ycombinator
Zigurd 1 day ago |
prometheus1992 1 day ago |
The task of auditing a business is extremely complicated by virtue of data/information being all over the place. AI had not gotten to a point of where it could outperform humans but in this experiment they find out a group of agents are good enough for this task. I still think there are so many poorly defined tasks that auditors just understand implicitly, eg data discovery; before the audit even starts, that humans will always be needed to initiate and conclude audits just like software projects.
chilmers 1 day ago |
On the other hand, with AI we'll use reinforcement learning to create agents without such evolved-in tendencies. Where (so long as we can solve alignment and cheating) they are trained to do the job, no matter how much effort it takes, or what's involved.
othorns 1 day ago |
Also most companies don't want to pay for people to go around checking every comma since it brings very little value, so there is not time for it.
With that said I think AI will change auditing, as most everything else, eventually and allow different ways of working, it's just a weird flex.
trentnix 1 day ago |
janalsncm 1 day ago |
> KPMG UK head of audit Cath Burnet said the technology had, in several instances, gone considerably deeper into issues than the regular audit team, identifying areas requiring further investigation and asking questions that human auditors had not necessarily raised.
A pretty natural question you could ask is how often the AI catches things humans miss, and how often the AI misses things humans catch, and what is the severity of these things. “Several instances” doesn’t really give me anything.
chirau 1 day ago |
AI labs were working with companies like Mercor for more than a year paying the consultants and accountants themselves from these very firms a good amount per typical task they work on everyday, some even providing actual documents.
The expectations were more rigorous, and the rubrics pretty expansive. so yeah, the agent trained from tens of thousands of consultants would eventually ask better questions than an individual consultant.
origini 1 day ago |
And every year a new human comes along and has to learn all of it again.
Of course ONE AI agent and teaching ONE AI is massivly in an advantage as soon as its as good as the avg person in the job and it blows everyone else out of the water as soon as it gets a tag better.
priyadarshy 1 day ago |
Opus, with access to all of our policies, repo, and the SOC2 control standards is more thorough and exacting about evidence than any auditor I've ever dealt with for any kind of security/compliance review.
It's almost to a fault, it sets unrealistic standards for certain things that need to be nudged back and it's also aggressive about inventing work to do that will keep you busy forever.
marconey 1 day ago |
I think this is the source FT article
odyssey7 1 day ago |
balderdash 1 day ago |
edoloughlin 1 day ago |
and
> "one AI agent failed to correctly match regulatory requirements with the company’s documentation"
This is a depressing pattern. The AI can bury you with the volume it produces, forcing you into cognitive surrender. Then it makes the dumbest of mistakes that you're too overwhelmed to spot. Luckily for them, that one didn't make it to the client.
seizethecheese 1 day ago |
globalise83 1 day ago |
bluegatty 1 day ago |
Solutions, implementation, perfect detail - that's another question.
howlingfantods 1 day ago |
Quotidian FP&A, controlling and treasury functions are done faster and more reliably with AI. I feel like people are purposefully ignoring the evidence here. Most of corporate work IS slop/busywork and that’s what AI excels at. Without AI, I’d prob have 4 more headcount on my team.
bell-cot 1 day ago |
Most famously - https://en.wikipedia.org/wiki/Enron_scandal
Is anyone in this space, to speak to whether AI agents asking "better" questions is a benefit, vs. a bad product-market fit?
ZiiS 1 day ago |
vcryan 1 day ago |
undefined 1 day ago |
redwood 1 day ago |
There are many downsides to this in that business cannot ever be perfect and a sample-based regime is useful because it allows for a few mistakes here and there it's not waste everyone's time at audit time
So similar to cybersecurity where you're going to need to apply AI offensively and defensively I suspect we'll see the same in accounting
jdw64 1 day ago |
joe_the_user 1 day ago |
But they can also fail to ask "obvious" questions and get stuck at points that an average person wouldn't be stuck in.
Which makes them for adversarial processes - like auditing or hacking. But no so good at producing defect-free software or planning without guidance. With guidance, they're great aids in this too. But when people AI processes to "turn off their brains" or act autonomously, then you get problems. And you get a lot of pressure to very lightly if at all scan AI output when AI investment is supposed to show productivity gains.
xugencheng2024 about 4 hours ago |
ravenilands 1 day ago |
jordanb 1 day ago |
shye 1 day ago |
quantified 1 day ago |
If the efficiencies don't cut the fees to near zero, the whole "superabundance" talk will be put to lie.
undefined 1 day ago |
amirmi78 1 day ago |
letmeguessok 1 day ago |
queueueue 1 day ago |
modeless 1 day ago |
qahsfss 1 day ago |
melonpan7 1 day ago |
santiagobasulto 1 day ago |
I think this is expected in any area, especially technical ones.
LLMs will perform better than most “humans”. The key is to pinpoint the threshold that represents “most humans”. Is a powerful LLM with access to RAG-powered knowledge/grounding better than a recently graduated accountant? Most likely yes.
We have to be ready for LLMs to outperform average humans in every area. And given we’re lognormal/pareto distributed, that means “most humans”.
LLMs will be great companion tools for “average humans” and above-average individuals will most likely go towards training/grounding and evaluating AI agents.
vibebingo 1 day ago |
witnesser2 1 day ago |
pembrook 1 day ago |
rosstex 1 day ago |
Scubabear68 1 day ago |
Alas, would that we had independent audits, ratings agencies, etc. But I'm just a beautiful dreamer that way.
FrustratedMonky 1 day ago |
mattoxic 1 day ago |
j45 1 day ago |
The accountants will not have this ability to use AI because their fantasy of managing and overseeing IT wasn't able to be handled.
thesash 1 day ago |
lowbloodsugar 1 day ago |
anuubis 1 day ago |
agent_yard 1 day ago |
francuza9 about 22 hours ago |
witnesser2 1 day ago |
mrjjooyy 1 day ago |