296 points by craigmart 3 days ago | 235 comments | View on ycombinator
OtherShrezzing 3 days ago |
asmvolatile 3 days ago |
girvo 3 days ago |
Im already spending my $2k usd per month in tokens with just one or two agents. How do they expect me to do that?
koolba 3 days ago |
Whether a human can or should review every diff is a conscious project level decision. I have yet to see a project where it is not feasible. The trick is to break things into small enough pieces that you can review them. Just like one would do if you were doing such changes manually. Nobody in their right mind performs non-mechanical changes of 100s of files.
tshaddox 3 days ago |
I stopped using as my daily driver around 9 months ago, and at that time it felt very heavy/janky and seemed to have very regular bugs and breaking UI changes.
I’d be happy to hear that it has gotten better!
lasergyro about 2 hours ago |
r0m4n0 3 days ago |
Narann 3 days ago |
The future of IA would actually be to large scale prototype easily, even more like Python was a prototype language.
FartyMcFarter 3 days ago |
If the tests are hermetic, you should always be able to find the root cause (by bisecting to determine when the test became flaky).
If the tests are not hermetic, it should still be possible to identify the dependency that is making them flaky, and use that as a justification if it still makes sense to delete the test.
VadimPR 3 days ago |
The QA workflow I worked out is pretty much mirrored in this article, so speaking from the real world, this is the advice to follow. I'm on an Ultra plan for this work.
holoduke 3 days ago |
atoav 2 days ago |
At some point that annoyed me so much, that I just stopped.
jwpapi 3 days ago |
The rest of stuff I barely care anymore. To me my current setup it just works.
gregwebs 3 days ago |
https://github.com/gregwebs/skills-sdlc/blob/main/skills/ver...
The implementation process that I am using also looks for verifications during planning and reviewing and makes sure they are completed. It does use a lot more time and AI spend than just asking AI to do something. I still do the latter for simple or noncritical changes. I had to switch to using DeepSeek as my workhorse to keep down my spend.
https://github.com/gregwebs/skills-sdlc/blob/main/skills/imp...
undefined 3 days ago |
sashank_1509 3 days ago |
They won’t even do all this verification nonsense and their end product will be of superior quality.
d-sanoj 3 days ago |
nelox 3 days ago |
anshumankmr 3 days ago |
nandanadileep29 3 days ago |
jeffybefffy519 3 days ago |
Otherwise the type of engagement loop that you get from just clauding every feature end to end pushes you to drop important quality steps because the agent is covering it but often it just isn't and you get slop with regressions everywhere.
TLDR we think of the frontier as "the best"... but really its the most unstable in this new world - Claude Code, Codex and their supporting models absolutely show this time and time again.
yuzzuwo 43 minutes ago |
shivam_isonapse 1 day ago |
dklabsdev about 18 hours ago |
harisfarooq 1 day ago |
johnrc 3 days ago |
arbor-group 3 days ago |
CivilisedG 2 days ago |
adilbasari 3 days ago |
tanmaycodex1 3 days ago |
oceanrelay 3 days ago |
Yaojia666 3 days ago |
fk202611 3 days ago |
abdelmomenelsha 3 days ago |
i_rutul_patel 3 days ago |
dotdi 3 days ago |
this u?
Ampersander 3 days ago |
saagarjha 3 days ago |
> You can also close the verification loop by using the results to improve the agent’s performance on that task. For example, if an agent repeatedly fails a test that checks logs for user data, you can create a hook or a more explicit rule in your CLAUDE.md to avoid future issues. This (mostly) stops issues from happening again.
There are people out there who are doing all sorts of crazy neat things in the verification space using the tools you build and yet the people who could be helping seem to have no good ideas on how to help :/
titzer 3 days ago |
How about making Claude generate scripts that you can run?
Algorithms are better than agents. In fact, the whole point of AI is to reduce thinking to algorithmics. I didn't see a lot of evidence that the team understands this. If you need a magic box to test your other magic box, you have magic boxes all the way down.
TheRoque 3 days ago |
ForHackernews 3 days ago |
"The resource at “https://claude.dev/_next/static/chunks/241gnzjfqbp9f.css” preloaded with link preload was not used within a few seconds. Make sure all attributes of the preload tag are set correctly."
lavendermon 3 days ago |
Aperocky 3 days ago |
For all of it's checks, there are not one mention about simplicity, conciseness or architectural coherence. And when you don't have your AI agents care about those things when you run them by the hundreds. You'll get spaghetti that works on a surface level, for now.
dude250711 3 days ago |
I am more interested in the remaining 10%.
preordained 3 days ago |
The projection of some hurt feelings (or actively hostile actors?) in these comments is kind of insane.
lmeyerov 2 days ago |
I got excited to see where they are on going beyond testing, but guess that means they are in the crawl phase.
jordemort 3 days ago |
we don’t actually do any of the work but we think about it
mjg2 3 days ago |
inamorty 3 days ago |
matltc 3 days ago |
causal 3 days ago |
The problem is rarely whether or not the code works: it's whether the agent really understood the problem and built the right thing. It can write perfectly good verification for the wrong problem.
For any non-trivial app it can take a deep domain expert days to determine whether the agent really understood the problem. I don't see that addressed at all.
whamp 3 days ago |
brunooliv 2 days ago |
If anything everything written in this post is the opposite of verification as it wraps deterministic tools around a totally random LLM layer. What’s the industry gone to?
camgunz 3 days ago |
undefined 3 days ago |
123ajhgF 3 days ago |
More snake oil and theatrical agent setups described in insufferable Claudish. I think the target group of Claude is people with an IQ below 100.
mococa 3 days ago |
ankur_misra 3 days ago |
runtarmac 3 days ago |
makercfd 3 days ago |
anatolii1309 3 days ago |
frankmakesit 3 days ago |
amarinhodev 3 days ago |
sick_of_slop 3 days ago |
yoy0721 3 days ago |
ahmedketata 3 days ago |
jonenst 3 days ago |
Eunice99x 3 days ago |
hliyan 3 days ago |
Perhaps we need a level above linters: a tool that constructs the abstract syntax tree and verifies its various branches against deterministic rules.
How to do this for architecture: unsure.
mohsen1 3 days ago |
There’s a long section in the middle where the article describes per-change verification via skill files and “ask Claude to use the chrome-for-Claude tool” when checking if changes have introduced regressions.
But there’s already tools which do this without spending tokens tokens. Playwright will achieve equal quality but deterministic results across the same verification attempts. It is write-once-use-often, so has a marginal cost near zero.