> It’s actually fine if agents make a lot of boneheaded mistakes. What’s not ok is if they keep making the same mistakes. 

I worked in life sciences for a bit. There is a process in clinical trials called corrective and preventative actions (CAPA). You'll also find this in other areas where failure tolerance is low (e.g. aircraft).

It's simple: when a mistake happens, you run your CAPA process (Google CAPA form and see examples to extrapolate what that process might look like) and determine the root cause and the correction to the process that allowed the mistake to happen in the first place.

(At least as a SaaS vendor in life sciences, when we had a CAPA (e.g. after a SEV0 failure), it would be folded into our SOPs and then we would be required to retrain on the SOP. Auditors would want to see our evidence of CAPAs, the versions of our SOPs, the records of training. All to extreme for most shops, but I add this for context/color)

This is something most eng shops do not have the discipline for since it requires some diligence.

Should it be fully agentic? Should there be human intervention here to approve the CAPA? Open questions to be answered.

Sorry to get hung up on what is probably just flavor text, but the first sentence.. "Agents can oneshot games that are actually fun", uhhh such as?

Would love to be proven wrong here, feels like I've just seen the same basic one-level kart racer in like 3 model announcements. Not even sure if they were one-shotted, I certainly would not describe them as "fun"

Author here, hello! Happy to expand on how we're thinking about this if any of it is unclear.

We wrote this post as part of a launch, which you can check out here: https://x.com/danlovesproofs/status/2095182189499711759

My bet is that we'll see a second layer of harness emerge, as self-driving codebases become the target. There will be an application facing harness, orthogonal to the agent facing harness. The app harness will represent the software factory that is emergent for the specific application being developed.

Anyway, here's mine, still wip:

https://hale-lang.org/docs/dna/

https://github.com/hale-lang/hale/issues/690

If you have a logo in the top left corner of your company blog the logo should route to the home page so I can see what product you're actually building.

It's a losing proposition. Self-driving cars go from point A to point B, and it's really easy to specify point B. My experience so far using agentic AI 24/7 for almost a year has been the AI can drive almost anywhere, so the whole problem is specifying where to drive. This doesn't get solved with faster, better AI, because all that will do is allow it to autonomously drive your codebase to random places you didn't want it to go. Then you're still left with the problem of specifying the problem.

But then there's the cases where the AI can't actually drive. What if you tell the agent to invent AGI? Or time travel? Where does it drive to? Does it tell you that it can't? How does it know it can't just drive here? How you you know? At the end of the day these are not wish granting machines, so someone connected to reality is still going to have to make decisions. And that person is going to be the one held liable for whatever the AI does so would they want a self-driving codebase in the first place?

Makes sense to me - I see it going down two separate routes in parallel:

One is to take the human out the OODA loop for cyber defense. Servlet libs (for example) are going to become fluid, self-modifying things w/ contracts that operate much differenlty from how they do today. The engineering practice around these things will need to change.

The second is that UX will be self-modifying. Just like how pi can modify itself, I can see this being a general practice for user-facing applications. Perhaps a text-box in which users can describe tweaks in can request changes to how the application functions. Engineering an application will focus on modeling the non-negotiables of how an application works, and providing the correct primitives for user-driven LLMs to modify software on the fly, as well as track and rollback changes. There'll also be funsies around how to ensure that ads get delivered regardless of the user trying to get rid of them.

This would be more convincing if there were a simple stat on the home page of the dispositions of the reported bugs (fixed/no-repro/won't-fix).

As a random sample of one, I looked at one of the bugs this reported on Tailscale (first thing on the homepage) [0], and the pull request ends with "Apologies for the lack of due diligence here. I'll go ahead and close this out."

[0] https://github.com/tailscale/tailscale/pull/17843

Is our fate to become more and more product designers and less and less software engineers?

I feel that way.

But I'm now able to actually create and release things. Before, I used to obsess over code architecture, optimizing every single line. I could and can craft beautiful code. And then I always burned out and lost interest (gemini). Now I don't touch code at all and have been actually creating and releasing the ideas I've always had in mind. Then I realized that's what I always wanted. The code was always a means to an end. But that's me.

In the end, we already have processes that work for humans and we know the types of testing, verification and review that makes a codebase grow healthily. We just need to adapt these designs the best we can to AIs.

AI have a lot of advantages over humans like speed, controllability, good defaults (e.g. Claude will write helpful comments unprompted which is better than most engineers), and the ability to implement best practices like comprehensive testing with much lower cost than a human team. Using AI in my personal projects, I am able to write software at a much lower bug rate than manually, because now I can make an automated test suite whereas previously I had no time to do so. However, AI also has downsides like rigidity, limited context window, and lack of creativity by default which needs to be resolved with harnessing (e.g. different agents with separate, well-compressed contexts) or worked around with human review layers.

I suspect the best core pattern for coding automation will be to create an engineering manager agent layer that can flexibly apply a set of engineering practices to the concrete problems at hand. It would handle all coding agent pings before it reaches a human, with prompting so that it reliably considers the big picture when making decisions. Very rarely, it would escalate an issue or plan review directly to humans when it is truly blocking, but most likely it would organize the information it obtains from the lower layer coding agents and surface them as reports for humans to review.

Kind of feels like we're in a similar place with AI programming as we are with self driving vehicles. Good enough for the typical case, maybe even better than most humans, but bad when dealing with novel situations or situations which require a really cohesive model of the system (code, business requirements, physical world, whatever).

My answer to this is adding an engineering assurance module to my specification and verification toolkit. Every project has a different definition of success and quality. My goal is to provide a consistent way to define, analyze, and report quality metrics tailored to each project.

https://github.com/agent-ix/engineering-assurance https://github.com/agent-ix/quoin

> Global memory

I don't get why we need global memory for code? Aren't code comments (even if invented for humans) the ideal place where to put "memories"?

The things in the 'what should be self-driving' section are the exact things i give to juniors so they can build their mental model of the codebase.

Now, I agree it'd be fantastic to have that automated, but how do we then share that context with the humans that will drive the non-self-driving actions?

I'm hoping the "blogging about doing" format will fade away, now that doing is cheaper than blogging. It used to be that the doing part was expensive, and risky. Now you can just do 5 things and throw away 4. So I'd rather see the finished product than talk about it.

I have integrated AI into my application. The source code is embedded into the binary. When someone questions the numbers in my app it can debug itself. If it truly finds a bug in itself it will tell the user and open a ticket. Soon we are going to let the application fix itself.

This article hints at it, but I think the blocker is environment.

How can you self drive an app on windows? There's no clear UI framework, design pattern, nothing which can bullet proof your app.

I bet you'd have better luck on Plan 9.

Remove the choice. Make frameworks which have limited options which are usable by default.

Not sure about you, but I've been writing real world software using agents for a while now. https://github.com/jaksa76/microfactory

I suspect that if you're not careful with agent memory it creates a danger of agent-driven cargo-cult behavior. I've watched this in my own ad-hoc agent loops where it starts with something basic, like the first agent tried to run some gigantic dependency inspection command and OOMed the local JVM and eventually recorded a workaround (to enable it to run gigantic dependency inspection commands...), and by time I get a few more agents into the loop, agents have written entire paragraphs about testing and validating local dev environment memory configurations that are mostly irrelevant to whatever is being worked on.

In general I've seen other issues like this where small errors and irrelevant comments in the codebase spin out into larger problems that consume annoying amounts of time/tokens. Maybe Anthropic and OpenAI don't notice this because they're in an "infinite monkeys with typewriters" scenario, but it's noticeable to me when the agent in my CLI has been spinning for 15 minutes contemplating irrelevant details

Super exciting, and absolute killers on the customer testimonial page.

Sounds like the next shitty Vercel ad at re:Invent.

   > Put another way: Detail’s secret plan is to create a product that bootstraps its own work, and then use that work to help engineering teams know what investments to make and how well they’re working.

Sorry, guys, I still don't understand what are you selling. Prompts? Design manual for building testable system? In which way agent-friendly dev env differs from human-friendly dev env?

I currently see a shit ton of work involved with "scaling" (aka cleaning up) the AI generated slop that was output in the past 2-3 years.

undefined

> Agents can oneshot games that are actually fun.

Stopped reading right there.

> Agents can oneshot games that are actually fun.

HAHAHAHAHAHAHAHAAH

here we go again

“how can we make ourselves even MORE replaceable?”

The missing piece in most versions of this is the feedback loop, not the harness. A codebase can only drive itself as far as it can tell whether it's still on the road: tests that fail for the right reason, error budgets, a way to notice that a change made something slower rather than broken.

Without that, "self-driving" degrades into applying diffs that compile. With it, the interesting question becomes what the system is allowed to do when the signal goes red - revert, retry, or stop and ask - and that's a policy decision nobody has good defaults for yet.

[dead]

I'm playing around with ecosystem driven development (hear me out ;)):

So i want claude to build a small game for me so of course i create one big file in which i write everything then i tell claude to analyse it and grill me (grill me skill) to clarify all smaller details.

This alone might lead to really good small sfotware but I still have to push it sometimes.

Now instead of doing this, i tell claude to build a small tool which generates a dashboard and memory and which can save specs and ask me choicses (do you prefer this color over the other, what do you think about problem A? What solution would you prefer? A, b, c or something else).

This does a few things:

1. claude doesn't has to save/store everything in a context 2. claude can now talk to that tool to ask it stuff 3. claude can now use the tool as a todo list 4. claude now can more easily spin up more agents in parallel 5. i have a nice interface and i can solve issues while claude works on unblocked tasks 6. I actually can follow the progress a lot easier

The only problem with this is: with the next update, you have to reinvestigate how claude was finetuned and adjust. A few month ago /goal was really good, now you need it a lot less because claude will do something for an hour without /goal

And the spec file only started to work after November/Opus moment but it got so good, that i can pack A LOT of stuff in a half structured markdown file and let it code what I need.

The progress is still too fast for the whole ecosystem