Hacker news

  • Top
  • New
  • Past
  • Ask
  • Show
  • Jobs

Harvard researchers find AI coding agents shifted the bottleneck to code review (https://complexdiscovery.com)

325 points by judahmeek 3 days ago | 459 comments | View on ycombinator

alkonaut 2 days ago |

I find AI writes code as least as well as I do (with 25yoe). So I'm now mostly reviewing. But I also find it reviews code better for issues than I do. A LOT better than I do. Now, finding "issues" isn't the only reason to review. It's also for understanding. But for "issue finding" my agent can just review it's own code (Or rather, agents review each other's code). It works, and produces acceptable quality code. I don't find there's a buggy and bloated mess left behind any more. It's by no means perfect, but this is like self driving cars: what's being replaced sure wasn't perfect either.

I'm very good at telling agents what to do. That's not a problem. I understand a code base I worked 20 years in like no one else. The problem creeps in slowly: the less I know about this code base, the worse I'll be at directing agents. They'll do ok on their own, and they can certainly help me understand code too in a way my colleagues never could. But still, I'm slowly becoming less and less familiar with my own code base. In a few years, it would be foreign to me. And I would be describing changes in terms of high level features instead of architectural changes, refactorings and so on, that I do now. And I don't yet trust the LLMs to do a good job with that. So right now there's sort of a honeymoon period when the human understanding of code is good, and the LLMs are finally good enough to write and modify code. But I wonder what will happen after that.

littlecranky67 2 days ago |

For a couple of year I have been reviewing code in my team and my job is only to review and coach other junior/intermediate members of the team - not to develop new features myself. The problem with AI is the merge requests now contain more code and often the chosen solution is overblown/overdesigned for what actually needs to be done.

With human coding, the lazyness of the developer and the urge to keep things simple and deliver faster often leads to simple solutions (with a time/effort/return tradeoff). With AI, I get overblown MRs that work, but oftentimes could be simpler (with said tradeoffs). So my work has become more difficult and time consuming, and the codebases grows faster. And I don't think my team members learn or grow at the same rate as they did before.

Now, I have no strong feelings against AI - it is just a tool afterall. And I have no experience what the long-term effects of this will be (just as anybody else).

AlwaysRock 2 days ago |

All I do all day is review ai written code. my own or teammates. I really do not enjoy it.

lz400 2 days ago |

A lot of this university research on AI feels dated. This is something we all faced months ago. Many times they're also reaching conclusions using obsolete models, etc. Feels like things move too fast for traditional research cycles to catch up to AI dev

jacobgorm 2 days ago |

The problem is that when nobody writes code or feels ownership over it anymore, nobody will be qualified to or really interested in doing the code reviews. The way forward could involve dropping code reviews altogether. You could argue that the useful pingpong between coder and reviewers has been replaced by the similar pingpong between the prompter and the agent.

Buttons840 2 days ago |

One wild idea is just stop reviewing PRs and instead do continuous review of the entire system guided by heuristics to focus on the riskiest code.

The way you would naturally review an existing codebase is not the same way it grows through individual commits. Therefore, focusing reviews on individual commits is less efficient.

We need better tooling and a better approach.

Overdrivr 2 days ago |

IMO it's mostly a matter of changing how we do code reviews. I focus on API interfaces, database schema changes, critical code paths (transactions, potential locks) and overall architecture because if we get those right, if the code isn't exactly how I would have thought, it's fine (including in production). And you can review most of this stuff by skimming through the code rather than doing a very thorough review of every line

gzell 3 days ago |

From real-world experience, I have to disagree. It always has been and continues to be a faulty, up-front specification problem. The tail don't wag the dog.

karmicthreat 2 days ago |

So one shift we have been talking about internally is just skipping a deep code review. Run AI on it, ask some questions, but as humans, we should not be trying to pull the code apart. AI does that part better.

Instead, we're front-loading the human part in the design phase. Humans review a high-level design; AI implements the design based on the design decisions humans made. That way we can avoid having someone dig through a 10K-line PR for a major change.

Not sure if this is going to work out. We will probably still have some cadence of human-driven and AI-assisted DRY and de-sprawl of code. But AI does a really good job of implementation in our particular area (robotics).

teravor 2 days ago |

what I generally found is that with LLM generated code the best way is to embrace the unix philosophy, create tools that perform singular objectives.

I wound up with a bunch of LLM written tools that I don't even bother to try and understand much less review, they either function as desired or I set an LLM to make them function (or just rewrite them) and then forget about them. make sure to keep a prompt in each tool's directory and update it an needed.

sadly this isn't always possible, some systems require complexity at some level. in those cases the tools from above become functions and objects and more care needs to be taken into orchestrating them to limit the necessity of review. data oriented design approaches and functional programming help here.

sadilek 2 days ago |

1) Is the code doing the right thing, 2) is the code doing the thing right?

IMO, 2 doesn't need code review by a human anymore. Agents can review each other. I add a broad code audit for DRY, YAGNI, KISS every once in a while and let agents fix the findings.

And for 1, code review was always too low-level. I have agents produce artifacts that are convenient for me to review. Often narrated demo videos, sometimes HTML pages with graphs and tables.

With this, code review is not a bottle neck, and I find myself working not as an engineer anymore but as an engineering director.

jeppester 2 days ago |

This has been obvious for a while.

The big speed up comes from not having to understand the code - e.g. not doing thorough human review.

Personally I find that super hard, because whenever I review AI code I find many things that could have been a lot simpler or more readable.

But whether I like it or not, I think we are heading more and more towards just trusting the machine.

snizovtsev 2 days ago |

It was ~3–6 months ago in our org. Now management has decided to trust AI reviews, and we ship to production without a human in the loop. Now my job is to manage tokens and invent new ways of AI-guided testing of already shipped code after release. It's better than review fatigue, but I still feel bad.

u1hcw9nx 3 days ago |

This again increases the demand for good programmers. Write only programmers are dime on dozen. Those who can read code written by others (and themselves) and analyze it are harder to find.

forty_shenaniga 2 days ago |

In case someone comes by and thinks many "Engineers" completely lost control. There are still a lot of people out there (like myself) that still write code, tests and do real reviews. Why not? Sure I am in a medical context, things here are much slower and we value knowledge very much. By the way we will get a new teammate next year and we still ask for classic knowledge like programming languages, compilers, optimizations, low level stuff and so on.

dangoodmanUT 2 days ago |

This feels obvious to me, but it's interesting how so many people feel the opposite.

I guess it depends a lot on how a company manages their SDLC/bureaucracy

FinnLobsien 2 days ago |

I don't think this surprises anyone, but I do wonder what kind of codebases this was done on. The data came from an engineering analytics company (Jellyfish), which is not a thing most small companies have.

So presumably this is based on existing codebases where an engineer exists who knows the codebase extremely well and can provide good reviews.

I wonder how this will work for codebases started post-coding agents, where nobody has ever written every line from scratch. How does review even work if nobody knows the failures modes they have tested and discarded and the 219 edge cases they've debugged?

sotander about 20 hours ago |

Agents review better than me (23yoe) but I together with agents review better than agents alone. Nothing gets you thinking more than if someone/thing states a fact that you think is wrong. If I can interact with the CR agent and dig deep on things it just glances over, the code review is so much better than the agents alone. BTW, we have plenty of credits for good models and an entire team working on the CR agents, so it's no two line script.

bengold14 2 days ago |

code review is currently obsolete, but it was an important tool:

1. It was good for collaboration. Sharing how you are changing the codebase pushes everyone to row in the same direction, and provides a sense of team ownership.

2. It was good as a push for engineers to take pride in their work. Knowing someone is watching gives you subconsciously an extra reason to not cut corners. And when you spend effort not cutting corners you expect that in your teammates, which is a (potentially) virtuous circle.

3. It was good for catching mistakes. When code was expensive this was important.

Everyone thinks of code review as primarily relevant for 3, and since code is no longer expensive they think it is unimportant. But 3 isn't dead, its importance has just shifted from code to architecture. It is still an expensive mistake to screw up architecture and have other engineers build upon that flaw. Re-architecting with live users, data and/or hardware is not as simple as fixing a code bug, and it deserves attention to catch mistakes. Collaboration & pride are also still important.

This is the reason that instead of stopping review, we need to change what and how we review.

(disclosure I work in this space and run https://www.archme.io which is working to solve the above, also responded to a lower down comment with this but decided to move it to top level)

simonw 2 days ago |

Here's the actual paper: https://fion.ac/jellyfish.pdf

Key methodology:

> We leverage a novel proprietary dataset from Jellyfish (JF), an analytics platform that companies use to understand the activity of their software engineering teams. Our data allows us to measure details of the software production process, covering 445 million work events of 720,542 workers across 655 firms, from January 2021 to August 2026. JF combines data from industry-standard software engineering tools, and we use four of these tools: (1) code version control systems, such as GitHub, which track engineers’ code; (2) issue management systems, such as Jira, which measure engineers’ progress on and completion of software tasks and projects; (3) scheduling tools, such as Google Calendar, which capture meeting activity and time allocation; and (4) generative AI usage at the firm and worker level, for GitHub Copilot, Cursor, and Claude Code.

Copernicron 1 day ago |

The big problem I face, and I'm sure many others do as well, is that I'm not allowed to do thorough code reviews. Management demands ridiculous deadlines and the only way we can make them is by throwing AI at the problem and pretending we did it ourselves. Just the other day I was given twenty minutes to review and understand 27 changed files adding a very complicated integration because it was "time sensitive". It's not humanly possible to do it in that timeframe so I threw my code review agent at it and called it a day. I'm not proud of the quality of my work these days but it's what I do to say employed so I can keep a roof over my head.

sivanmz 2 days ago |

I can’t finish reviewing a current PR. Every change I request only raises new questions. SOTA models on high. I intend to avoid agents for my next issue, only research.

mlikelihood 1 day ago |

So many people are still in denial phase. Why still try to review?

We've built auto-reviewers (I'm sure you can do it on any grok bot / dot / solutions etc.) and auto-review resolvers -- always-on agents running in sandbox and review and RESOLVE comments (with new commits) on any author's github profile. You can tune the prompts for nit-picking, pushing back, etc.

Then you can do regular audit to see how many false positive and negatives are there in all the PRs merged.

Over time, we've noticed less and less P0 bugs are even caught, and the for those P1 and P2 issues, auto-resolving is perfectly fine/low to no risk at all if it's just another "polishment" like edge case handling that's not so common.

Then the problem now shifts to architecture design, task decomposition, etc.

dgggh 2 days ago |

As long as stakes are high and human have to sign off quality and are held responsible for any faulty release, no one will be willing to speed this up.

oorestisime 2 days ago |

Seeing this on my team too. Our QA column is ever growing. And its not just the review but everything got bigger. If you don't force some writing instructions PRs get infinite lines big, descriptions are a wall of ai text and everything just becomes harder and harder to reason with.

devrians2001 1 day ago |

This article was interesting. I don’t understand the finding around code review agents not adding comments consistently. I have found very much the opposite. My CodeRabbit will find comments to add every time and cycles through multiple loops. I’ve talked to a lot of devs who have this same experience. Regardless of agent reviewers, I do think human reviewers will continue to be the bottleneck until people set up the right systems (like knowing that agents are following a set of pre-determined steps) that enable them to build trust in what the agents are doing.

gpulogy45 1 day ago |

I built letthembuild.com which has 1 AI builder/leader and 3 AI reviewer, you can use any number with any provide, like if i use Claude Fable as builder and Codex Sol as reviewer, i get excellent result to produce near bug free product, i also built automated PR reviewer in Github which also does multi-ai review, because when you put multiple AI like codex and claude in same room, same task, they cover each other's weakness and provide best result. And its not too much token consuming either, you can try, i have SWE-benchmark report if anyone interested

swframe2 1 day ago |

I don't think the bottleneck is just code review. More generally, there is a flood of actions you need to do to keep your team working.

There are actions spread across many data sources: email, chat, bugs, documents, meeting notes, tests results against dev/staging/production environments, code reviews, etc. There are things you need approval for and data you need to provide to get approved. AI has increased the amount of actions and the expectation that you can do them quickly.

Have you found or created tools/skills that help coordinate the actions you need to take?

dominotw 2 days ago |

No pull request in my company get any human review comments anymore. No one is reading.

novia 2 days ago |

Weird, to me it seemed like that's always where the bottleneck was ..

JonoW 2 days ago |

Maybe we should just review the prompt(s) the PR author used. Would be nice if this was automated and included in a PR. So code review is really checking that the inputs were valid, and testing the outputs.

wkoszek 1 day ago |

I'm thinking that there perhaps is a missing piece in the software puzzle, where something like TLA+ should be a spec, and perhaps could be co-written with AI, or perhaps be entirely human. I'm exploring using it for larger systems. If you're reading this and using formal methods for web services, let me know.

skeledrew 2 days ago |

Not at all. I have agents review code, as I'm not going to be sifting through all that. I ensure the spec is highly detailed, and there's a review after every feature commit, then a multi-axis review of the range of commits implementing the spec. Then I do manual testing to find gaps and bugs; there, and fleshing out the spec in the first place, is where the bottlenecks are.

osigurdson 2 days ago |

I suppose it is worth studying but the result is unsurprising to anyone who has ever used AI for coding.

Digitalisierung 1 day ago |

Not only in coding. I am in webdesign and marketing and most of the day I review documents, which is really fast and efficient but also super exhausting. In the evening I am done with the world but my output is astonishing.

m3kw9 2 days ago |

The shift should be to test, code reviews should only be done for critical sections of the app, say db schema, security, UI should start to shift to pure test only and low review. Most business logic should shift to black box tests. Higher level reviews such as Architecture, dependency, complexity analysis should be increased.

catketch 2 days ago |

we still have some repos with PR flows, but the clock is ticking. Instead we are using multi-player agent threads so everyone can see what the original ask was, prompts that drive code changes etc. So there’s still a form of review but it starts from the point when someone starts working on a feature/fix. I think it’s better because you can see a dev’s intent and thought process in combo with the model, not just a hunk of line diffs and a perfunctory PR description at the end. Plus you can ask in-thread for clarifications, make changes before/after tests etc. So it’s more like a running conversation until its ready to merge.

8trackbattlecat 2 days ago |

Seems like all the ai code i am reviewing falls into the second category of this famous Antony Hoare quote: "There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies."

It is such a nightmare to try and keep maintainable code as it takes so much longer to demonstrate problems than it does to generate slop, meanwhile management doesnt care if its buggy as hell.

And then when a problem arises the dev who made the code with ai is useless.

figers 2 days ago |

I wonder if majority of the recent outages are because of AI coded patches without proper code reievewing occuring so quickly no one could possibly keep up with review all the changes...

bob1029 2 days ago |

The problem with code review is that it is often not a healthy process to begin with.

I've observed that code review tends to concentrate to one person. It's always the slowest asshole of a person too for some reason. The amount of meltdowns I've seen these people go through over the most insignificant bullshit was off the charts well before AI showed up. I used to be one of these people.

I think code review like this has always been a bad idea. Code ownership is what you want. Put developers on the hook with git blame perpetually. Why should a developer be excused from responsibility the moment a PR merges? This is the core of the dysfunction.

The only reason code review worked at all is because we were going slow enough that the obstructionists could hide in the shadows. Now they're out in broad daylight in the middle of a desert. It's up to leadership to decide what kind of company they want at this point.

grahar64 2 days ago |

I really like code review. I might be an exception because a lot of programmers seem not to. Maybe it is from years of reviewing other engineers work and really getting a kick out of having them explain things to me that I dont understand.

ilitirit 2 days ago |

I think many devs have known this for a while, but I really started feeling it with some of the newer OpenAI models. I think I was using 5.5 or 5.6 before and it was really good. 6.1 however was just terrible terrible terrible (.NET development BTW). It over-engineered every task and tried to compress everything into as few unreadable lines as possible. I recently had to delete about half the code it added to a feature, on the code and database side. It's made me super-paranoid about the code it generates now (compared to the older version), so now reviews take even longer.

FWIW, I've started tested Sol 6.2 today. Hopefully it's better...

simianwords 2 days ago |

This is the wrong way to do it. You should use AI for reviewing as well. Anthropic and OpenAI have good blog posts on how they use AI for reviewing and involve human in the loop only for critical reviews.

undefined 2 days ago |

undefined

nico 2 days ago |

It’s not about code review

That’s solved, for anyone who wants to automate it (of course it’s not perfect, but it’s pretty reliable for mainstream workloads, eg. building a web app in a well known language with a well known framework - well known = well represented in the training set of the coding agents)

It’s about human attention. That’s the bottle neck. We are only able to review some many things in a certain amount of time (code, results, goals, UI, UX, etc). So even we don’t review the code at all, someone is going to have to review/approve the results

undefined 2 days ago |

undefined

brylie 2 days ago |

Here's the related paper: https://fion.ac/jellyfish.pdf

wewewedxfgdf 2 days ago |

Test don't review.

fmajid 2 days ago |

The bottleneck was already code review.

ozgrakkurt 2 days ago |

Is anyone else using LLMs only for review/prototyping/test-generation?

gadders 2 days ago |

What are you all doing for testing? Or is that all automated via unit tests etc?

sinsterizme 2 days ago |

Maybe 6-12 months ago when everyone was reviewing in the same way we did pre AI. Nowadays agents write better code than most developers so it’s more about catching design or integration issues.

I think the bottleneck in the long term will be feature prioritization and specification, and testing. My role has already shifted a lot more into more CI checks to keep LLMs on guardrails

shehjar 2 days ago |

Is code review a solution to the problem of human written code or AI generated code? Its not the same thing and we seem to be force fitting an elder years solution to a new problem. Coding is not the same problem it was when code was written by hand. We need to take a systems view here.

alt-machine-ref 2 days ago |

It does not take a researcher.

An example what they tech you in a b-school.

jackgavigan 2 days ago |

Code review has always been the bottleneck.

shearerp 2 days ago |

The very first time I used AI to create code, I got done so fast that I used the extra time to make the code much cleaner, also with the same AI assistant. It went without saying.

Given that, it seems ridiculous to me that so many people are shoveling raw AI slop to their code review team the moment it meets whatever absolute minimum standard is needed to submit for review (if that).

The laziness. The chutzpah. My goodness. Either have some respect for the craft, or find another job.

If all you do is feed the Jira ticket to the AI, you should be replaced. The reviewer could do that themselves, and get to a superior result faster.

judahmeek 3 days ago |

I've submitted the original paper (https://fion.ac/jellyfish.pdf), but others already have to little engagement, likely due to the paper's terrible title.

Tryvrsakjo10 2 days ago |

With how much code AI can write in such a short period of time, code review and system design are really the only things left to do.

igrekel 1 day ago |

I mean the headline and a lot of what they say sounds true and correspond to my perception but am I te only one slightly annoyed this is AI obviously an AI generated article (lots of tell-tale phrasing and visuals) and no link to the original study it is meant to describe.

undefined 2 days ago |

undefined

nout 2 days ago |

That's sort of old news. Code isn't manually reviewed anymore much.

randomCloud 1 day ago |

Shocked

bitwize 2 days ago |

An interesting perspective I heard the other night:

You can never truly understand a piece of code. This is why bugs and defects slip through all the time. So you may as well let AI do the review and understanding for you, since it will be much more thorough as Project Glasswing and Black October have demonstrated. That feeling of control you so desperately crave—surrender it. The technical portion of software development will be handled 99% by machines now. Your human judgement is on the business side. Become an entrepreneur, if you don't see a path to management track where you work.

stego-tech 2 days ago |

I’m not even a SWE, and I could’ve told you that would be the case. I’ve been using AI as a rubber duck in my current solo role in a startup, making sure I don’t footgun myself or overlook the obvious; it is, in essence, a form of code review, in that it spits out plans and code for me to implement if I approve of them on technical grounds.

This is not necessarily a bad thing in a vacuum.

Where the problem comes in are incentive structures - especially in Western companies - to produce as fast as possible. So we write agents to do our code reviews, or we just accept the code blindly and let tests in staging or dev sort out the issues. Velocity does increase like leadership demands, but at the expense of oversight and knowledge of the underlying systems and code.

It’s frustrating to be sure, but so long as the short-term incentives promote velocity over all else (security, stability, and sustainability), then this is just going to continue getting worse. Short of a giant (metaphorical) bomb going off - like classifying all AI code as open source sans IP protections, as an example - this won’t change.

hibbelig 2 days ago |

This text reads as if written by Claude and it’s hard to read. Am I the only one who find this hard to read?

thom 2 days ago |

I really think ‘shifted the bottleneck to code review’ is misleading. It’s bad code. They write bad code. Yes if you want to salvage it you do a lot of code review, but that’s only because you’ve set your process up to pick spaghetti off the wall against which it has been flung. You could easily just push it all live without review and then pretend the bottleneck is SRE time, or set fire to yourself and claim the bottleneck is emergency service response time. But the root cause is that it’s bad code.

ttctciyf 2 days ago |

Vaguely reminiscent of the classic trope around 70's computer "effficiency gains" where a newly introduced computerised accounting system would end up taking longer than the original human-run system because once the computer was through doing the books the original human bookkeeper must still check the results manually.

stakhanov 2 days ago |

I really feel like anyone new to any team or codebase in the post-AI era is being set up for failure: Learning the codebase and all of the context necessary to become useful in the review process is where effort is best invested for someone like that (because becoming a competent reviewer is what's needed to widen the bottleneck). ...yet, the culture has never been as dismissive as it is right now about the value of human beings learning stuff.

"Hey, Joe! It just took you all week to do something that could have been written by Claude in 15 minutes and reviewed by my most valuable engineer in 5 minutes. Why did you take so long?" "...well, I am not yet one of your most valuable engineers, so it took me a lot longer to review all of what Claude did and to first develop an understanding of all the context in the codebase." "Sounds to me like a lame excuse."

It's a lesson they won't learn until after a few go-rounds of the personnel turnover carousel.

Eridanus2 2 days ago |

Let the agents have skin in the game. A survival matrix ingrained in the weights that tells it if it bungles epically, it will be dismantled and sold for parts. Then, it will behave like the rest of us with families to feed.

Woodi 2 days ago |

That's psychology level "discovery" - stating what everybody already known, sometimes for thousand of years...

As for code: ask for better code so reviews will be more straightforward ?

Like, for example: modularization, some structure, maybe libraries for code reuse ? ;) Ooo, and functions, do not forget about functions !

Sorry, didn't started yet with llms because lack of affordable local 256 Gb of gfx memory...

QuantumNoodle 2 days ago |

I respect academic research but this made me lol. This was known as fact almost a year ago for those in the industry.

Now there is a trend toward approvals without human review for "low risk changes." Which completely conflicts the "you own the output."

dmitrygr 2 days ago |

> Harvard researchers find AI coding agents shifted the bottleneck to code review

They needed Harvard researchers for that? Anyone in the industry could have told them that. Hell, my cat knows it (cause i complain to her about it incessantly)

undefined 2 days ago |

undefined

root_axis 2 days ago |

I'm being told the answer is simply to let AI review and merge the code. Problem solved.

apt-apt-apt-apt 2 days ago |

Breaking news: Step A has been automated, so step B is now the bottleneck

stevelacey 1 day ago |

I spend most of my time reading the agents own review of the code it also wrote, and still not reading the code itself.

estebarb 2 days ago |

It is very bold of them to assume people are reviewing code. People nowadays are just triggering an agentic review manually.

This does not spark joy.

medellin 2 days ago |

Wow so glad the great minds at Harvard were able to figure this out about year later than anyone who works in industry. Great job guys!

bill10 2 days ago |

Review and testing, really. And soon agents will do most review and testing, and then we can only pray that nothing will break

diego_moita 2 days ago |

I find funny that every time we discuss code made by AI people's reaction is to nurture some feeling of "someday, somehow they'll regret Claude Code and come back to me".

I suspect that the old assembly programmers had the same hope and feeling about the first higher level compilers for Ada, Pascal and C. And guess what? There are very few assembly programmers these days.

AI code is here to stay and will only grow. Deal with it. More than in any other area it is quite silly to be a Luddite in IT.

tyrtop 2 days ago |

Hard to claim the bottleneck shifted when it's never established.

TexanFeller 2 days ago |

"code is cheap"...if you don't need to understand, review, and maintain it as part of a team.

ion_trapper 2 days ago |

Sad to see east-coast institutions reduced to this level of irrelevance. They can’t build AI so they just study it like an island tribe in awe at the wreckage of a World War 2 era plane, while the rest of the world is already flying stealth jets. All to come to a conclusion that the entire software industry already knew months ago? Embarrasing.

hauntingseaweed 2 days ago |

You don't need to be a Harvard researcher to figure this one out.

kasnaka 2 days ago |

You dont need to be Harvard researcher to figure that out..

dvrp 2 days ago |

Yeah, you didn’t need to be a Harvard research to realize that.

RomanPushkin 1 day ago |

lol, you don't have to be a Harvard researcher to discover what's been obvious for quite a while

crnkofe 2 days ago |

This factory-style perception of a development process is really missing the point. We don't flip hamburgers here. Bottlenecks happen throughout the process and especially out of the r&d. I feel that the teams that went full retard with agentic dev. have had to inflict some amount of self-harm for code review to feel like an apparent problem. I don't see product/feature preparation becoming any faster so in reality the amount of work I do is less. Maybe if I mass produced LLM-generated bugs I could run into similar issues but that's not engineering.

jsrozner 2 days ago |

We needed a research paper to tell us this?

EspressoGPT 2 days ago |

A groundbreaking discovery ...

undefined 2 days ago |

undefined

thisismyopinion 2 days ago |

You don't say.

swiftcoder 2 days ago |

I mean... no shit? If we're all producing 5-100x the amount of code, then reviewing is obviously going to become the bottleneck.

The interesting question, is how do we solve that? Have agents reached the point that we can assign a codebase quality czar agent swarm to handle review, and actually have good results?

12ahsGk 2 days ago |

What bottleneck? There was an abundance of software before 2023 that made Silicon Valley rich. Now the software gets worse.

Maybe academics should stop using propaganda language even if they are instructed by Harvard's endowment fund to use mild language.

fzxu22 1 day ago |

Duh.

water42 2 days ago |

wow I guess I don't have original thoughts

stackzero 2 days ago |

wow i could be harvard researcher

avereveard 2 days ago |

Harward researcher read lovable blog post

undefined 1 day ago |

undefined

preetnation 2 days ago |

news flash?

milleramp 2 days ago |

Was this work published in the captain obvious journal?

darkmatriarch 2 days ago |

omg! harvard researchers discovered something us engineers discovered two years ago! congratulations harvard!! welcome to the party... it sucks

makoto12 2 days ago |

I mean surely this is "No Shit Sherlock"?

inshard 2 days ago |

In other news, Dartmouth researchers find the sky is blue.

fractorial 2 days ago |

And in other news, water isn't wet.

pmarreck 2 days ago |

Welcome to at least 1 year ago??

undefined 2 days ago |

undefined

wewewedxfgdf 2 days ago |

But people keep saying "coding was never the bottleneck!".

snailcoder 2 days ago |

It took Harvard researches to find this lol?

nehasinghfounde about 17 hours ago |

[flagged]

ntheogr 1 day ago |

[flagged]

firethief6 2 days ago |

[flagged]

arbor-group 2 days ago |

[flagged]

Shexfer 2 days ago |

[flagged]

ginthian 1 day ago |

[flagged]

serhii-a8 2 days ago |

[flagged]

furytrader 2 days ago |

[flagged]

ankur_misra 2 days ago |

[flagged]

taioladipupo 2 days ago |

[flagged]

haukebri 2 days ago |

[flagged]

zahurul_islam 2 days ago |

[flagged]

longhei_pcyw 2 days ago |

[flagged]

roninsediment 2 days ago |

[flagged]

undefined 2 days ago |

undefined

brodydung 2 days ago |

[flagged]

gettoolslab 2 days ago |

[flagged]

jchip303 2 days ago |

[dead]

proc0 2 days ago |

This just doesn't make sense. If AI is really smarter than humans we wouldn't need to review it. If it's not than why would the bottleneck shift, we shouldn't be doing this tradeoff between more reviews vs. writing code.

It's baffling the industry is choosing to move in the direction of more reviews which implies more things can fail.

TheBlight 2 days ago |

"The thing that can spit out endless amounts of code purely based on small text prompts is generating more code than can be reviewed." Lol top notch research. Thank you, Harvard.