Here's my constant question:

Everyone's going so fast that they keep hitting walls. Review, CI, product asking for things, whatever.

Why have we not seen an improvements in products?

While every post and thread feels like a 90's wall street office, the new android and iphone ship with fewer features than usual. No indie guys come up with a linux-sized alternative OS. Switch 2 remains unhacked. Windows takes 3 seconds to show the right click menu.

Is everyone just running full speed in circles or something?

In my case it's not the CI that's the bottleneck. It's the human testing side. Does it work, sure. But does it actually do the thing we want (and more importantly) does it do it in a way our customers will understand and actually like?

> Moving our workloads off GitHub Actions to third-party runners with faster CPUs, higher-performance storage, and better cache infrastructure gave us faster machines to run the same pipeline on

Yeah, was not surprised to read this. Actions is convenient if you already use GitHub, but it can also be pretty slow. Given reliability is also a major issue with GitHub these days I expect to see more orgs moving to different pipelines

Good thing Linear has been finished for a long time doesn't need more features, so AI coding can go slow. Oh damn, it's busy becoming the next Jira :(

My biggest takeaway (as a one man band 20 leagues below this spending time optimizing Gitlab free plan and self hosted runners): they did not worry about any of this until 100M ARR and more 1B valuation[0].

[0] https://linear.app/now/sharing-growth-with-the-people-buildi...

With the widespread increase in speed/commit cadence/content added to repos, I wonder if there's an angle here for companies with a lot of CI/CD needs to simply start self-hosting their CI/CD machines on premises.

In my experience, self hosting your CI/CD runners had the biggest impact in cost savings throughout, while also allowing for more powerful machines, which directly means quicker CI/CD runs

If you can pay the setup cost, Bazel will get you build times ~10s with a warm cache for even massive projects.

> Despite our test suites almost quadrupling since the start of the year

Did the tests produce four times as much value, though?

My take: it seems like systems should become smaller, more isolated, and contract-oriented.

I have been a long time proponent of monoliths, but it seems like agents would be happier with smaller, more isolated services. The more isolated, the better. Contracts between the service components only. Then it can iterate internally as long as it satisfies the contract. If it needs to, it can version the contract and keep iterating.

I suspect a substantial part of this is an avalanche of useless testing.

If you even review PRs still: when was the last time you didn't just skip over tests? And if you ever looked at tests in an LLM-heavy PR, how many of those tests tested something useful, and not just built-ins and trivial behaviours?

There's at least some awareness in the industry of how LLMs generate a lot of boilerplate in business logic. It feels like we're much less aware of how much of it is in tests.

Agentic Coding has been a huge strain on CI, I have been using Bazel to improve our build times and ultimately building customized runners to improve our CI

Anyways great blog post from linear team a lot to learn from it

I think to solve this problem properly, we have to stop treating "build" and "test" as separate buckets of work, to be designed and scaled separately. It's all CI. As soon as you try seriously scaling out tests, you will run into build bottlenecks. To truly scale CI you need a scheduler that understands your build, test environment, and all the glue in between, well enough to schedule it in a way that actually speeds things up. That is very difficult and not something that even the best build tools can do - yes, even Bazel. Bazel can run tests but it's not nearly as good at it than at building.

I call this moment "the Great CI Bottleneck of 2026". Here are my thoughts on it: https://dagger.io/blog/the-great-ci-bottleneck-of-2026/

I wonder if the GitHub actions outages we keep seeing is due to themselves making self-hosted runners paid, hence bringing broke/cheap users back from hosted runners to their garbage infra.Meanwhile I'm running my Codeberg Actions on the free Oracle ARM machine 2 cores 12GB ram (previously 4 cores 24GB) and way more reliable. Keep winning bozos.

Edit: After second thought, I guess "alternative runner" providers still have to pay the self-hosted tax. So M$ actually saves and makes money by not scaling their infra and driving people to alternative providers they can tax freely. Actual geniuses.

Our CI has thousands of tests and is done in 2 minutes. Everytime it grows above 3 I add more parallelization and keep it ultra fast. Been working great for our team

undefined

vitest load balancing on test duration times instead of by file would be a nice performance win

We are working on something here, pushgate.dev. It forces the agent to run tests; we are also working on adding support for static analysis of bundles using Jev. Onboarding is a bit of a mess right now, but send me a message [email protected] if you are interested.

Speed up CI and the next bottleneck just moves to deploy and rollback, which do not scale the same way.

undefined

Heh. I fixed my CI/CD flows last year to be no more than 60 seconds: https://pics.ealex.net/share/2IpZSt2cxrKWzeUokNNmk4eRI1jjzF2...

And this includes _full_ linting and unit tests.

I did that by self-hosting Github Runners, using Podman to build images (it doesn't squirt your build context through a socket every time!), and using Docker images as content-addressable cache.

I will do a write-up about it this week...

I just feel apps like linear are increasingly getting in the way of full send agentic development where sub agent orchestration is done through agent to agent messaging, work trees, on demand git restructuring and epoch specific coordination plains, often .md files. The smaller the human component of total product development gets, the more this may be the case.

> Despite our test suites almost quadrupling since the start of the year

I think the SDLC needs to be re-evaluated, tests, specifically unit tests, are just cosmetics. "wow its so easy to get high coverage now!" as opposed to why we're doing it in the first place

I don't find agents to be using tests any differently than a junior or mid level developer. I don't find humans to be using tests any differently than a junior or mid level developer either.

Basically, the tests are never guiding the features, they aren't highlighting regressions, the test is simply modified for the updated application. Unit tests especially.

I have found end to end tests to be useful, and also unblocked. They can tell when the actual user experience has changed, and go back to the feature's implementation to stop altering the user experience in unexpected ways.

additionally, I find this across every industry that was worried about AI. basically, whatever best practice was neglected due to lack of investment into your org is now being done without additional investment into your org. that's a good thing. now we need to look at the purpose of the best practice, and if that purpose itself was already solved in the process.

My recent winnowwallet.com still has build times of over 10 minutes. I'm convinced the agent really wants build times around 5m to move at a quick pace. Also to not drive me insane. It took a ton of work to get it down from 45 minutes because my tests launch a full version of the app, and walk it through major usecases while recording video and screenshots. I then use AI to qa this. It also generates its webpage this way. All from CI/CD

I cannot believe we are in 2026 and CI/CD hasn't evolved enough to even consider hot updates. Burn your CI/CD pipelines to the ground and start over again without any of the slop. Don't let anyone who calls themselves DevOps Engineers design it again, only people who are System Administrators.

[dead]

[dead]

[flagged]

undefined

(Disclosure: I'm a cofounder of RWX) For anybody wanting to solve similar problems and considering Bazel, take a look RWX. It's built around the same concepts of content-based caching and graph-based task execution, but it's far more runtime agnostic and easier to adopt. https://rwx.com

Maybe I'm alone in this, but as someone who is in tech, I don't know what CI is, and I don't think it's unreasonable to expect it to have an expansion within the article the first time you use it...

AI is in the cultural zeitgeist, but you gotta expand most other things at least once.