Hacker news

  • Top
  • New
  • Past
  • Ask
  • Show
  • Jobs

Launch HN: StarSling (YC P25) – Self-improving CI for agent-volume repos

87 points by yonasb 3 days ago | 52 comments | View on ycombinator

colek42 3 days ago |

We should probably be measuring deployable merges to main. The math just doesn't work for pipeline based CI as you add more agents.

Three things matter.

1 -- How many non-conflicting changes can actually be made at one time (Fan Out)

2 -- How long a critical path takes

3 -- How many times you have to run that critical path. Most orgs run it 2-3x, once before the PR in the agent loop, once in CI, and another in the MQ

This is why most orgs cant get past about 12 PRs an hour.

devkumardev2001 about 22 hours ago |

Same test IDs on both runs is the right bar. I’d add one more check before calling a re-shard a win: Playwright shards by file, not by individual test. So equal file counts don’t mean equal workloads. One slow file can dominate a shard, and once it moves, another may become the bottleneck. Track the longest shard’s duration over time, not just the median.

On Bnjoroge’s question, compare the total test count in the merged report with npx playwright test --list on the same commit. That gives you a simple check that the expected tests are represented in the results.

Two lessons from running Playwright in Kubernetes: CPU throttling can cause timeouts that look like flaky tests, so monitor throttling before increasing retries. And keep flaky tests in a separate, non-blocking job rather than letting an optimizer remove them. Otherwise, you improve the metric without fixing the underlying issue.

Disclosure: I work on TestDino, a Playwright reporting tool.

emscb 2 days ago |

So the agent compares the changed workflow with the existing one and judges whether the speedup is significant. But a change that re-shards or skips tests would always look like a win on speed alone, wouldn't it? How do you check that such changes still catch the bugs the old workflow caught? I had the same worry when I trimmed our tests because CI was too slow.

bekacru 2 days ago |

Bereket from Better Auth - we've been on StarSling for about a year now, it's been such a hard a problem for us ot solve CI to the amount of PRs we get and continues need of optimization and faster runners, im so bullish!

ansh25lehri 1 day ago |

Interesting concept. Is there a plan to support Kubernetes-native CI like Argo Workflows, or is GitHub Actions the focus for now?

Also, since some wins come from app code (like the better-auth fix), how do you decide when a change outside the workflow/test files is in scope? Beyond the test-parity check, is there a guardrail on how much business code the agent can touch?

Shexfer 1 day ago |

The parity check — same test IDs, same coverage — answers the "re-sharding vs deleting tests" worry, but it still trusts the test set as the oracle. The failure I'd want guarded is a change that keeps the same tests but makes them faster by weakening what they assert: a looser timeout, a mocked dependency, a skipped setup.

Is the agent diffing the test bodies as well as the run results? A guard on test-file changes would make the speedup claim much easier to trust.

jstrasner 3 days ago |

So good! CI is still such a pain. Out of curiosity, do you use any open source harnesses under the hood or did you roll your own?

codedrift_ 1 day ago |

In a former life, I ran platform teams and the CI speedups we fought for rarely lasted. A few months later someone adds a sleep to a test or a new job skips the cache, and the build is slow again.

How does StarSling handle that drift? Does the loop keep watching a workflow after a PR merges, and will it catch a regression on a win it already shipped?

esafak 3 days ago |

As you note, we can do those optimizations ourselves. The kind of person looking for CI runners is likely doing that already. What I value is speed, cost, scheduling delay, reliability, observability, local reproducibility, not using YAML... There is a lot of value to provide but workflow optimization is not where I would look.

poxlr 2 days ago |

Curious how the inference cost of the agent loop compares to the CI minutes it saves. On a smaller repo I could see it eating most of the savings. Is there a repo size where it starts to clearly pay off?

sochs 3 days ago |

Off topic but every time I read a name like "Star Sling" I get excited to see another space startup tackling a difficult challenge; and so when it turns out to be CI for agents, it's a letdown and an annoyance. Idk, this project seems cool, but maybe pick a less pretentious name.

brunoarueira 2 days ago |

Congrats on the launch! In the past pre-AI I did an improvement to the github actions pipeline from 20 minutes down to 6, it's a relief when you improve things like this and improve the team throughput!

jbqueries 2 days ago |

An agent that opens PRs to speed up the CI for the PRs opened by agents. Ouroboros, now with per-second billing.

Actual problem tho, congrats on the launch!

MisterMunchkin 1 day ago |

What’s stopping Anthropic from just reading your agent logs and reproducing your product in 30 minutes?

dotmike 3 days ago |

Congrats on the launch! Will give it a run. Why is your CI fast? Curious to hear the technical details

d-sanoj 3 days ago |

Congrats on the launch. This seems interesting.

tristenharr 2 days ago |

Hmm this is very interesting! Good idea.

buildwrangler 3 days ago |

Congrats! I spent alot of time scaling really large and complex CI and integration testing builds so its great seeing a product that can help devs with this.

gkoberger 3 days ago |

Congrats on the launch! These benchmarks look really great, can't wait to start using this.

What are some of the most interesting things the AI has caught + fixed so far?

kinfish 3 days ago |

congrats on the launch!