198 points by etoxin 3 days ago | 98 comments | View on ycombinator
mafro 3 days ago |
ricardobeat 3 days ago |
What I'd like to see is what it is, how and why it works, ideally backed by some benchmarks. Otherwise it looks like just another pile of skills with unknown outcomes.
twen_ty 3 days ago |
In my experience, spec drift is the main reason why none of these tools work. Maybe they work for one shot greenfield feature generation but in a large, multi developer long lived code bases, specs rot and end up being more pain than they are worth.
cg-enterprise 3 days ago |
I also ended up doing my own thing mainly to address several omissions in the existing frameworks (for SDD I prefer to use Superpowers and/or Matt Pocock's skills):
1. Artifact staleness and tracking - if you have a structure around starting with something like an ADR, common patterns for the whole repo etc., it's super hard to keep track of and actually keep it up to date. You make a strong early decision in an ADR and realize that you have to change it later on, or diverge. These changes get rarely properly recorded.
2. Review loop - Same model review isn't enough, I want bunch of models bouncing off each other, whilst still using my subscription and not API.
3. Feature creep and deferral tracking - it happens a lot that you encounter either during review or one of the validation phases that you also need to implement x, which is not covered by the original spec. There are several options to handle that, with the key that all those decisions need to be tracked and at some point decided by a human.
4. Custom workflow with governance - I have my own preferred SDLC if you can call it that, which includes rounds of agentic review of spec, before manual approval gate, review loop with certain specification depending on the codebase, feature size, deferral rules.
5. Ceremony based on context - Because it would happen that some of the ceremony would get in the way at some points (like producing a 100+ loc spec for 10loc change) I ended up basically developing a flow to decide whether a feature actually needs the full ceremony (full lane) or we can simply use the native plan feature (fast lane), so that I don't have to go through the whole ordeal of steps, when I need a tiny change.
I forked this code and added bits that matched my flow and it works out pretty well https://github.com/nutthouse/tutti
wyum 3 days ago |
If you like this / SDD, I'd appreciate your feedback:
https://github.com/spekk-ai/spekk-cli
Similar iterative specs philosophy. Ours is a bit different because we focus on declarative specs and installable agent skills. We chose Go for simplicity and minimal requirements (single binary).
sheepscreek 3 days ago |
hmokiguess 3 days ago |
I am not yet convinced this space has been claimed in a way like say how “React” became the framework of choice in frontend development.
These initiatives all seem to tackle a “nice to have” or “quality of life” impact in the workflows in my opinion. Haven’t yet seen a proposal that addresses the larger problems, which, I think are even hard to quantify and explain still.
AI is great as a writing aid, still bad as a reading aid. To me the bottleneck remains the human and how can you help the human make better and more informed steering decisions backed by data driven and verifiable proofs against your system.
The organizing aspect of information feels like a step towards the bigger picture but not that much of a leap yet.
mmargenot 3 days ago |
I like the overall idea of this, but the writing of the spec to me is more like building the thing than the implementation itself, and needs more active human (and agent) cycles.
slowmovintarget 3 days ago |
It works fairly well, and it is definitely less heavy than SpecKit.
dmos62 3 days ago |
It's also worth noting that SDD is such a wide variety of approaches that the term on its own says very little. For example, SpecKit and SpecDD are both very capable SDD frameworks, yet they have only minimal overlap: SpecDD describes system components (with emphasis on boundaries), while SpecKit is a fairly advanced process for changing specs.
jmathai 3 days ago |
Last week, I gave a 471 line spec to implement a major feature and it didn’t flinch. I wrote about it here.
https://jaisenmathai.com/articles/sojourn-for-ios-was-45-one...
open-paren 3 days ago |
i find a small, human written spec to be much more effective than these large spec documents.
the idea is that you iterate with your agent to write the spec, you implement, then eventually that spec gets merged into a "spec corpus" that describes all the behavior of the repository. but i don't think that prose can ever enumerate all the behaviors required of code, nor should it. the spec almost immediately becomes out of date.
CharlieDigital 3 days ago |
Concepts links here: https://github.com/Fission-AI/OpenSpec/blob/main/docs-lab/gu...
All the docs here are the templates rather than the actual file (I presume: https://github.com/Fission-AI/OpenSpec/blob/main/docs/concep...)
Somehow not very confidence inspiring...
honkycat 3 days ago |
Easily my favorite spec driven development framework.
It scored really well in our internal evals as well.
I think it has the most sane ceremony and its model fits my mental model really well.
I've also been working on a TUI that will automatically generate "phases" with each one being a openspec spec.
I would highly recommend trying out SpecDriven development. I found it to be the most productive way to work with LLMs for larger tasks, and I have found that it improves performance on larger tasks.
To me, this is where LLMs should go. Collaborating on shared documents that serve as a contract that then gets evaluated post-implementation
I've made multiple attempts to write domain-specific languages for LLMs to use to guide software architecture so that I can have higher-quality software architecture and also so that it can communicate ideas to me in a more terse way.
One thing that I think LLMs are lacking right now is information density. I'm a guitarist, and I like this game called Rocksmith, but I think that its user interface kind of sucks. It's fun to play along with the songs, and it's fun that it scores me on the songs and gamifies playing guitar. For dense notes, the user interface just isn't very good.
That kind of made me think more about information density. A sheet of notes is very dense, but it takes a little more time to process. Guitar tablature is slightly less dense, but I think it strikes a better balance between treble clef and Rocksmith. Really, I'd rather have all three of those presented to me.
This is really where I'd like to go with how I'm writing software now. LLMs: I'd love to be able to just create a specification that is very dense and describes domain-driven design concepts to the LLM, and then have a workflow that will do adversarial review to evaluate those concepts after implementing a phase.
This also kind of solves part of the problem with design decisions and artifact storage and all those things that we kind of see LLMs scatter around a codebase. If it exists in the spec, then it can be referenced later, and you can document changes, etc. Also, if you do the spec right, it could be language-agnostic.
_blk 3 days ago |
GSD is quite OK but very token hungry (~4x but it buys traceability and has quite a powerful autonomous mode) I use gsd-meta-manager to manage multiple GSD projects in parallel without having to use a skill to query each project individually to see where they're at. Does openspec have something like that too?
bergie 2 days ago |
Then for implementation, I tell Pi to read the appropriate work document. Pi can also post updates as we go along.
gps372 3 days ago |
Also, this looks like something which leadership level folks need to adopt first and then somehow it needs to trickle down to PI planning and sprint planning. Would like to hear someone's experience on how this has got adopted in their org.
pramodbiligiri 3 days ago |
8cvor6j844qw_d6 3 days ago |
There seems to be a spectrum from fluid, iterative workflows like OpenSpec to more up-front alignment and control like Matt Pocock skills.
Curious what people have settled on.
nicotejera 3 days ago |
kimseungyong 2 days ago |
I still can't believe that an LLM can maintain all major service domains as documents
If too many documents are submitted, it will be hard to review, and if the domain is incorrectly set without me realizing it, the cost will be high when discovered later
jeremyjh 3 days ago |
Lately I've been using stock OMP and its close to the right balance but not quite enough of the brainstorming and spec maintenance built in. I've tried to layer some simple stuff on myself but with mixed results.
whinvik 3 days ago |
And my issue is always, how do I review so many files. Is the purpose just to go from 1 requirement prompt to so many spec files? How do I know the specs actually reflect the requirement?
I always feel I come away from projects like this even more confused.
makapuf 3 days ago |
esafak 3 days ago |
nullbio 3 days ago |
recroad 3 days ago |
Mugshelf 3 days ago |
chandlerklein 3 days ago |
evaltoken 3 days ago |
wyre 3 days ago |
Why does it need to have an entire CLI?
lofaszvanitt 3 days ago |
grohan 3 days ago |
aitoolcrux 3 days ago |
shivrajnag 3 days ago |
Wendell58 3 days ago |
Shacharp 3 days ago |
Recently, I've completely ditched the specification part. I found they just weren't useful over the longer term. I used an LLM to assess in both directions whether the code matched the specs and whether the specs matched the code. On both software projects this came out with huge divergence from spec to code.
Basically the old theory is true - the code IS the specification.
What I did find very useful and have retained is the process flow. Create a proposal, review the proposal, implement, review the code. Also useful was building and maintaining ADRs and invariant logs for where a unit test cannot be made to verify behaviour. The process and the ADRs, unit test, invariant log all help the software stay coherent as the LLM churns on it over many unconnected contexts.