242 points by zerop 9 days ago | 40 comments | View on ycombinator
armcat 9 days ago |
mickeyp 9 days ago |
Most embedding/BERT models are far too small to truly grasp the nettles when an AI ask questions to them; and if you condition the AI to only search for really basic things, then... you're basically back to grep + regex.
You need to prune all manner of useless detritus that might gum up your matches like test files, documentation, and sidecars that semantically overlap or you're just going to match your tests. That's fine if you're building a document store for a business, but if you want a "one-and-done" solution for your codebase it's rarely good enough.
And if you do use a very large embedding model (which means it'll require running in the cloud as most company computers won't run it well), then... well, you're better off just using a language model as a subagent to scout ahead for you and elide the embedding model to begin with.
comandillos 9 days ago |
lavendermon 9 days ago |
I enjoy watching the slow burn approach Apple is deploying as they quietly gear up to offer some % of local inference on iPhones to provide most of the AI services consumers want. Local inference could enable a very low price point for AI services that Apple would leverage alongside their massive ecosystem advantages and privacy claims to compete very effectively for personal assistant AI subscriptions.
brrrrrm 9 days ago |
"why not use grep?"
pants2 9 days ago |
Foobar8568 8 days ago |
JustRouzbeh 7 days ago |
From my own experience, these learned methods from Apple, Spotify and others, are highly relevant to hyperscale only - when you are dealing with hundreds of millions of items to choose from (or more), and require super high relevance in the top-5. For all other use cases (up to a few million items) I found that just building a BM25 index, and giving the LLM a BM25 "search tool" and casting a "wide net" wins every single time compared to learned methods (though the latter are more publishable in recsys and icml).
[1] https://research.atspotify.com/2025/9/you-say-search-i-say-r...