Hacker news

  • Top
  • New
  • Past
  • Ask
  • Show
  • Jobs

Not Everything Needs an LLM (https://newsletter.posthog.com)

186 points by speckx 3 days ago | 75 comments | View on ycombinator

socketcluster 3 days ago |

It is true that they don't tend to give simple answers.

But it reveals something about human biases that we tend to seek out simple answers to complex questions. We tend to think that because a question is short, that it's simple. We forgot how many details are involved and how much is actually going on underneath. Reality itself is often annoyingly complex.

All living creatures, including humans, are conditioned to simplify extremely complex things into simple concepts which we can react to. We only understand things to the extent that we believe they affect us.

b3ing 2 days ago |

Problem is tech is full of PMs and other managers that don’t know technology. Things that could be algorithms, functions, programs, regex filters, etc are now LLMs because they don’t know anything else to solve problems

dzink 3 days ago |

PostHog should not be using them on customer data, period.

throwawayffffas 2 days ago |

Using these decision models as a model router sounds really bad to me the models will most likely be newer than the decision model and thus the decision model will have no information about the capabilities of the model other than the short blurb you feed it.

TacticalCoder 2 days ago |

> A probability score of 0.85 still means ~150 wrong answers per 1,000 checks, so plan ahead for false positives and false negatives.

But that's only true if we believe the value of 0.85 in itself is correct!? For all we know it could be 0.75 or 0.9 no?

If were to make calls to something like JEV and know that the probability the answer is correct is, say, 0.85193847098713, wouldn't that be even more amazing than JEV and LLMs themselves!?

I'm not being facetious here: how do we know it's 0.85 and neither 0.75 nor 0.9?

And if 0.85 is "correct", why stop there? Why not 0.85193847098713?

Any explanation is very welcome.

lhk931122 2 days ago |

If this direction works out, I think service companies like Anthropic or OpenAI would provide it. If there is a lot of demand. Until then you have to set it up yourself locally, which is a hassle. Definitely, for components that act as sub agents in the agents I use personally, this direction actually looks useful for getting answers that rely even a little less on randomness and using them.

simianwords 2 days ago |

The fact that Jev can't do reasoning means that it can only be used for very basic decisions.

Jev can't answer this for example

    What is 6^8-3&2 mod 2? 0 or 1? 
You might think it is a contrived example but it is illustrative. It shows that almost most questions are reasoning shaped.

nostrebored 2 days ago |

I am so skeptical about these decision model rerankers. Training a reranker has never been easier, and many of them already return calibrated scores.

Decision models are often larger, slower, non colocated… it just seems like a strange tradeoff to make.

npteljes 2 days ago |

This reminded me of the Copilot Fusion feature. It tackles a similar problem in a similar way: classify or even split up the task at hand, and trying to use the best qualified model for each step.

HackerThemAll 2 days ago |

Let's reflect that statement: barely anything needs an LLM.

fauria 2 days ago |

Related: Amazon saves millions by replacing Sonnet 4.6 with Strands Decider - https://news.ycombinator.com/item?id=50436640

munishjindal931 1 day ago |

but many good things do need LLM. yes its so true that not everything needs that but how the things are moving it feels like everything soon will start needing it

bryanhogan 2 days ago |

Hearing about use cases for decision models and seeing how they are being explored is quite interesting, but I think the title of this post is poorly chosen.

saejox 2 days ago |

I have been using structured json outputs since gpt 4 days. Never understood this jev hype

aabdi 3 days ago |

Contrarian take. I think everyone should use LLMs and the most expensive possible, unless they know how to run evals (both on line and off line).

Yes its expensive, ugly, etc. But...

I've seen enough IRL that people just create really bad evals that do not reflect real world. Then the product is just bad. Outright broken for the p90 customer.

People don't know statistics. Offline Evals are hard. Sampling the entire population distribution is hard. Sampling the fat tail is harder. Online evals are big ops projects that are giant balls of death.

dieselgate 3 days ago |

Completely unrelated to the article itself but I landed on the posthog.com homepage after viewing their open positions and the landing page is cool. It's sort of a desktop UI and different from what I usually see, was expecting some other HR domain to come up.

nandanadileep29 2 days ago |

very true, many LLM usecases can be replaced with decision models like these, its wayy cheaper and faster.

vibebingo 2 days ago |

> What is the current conservation status of hedgehogs?

A harnessed LLM can look this up using tools.

Maybe LLM at least ti prepare context for decision model.

I guess if using a decision model context prep is key.

codedrift_ 1 day ago |

Not everything needs a decision model either. There's so many places people are shoving JEV in like it's a magic determinism layer. If your answer is squishy, then sure, use a squishy system.

> Decision models are up to 444.6x cheaper and 193.6x faster than frontier models

Yes, and two lines of even the worst JavaScript runs circles around making a service call

jakeydus 3 days ago |

"Not everything needs an LLM! Use a decision model!"

And what is a decision model exactly? An LLM with instructions to limit responses?

Huh.

Rapzid 2 days ago |

The approach of using decision models to gate more expensive work.. I mean, yeah.

But using protecting PII as an example use case is a bit bonkers?

tchalla 2 days ago |

Not everything needs a "decision model" with a "system one" API either but here we are.

jscd 2 days ago |

I continue to be astonished that a real company exists called "post hog"

undefined 3 days ago |

undefined

cognitif 2 days ago |

Wait, I don’t need an LLM to do arithmetic?

purusa0x6c 2 days ago |

exactly.. ppl are still using LLMs agents for simple direct workflows..

HacklesRaised 2 days ago |

almost nothing in fact...

brodydung 2 days ago |

[flagged]

johnh-hn 2 days ago |

[dead]

undefined 2 days ago |

undefined

chanhyounglee 2 days ago |

[flagged]

poxlr 2 days ago |

[flagged]

ClayCantrell 2 days ago |

[dead]

shipitx 2 days ago |

[flagged]

pj_mukh 2 days ago |

tl;dr Not everything needs an LLM, but everything needs a typesafe wrapper around an LLM.

Gotcha

gaal_dornick 3 days ago |

[dead]