Hacker news

  • Top
  • New
  • Past
  • Ask
  • Show
  • Jobs

Cerebras Disaggregated Inference (https://www.cerebras.ai)

108 points by emschwartz 2 days ago | 27 comments | View on ycombinator

aabdi 2 days ago |

5x throughput more than b4, same 10x tokens/s as compared to standard.

looks like disagg/wide ep is applicable to cerebras chips as well. kind of amazing.

https://www.cerebras.ai/blog/how-we-deploy-trillion-paramete...

hephaes7us 2 days ago |

It sounds like this might mean they could do real prefix caching?

Cerebras would be much more interesting to me if cache hits resulted in a commensurate savings on my end.

ismailmaj 2 days ago |

Glad they are upfront about their chips being a bad fit for prefill workloads, their previous article completely dodged the question: https://www.cerebras.ai/blog/how-we-deploy-trillion-paramete...

liorn 1 day ago |

I hope this convinces more teams who self-manage inference to test P/D disaggregation on their GPUs.

Many teams are worried to take the leap, but the technology and support around it are pretty solid. For some use cases the performance boost is significant.

speedping 2 days ago |

Cross-platform prefill/decode split is an incredible feat, kudos to them. I wonder how much time is spent transporting kv cache, and if the gains would be even higher for LLMs with ultra-compressed kv cache like the latest deepseek.

augment_me 2 days ago |

Have they solved the issue yet where they need to keep running ghost tokens to keep the board hot to prevent it from cracking, which causes massive energy draw in idle, and reduces lifetime by 3 times the estimate?

https://culperresearch.com/wp-content/uploads/2026/10/Culper...

villgax 2 days ago |

Lol why would anyone want to manage multiple vendors for the remotest non-LLM work even. Not everything is an LLM

LoganDark 2 days ago |

I hope this will be affordable one day.

zhuanxyz 2 days ago |

[flagged]