Hacker news

  • Top
  • New
  • Past
  • Ask
  • Show
  • Jobs

Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium (https://github.com)

54 points by sanchitmonga22 3 days ago | 8 comments | View on ycombinator

coredog64 3 days ago |

Good on these folks for doing something useful with Trainium. I'm always surprised that the Tranium ecosystem is as poor as it is. Amazon's market cap is nearly $3T and they can't throw a couple L6s and a modest token budget at this? Making Trainium work as seamlessly as possible as an Nvidia substitute is a clear flywheel play.

bookernath 3 days ago |

Tokens per minute is a strange metric, I usually see tok/s. A per-second figure would be more accessible in comparing this to alternatives.

undefined 3 days ago |

undefined

kevo1ution 3 days ago |

curious how the token/$ on trainuim compare to nvidia chips and amd. is the future "chipless" inference?

badmonster0 3 days ago |

how does this compare with sglang

aman179102 about 18 hours ago |

[flagged]

flipflowdev 1 day ago |

[dead]