Hackernews posts about TensorRT-LLM

1. Benchmarking LLM Inference Back Ends: VLLM, LMDeploy, MLC-LLM, TensorRT-LLM, TGI (www.bentoml.com)
   15 points by chaoyu almost 2 years ago | 1 comments
2. Show HN: LLGTRT: TensorRT-LLM+Rust server w/ OpenAI-compat and Structured Output (github.com)
   6 points by mmoskal over 1 year ago | discuss
3. TensorRT-LLM runtime now open-source (github.com)
   4 points by mmoskal about 1 year ago | 1 comments
4. Faster Open-Source Llama3 Serving with SGLang Runtime (vs. TensorRT-LLM, VLLM) (lmsys.org)
   4 points by yvbbrjdr almost 2 years ago | discuss
5. TensorRT LLM (github.com)
   2 points by kristianpaul about 1 month ago | discuss
6. 5x Faster Time to First Token with Nvidia TensorRT-LLM KV Cache Early Reuse (developer.nvidia.com)
   2 points by sandwichsphinx over 1 year ago | discuss
7. TensorRT-LLM on K8s
   2 points by hommes-r almost 2 years ago | discuss
8. Comparing LLM Optimization Tools: VLLM, LMDeploy, MLC-LLM, TensorRT-LLM, and TGI (www.bentoml.com)
   2 points by bbzjk7 almost 2 years ago | discuss
9. Comparing GenAI Inference Engines: TensorRT-LLM, VLLM, HF TGI, and LMDeploy
   1 points by juliensalinas about 1 year ago | 1 comments
10. Qwen2-7B-Instruct with TensorRT-LLM: consistently high tokens/SEC (www.inferless.com)
    1 points by agcat over 1 year ago | 1 comments
11. Automating Inference Optimizations with NVIDIA TensorRT LLM AutoDeploy (developer.nvidia.com)
    1 points by matt_d 3 months ago | discuss
12. Tuning TensorRT-LLM for Optimal Serving (www.bentoml.com)
    1 points by djhu9 over 1 year ago | discuss
13. Zero-Shot Text Classification on a low-end CPU-only machine?
    8 points by backend-dev-33 over 1 year ago | 12 comments
14. Show HN: Sweep, AI autocomplete for JetBrains that rewrites code (sweep.dev)
    5 points by williamzeng0 8 months ago | discuss
15. Show HN: LLMOne – Deploy LLMs from bare metal to production in hours (github.com)
    5 points by pescn 11 months ago | discuss
16. Show HN: Small hardware box that runs local LLMs and exposes an OpenAI API (axis-one-psi.vercel.app)
    2 points by mjupp1 6 months ago | discuss
17. Defining a Metric for LLM Responsiveness
    1 points by Thomas-Mecattaf over 1 year ago | 1 comments
18. Show HN: Automatically Build Nvidia TRT-LLM Engines (www.baseten.co)
    2 points by mikejulietbravo almost 2 years ago | discuss