Hackernews posts about TensorRT-LLM
- Qwen2-7B-Instruct with TensorRT-LLM: consistently high tokens/SEC (www.inferless.com)
- Tuning TensorRT-LLM for Optimal Serving (www.bentoml.com)
- NVIDIA introduces TensorRT-LLM for accelerating LLM inference on H100/A100 GPUs (developer.nvidia.com)
- Deploy Gemma 7B with TensorRT-LLM and achieve > 500 tok/s (docs.mystic.ai)
- Optimizing Inference on LLMs with NVIDIA TensorRT-LLM (developer.nvidia.com)
- Faster Mixtral inference with TensorRT-LLM and quantization (www.baseten.co)
- Nvidia Releases TensorRT-LLM (github.com)
- Benchmarking Nvidia TensorRT-LLM (jan.ai)
- Turbocharging Meta Llama 3 Performance with Nvidia TensorRT-LLM and Triton (developer.nvidia.com)
- LLMs up to 4x Faster With Latest NVIDIA Drivers on Windows (blogs.nvidia.com)
- Nvidia Hopper Leaps Ahead in Generative AI at MLPerf (blogs.nvidia.com)
- Show HN: Automatically Build Nvidia TRT-LLM Engines (www.baseten.co)