Hackernews posts about Llama.cpp

1. Show HN: KTransformers–236B Model and 1M Context LLM Inference on Local Machines (github.com)
   20 points by sssummer 22 days ago | 3 comments
2. Show HN: An optimizing model router with easy BYOK and serverless inference (keytrustee.org)
   1 points by CuriouslyC 11 days ago | discuss
3. Llama.cpp 30B runs with only 6GB of RAM now (github.com)
   1311 points by msoad over 1 year ago | 414 comments
4. Llama.cpp: Port of Facebook's LLaMA model in C/C++, with Apple Silicon support (github.com)
   989 points by mrtksn over 1 year ago | 284 comments
5. Llama.cpp: Full CUDA GPU Acceleration (github.com)
   728 points by gzer0 over 1 year ago | 310 comments
6. How Is LLaMa.cpp Possible? (finbarr.ca)
   685 points by birriel about 1 year ago | 227 comments
7. LLama.cpp now has a web interface (github.com)
   328 points by xal about 1 year ago | 49 comments
8. Running LLaMA 7B on a 64GB M2 MacBook Pro with Llama.cpp (til.simonwillison.net)
   225 points by marban over 1 year ago | 76 comments
9. Llama.rs – Rust port of llama.cpp for fast LLaMA inference on CPU (github.com)
   202 points by rrampage over 1 year ago | 24 comments
10. Show HN: LLaVaVision: An AI "Be My Eyes"-like web app with a llama.cpp backend (github.com)
    154 points by lxe 11 months ago | 19 comments
11. Show HN: Open-source load balancer for llama.cpp (github.com)
    151 points by mcharytoniuk 4 months ago | 20 comments
12. Why MMAP in llama.cpp hides true memory usage (twitter.com)
    136 points by flurly over 1 year ago | 98 comments
13. Llama.cpp can do 40 tok/s on M2 Max, 0% CPU usage, using all 38 GPU cores (twitter.com)
    123 points by samwillis over 1 year ago | 57 comments
14. Performance of llama.cpp on Apple Silicon A-series (github.com)
    100 points by mobilio 9 months ago | 41 comments
15. llama.cpp: Roadmap May 2023 (github.com)
    97 points by tosh over 1 year ago | 6 comments
16. Running Llama.cpp on AWS Instances (github.com)
    96 points by schappim 10 months ago | 10 comments
17. Revert for jart’s llama.cpp MMAP miracles (github.com)
    86 points by mmoustafa over 1 year ago | 86 comments
18. Llamanet: Zero-setup, zero-dependency OpenAI replacement powered by llama.cpp (github.com)
    45 points by cocktailpeanut 3 months ago | 5 comments
19. 21.2× faster than llama.cpp? plus 40% memory usage reduction (arxiv.org)
    43 points by helloericsf 3 months ago | 14 comments
20. Show HN: Llama.go – port of llama.cpp to pure Go (github.com)
    25 points by Ambix over 1 year ago | 5 comments
21. WIP Llama.cpp Vulkan Implementations (github.com)
    24 points by brucethemoose2 about 1 year ago | 1 comments
22. Show HN: Grammar Generator App for Llama.cpp (grammar.intrinsiclabs.ai)
    19 points by aduffy about 1 year ago | 6 comments
23. Gemma Is Added to Llama.cpp (github.com)
    17 points by behnamoh 7 months ago | discuss
24. LLaVA C++ server (based on llama.cpp) (github.com)
    14 points by trzy 11 months ago | 6 comments
25. Llama.cpp releases now ship with pre-built macOS binaries (twitter.com)
    12 points by schappim 6 months ago | 1 comments
26. Grok-1 Support for Llama.cpp (github.com)
    11 points by schappim 6 months ago | 2 comments
27. LLaMA-rs: a Rust port of llama.cpp for fast LLaMA inference on CPU (github.com)
    8 points by darthdeus over 1 year ago | discuss
28. Jlama (Java) outperforms llama.cpp in F32 Llama 7B Model (twitter.com)
    7 points by tjake about 1 year ago | 9 comments
29. Llama.cpp Working on Support for Llama3 (github.com)
    7 points by theolivenbaum 5 months ago | discuss
30. llama.cpp now supports StarCoder model series (github.com)
    6 points by wsxiaoys about 1 year ago | 1 comments
31. Show HN: LMQL becomes simpler and adds llama.cpp (lmql.ai)
    6 points by lbeurerkellner about 1 year ago | discuss
32. Llama.http: A Dockerized, simple HTTP API for llama.cpp (hub.docker.com)
    5 points by edfletcher_t137 about 1 year ago | 1 comments