Hackernews posts about MMLU

1. Show HN: LLM Benchmarking Suite (github.com)
   2 points by Dhyaneesh 25 days ago | discuss
2. ARM64 MMU: Unicorn Emulator vs. Apache NuttX RTOS (lupyuen.org)
   2 points by lupyuen 22 days ago | discuss
3. Optimizing Matrix Multiplication (coffeebeforearch.github.io)
   3 points by jxmorris12 16 days ago | discuss
4. The movie that's different every time you watch it (movieweb.com)
   45 points by mmoustafa 13 days ago | 25 comments
5. React Native ExecuTorch – Simplest way to run AI models on device in RN (www.producthunt.com)
   7 points by msluszniak1 25 days ago | discuss
6. SHOW HN:A New 34B Open Source LLM, Astonishing 78 Score in MMLU (GPT-4 MMLU:83) (github.com)
   21 points by liuxiaopai over 1 year ago | 5 comments
7. New medical LLM beats Med-PaLM-2, GPT-4 on MMLU benchmarks (huggingface.co)
   16 points by samjulien 9 months ago | 2 comments
8. RedPajama-Incite-7B-Instruct Outperforms LLaMA on MMLU (twitter.com)
   9 points by vipulved almost 2 years ago | 1 comments
9. SmartGPT: Major Benchmark Broken – 89.0% on MMLU and Exam Errors [video] (www.youtube.com)
   4 points by pps over 1 year ago | 5 comments
10. Errors in the MMLU: The Deep Learning Benchmark Is Wrong Surprisingly Often (derenrich.medium.com)
    3 points by kristianp over 1 year ago | discuss
11. Death of MMLU? Revealing the Sensitivity and Instability of LLM Leaderboards (arxiv.org)
    2 points by hishamyahya about 1 year ago | 1 comments
12. LLM Comparison/Test: 25 SOTA LLMs (Including QwQ) Through 59 MMLU-Pro CS Runs (huggingface.co)
    2 points by ororm 5 months ago | discuss
13. Multilingual MMLU Dataset from OpenAI (OpenAI/Mmmlu) (huggingface.co)
    2 points by ekojs 7 months ago | discuss
14. New Phi-3 Models (3B, 7B,14B), including 128k vision with impressive MMLU scores (twitter.com)
    2 points by brianjking 11 months ago | discuss
15. Tiny tweaks to tests like MMLU can shuffle model rankings (twitter.com)
    2 points by tosh about 1 year ago | discuss
16. Yi-34B, 76.3 on MMLU, Apache 2.0 (huggingface.co)
    2 points by Leary over 1 year ago | discuss
17. Errors in the MMLU: The Deep Learning Benchmark Is Wrong Surprisingly Often (derenrich.medium.com)
    2 points by brokensegue over 1 year ago | discuss
18. MMLU-Pro: Advanced edition of MMLU & new Leaderboard (huggingface.co)
    1 points by nopinsight 11 months ago | 1 comments
19. Gemini Benchmark – MMLU (compared with GPT-4-turbo, Mixtral) (hub.zenoml.com)
    1 points by zurfer about 1 year ago | 1 comments
20. iAsk Pro LLM achieves 93.89% on MMLU, beats GPT-4o and Claude 3.5 Sonnet (iask.ai)
    1 points by fariszr 8 months ago | discuss
21. MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Bench (huggingface.co)
    1 points by Morinee 11 months ago | discuss
22. Multitask Language Understanding (MMLU) on Helm (crfm.stanford.edu)
    1 points by tosh 12 months ago | discuss
23. OLMo 1.7–7B: A 24 point improvement on MMLU (blog.allenai.org)
    1 points by tosh about 1 year ago | discuss
24. MMLU Benchmark Broken (www.youtube.com)
    1 points by explosion-s over 1 year ago | discuss
25. Show HN: BenchFlow – run AI benchmarks as an API (github.com)
    24 points by xdotli about 1 month ago | 1 comments
26. Show HN: QwQ-32B APIs – o1 like reasoning at 1% the cost
    17 points by ozgune 3 months ago | 3 comments
27. Show HN: Open-source study to measure end user satisfaction levels with LLMs (open-llm-initiative.com)
    12 points by sparacha 8 months ago | 2 comments
28. Show HN: I built the LLM Comparison Tool I wish existed (llm-stats.com)
    7 points by JonathanChavez 4 months ago | 3 comments
29. Show HN: OptiLLMBench – Test how inference optimization tricks scale up LLMs
    2 points by codelion 2 months ago | discuss
30. Ask HN: Are LLMs getting better, how can you tell?
    1 points by ahamilton454 3 months ago | 1 comments
31. The most widely used benchmarks for evaluating LLMs
    1 points by kavaivaleri about 1 year ago | 1 comments
32. Are new LLM models trained on old benchmarks?
    1 points by rand0mwalk 10 months ago | discuss
33. Evaluation Platform for LLMs
    1 points by Dheemanthreddy about 1 year ago | discuss
34. What's Going on with the Open LLM Leaderboard? (huggingface.co)
    40 points by tim_sw almost 2 years ago | 2 comments
35. Overview of All Major LLM Benchmarks (www.confident-ai.com)
    1 points by 3d27 about 1 year ago | discuss