Hackernews posts about Common Crawl

1. PDF Trends 2026 Q2: Analysis of 20.6M PDFs from Common Crawl (pdf4wcag.com)
   3 points by smartdev01 17 days ago | 5 comments
2. Show HN: RankParse – An SEO and backlink data API built from Common Crawl (rankparse.com)
   1 points by abhibavishi 13 days ago | discuss
3. US publishers tell Common Crawl to stop scraping and delete archive (pressgazette.co.uk)
   30 points by thm 2 months ago | 10 comments
4. Common Crawl maintains a free, open repository of web crawl data (commoncrawl.org)
   27 points by doener over 1 year ago | 1 comments
5. Ask HN: Is Common Crawl used exhaustively by any search engine?
   8 points by n1xis10t 10 months ago | 1 comments
6. Show HN: Help improve language coverage in Common Crawl
   8 points by ccgreg about 1 year ago | discuss
7. Ask HN: Is there a service that offers Common Crawl as an API?
   7 points by georgehill over 1 year ago | 3 comments
8. Show HN: Infini-News – 1.36B news articles from Common Crawl, queryable in ms (cs2.uni-graz.at)
   7 points by ruggsea about 2 months ago | 1 comments
9. Common Crawl (commoncrawl.org)
   7 points by skibz about 1 month ago | discuss
10. Cybersecurity Data Extraction from Common Crawl (arxiv.org)
    5 points by PaulHoule 6 months ago | discuss
11. A Change to Common Crawl Dataset Size Reporting (commoncrawl.org)
    3 points by ccgreg 5 months ago | 1 comments
12. Cc-downloader: command-line tool for downloading Common Crawl data via HTTPS (github.com)
    2 points by simonpure over 1 year ago | discuss
13. Publishers Demand Accountability from Common Crawl over Unauthorized Use (www.newsmediaalliance.org)
    1 points by thm 4 months ago | 2 comments
14. Crawlgraph – Backlink lookup using Common Crawl ($99 lifetime) (crawlgraph.com)
    1 points by pucilpet 3 months ago | discuss
15. Publishers Tell Common Crawl to Stop Unauthorized Scraping (www.mediapost.com)
    1 points by jaredwiener 3 months ago | discuss
16. I made Common Crawl's 4.4B edges queryable for backlink lookups (crawlgraph.com)
    1 points by pucilpet 4 months ago | discuss
17. Show HN: I built a 50 site sampler from CommonCrawl refreshing every 30 minutes (randcrawl.com)
    1 points by whothatcodeguy 7 months ago | discuss
18. 200M unique domains extracted from Common Crawl (zenodo.org)
    1 points by networkcat 7 months ago | discuss
19. Show HN: Randomly discovered websites from the open internet every 60 minutes (randcrawl.com)
    4 points by whothatcodeguy 7 months ago | 1 comments
20. Show HN: Rhiza – easily create shortcuts and add entries to PATH (github.com)
    2 points by skardy over 1 year ago | discuss
21. Humanties Last War
    1 points by vlan121 10 months ago | 1 comments
22. Show HN: hot or not for .ai websites (ratemyaisite.com)
    1 points by prolly97 5 months ago | discuss
23. The Company Quietly Funneling Paywalled Articles to AI Developers (www.theatlantic.com)
    33 points by breve 10 months ago | 16 comments
24. The Nonprofit Feeding the Internet to AI Companies (www.theatlantic.com)
    10 points by ForHackernews 10 months ago | 5 comments
25. The Nonprofit Doing the AI Industry's Dirty Work (www.theatlantic.com)
    9 points by kgwgk 10 months ago | 2 comments
26. The Company Funneling Paywalled Articles to AI Developers (www.theatlantic.com)
    3 points by CaptainZapp 10 months ago | discuss
27. IPv6 Adoption Across the TopK Web Hosts (commoncrawl.org)
    2 points by miyuru 5 months ago | discuss
28. Crawler operators, please stop destroying the commons (lunnova.dev)
    7 points by xena over 1 year ago | discuss
29. Show HN: CrawlerCheck v1.5 – Operators Directory and 25 new AI crawlers (crawlercheck.com)
    1 points by bogozi 6 months ago | discuss
30. Show HN: Free tool to find RSS feeds, even if not linked on the page
    152 points by domysee almost 2 years ago | 48 comments
31. Show HN: Wispbit - Linter for AI coding agents (wispbit.com)
    31 points by dearilos 10 months ago | 14 comments
32. Quick but powerful research for AI agents with data scrapping and selenium
    8 points by alexvomwald over 1 year ago | 6 comments