Tuesday, November 4, 2025
No menu items!
HomeTechnologyA profile of nonprofit Common Crawl, which has scraped billions of webpages...

A profile of nonprofit Common Crawl, which has scraped billions of webpages since 2013, including paywalled ones, to build an archive used by OpenAI and others (Alex Reisner/The Atlantic)

Featured Podcasts


Invest Like the Best:


Luca Ferrari – Building Bending Spoons

The leading destination to learn about business and investing. We do this by showcasing exceptional talent and ideas.


Subscribe to Invest Like the Best.


Grit:


Rebuilding Front for the AI Era | CEO Dan O’Connell

Grit explores what it takes to create, build and scale world-class organizations.


Subscribe to Grit.


Lenny’s Podcast:


The woman behind Canva shares how she built a $42B company from nothing | Melanie Perkins

Interviews with world-class product leaders and growth experts to uncover actionable advice to help you build, launch, and grow your own product.


Subscribe to Lenny’s Podcast.


The Talk Show With John Gruber:


‘Meat Bags’, With Brian Mueller

The director’s commentary track for Daring Fireball. Long digressions on Apple, technology, design, movies, and more.


Subscribe to The Talk Show With John Gruber.


BG2 Pod:


All things AI w @altcap @sama & @satyanadella. A Halloween Special. 🎃🔥BG2 w/ Brad Gerstner

Open-source podcast on all things tech, markets, investing, and capitalism, hosted by Brad Gerstner.


Subscribe to BG2 Pod.

RELATED ARTICLES

Most Popular

Recent Comments