The data black hole at the center of AI

Dwarkesh Podcast · ~12 min · Listen

• Solo essay arguing that raw training data volume — not architectural cleverness — is the primary driver of frontier AI progress.
• Claims frontier models need roughly a million times more training data than a human needs to reach comparable proficiency (e.g., humans learn to drive with ~20 hours of practice).
• Notes that generating domain-specific RL training data (contractor-written examples, evaluations, reasoning traces) has become a multi-billion-dollar industry unto itself.
• Open-source models now trail the frontier by only about four months, suggesting data access — not secret algorithms — is the main gating factor on catch-up speed.
• Takeaway: labs can still profitably automate white-collar work despite the human/AI sample-efficiency gap by throwing compute at the problem, but closing that gap may be required for further progress toward human-like intelligence.