**Finding.** Challenging existing beliefs on data quality and LLMs, models trained on adequately filtered and deduplicated web data _alone_ can match the performance of models trained on curated data.