Skip to content

Sam's News — ai — 2026-10-09

AI

7 Chutes and Harvard Release 6.12 Billion LLM Request Dataset

Chutes AI and Harvard researchers published a year-long production trace of 6.12 billion LLM inference requests from April 2025 to April 2026, covering 9,174 models served to 314,970 users. The 91 GB dataset excludes prompts and outputs for privacy, offering the AI infrastructure research community unprecedented scale insights into how language models are actually served at production scale.

  • Dataset contains 6.12 billion LLM inference requests from April 11, 2025 through April 12, 2026
  • Covers 9,174 distinct models served across 314,970 users with 35.8 trillion input tokens processed
  • Stripped of prompts, responses, and tool calls; retains infrastructure metadata (timestamps, token counts, cache hits, latency)
  • 91 GB Parquet file hosted on Harvard servers with companion arXiv paper
  • Three orders of magnitude larger than typical public LLM datasets, which usually number in the millions of interactions

Sources: Shattered (via ibl.ai review) AI Web Searched