Technical notes: data source, aggregation and limitations
Data source
A year of production traffic from Chutes, a public LLM inference platform, released by researchers from Harvard, the University of Chicago and Chutes with the paper A Year in LLM Serving: Workload Evolution, Caching and Load-Balancing (data and code: github.com/HarvardMadSys/chutes_workload). The original trace is one row per request: 6.12 billion requests across 9,174 models in a 91 GB Parquet file. All credit for the dataset goes to its authors.
Aggregation
The data passes through two exact rollups. Neither one samples, so the totals match the original trace.
- Hour × model × function (1.34 million rows): for each hour, model and API function, the number of requests, plus the sum and the count of recorded values for input tokens, output tokens and time to first token. Models outside the 200 busiest are grouped together.
- Day × hour × family × model type × call type × streaming (396,205 rows, the file this page loads): the same sums and counts added up again. Individual models are left out (some are anonymized ids), and the families outside the top five are grouped as Other.
Every average on this page is a ratio of sums: the sum of a measure divided by the number of requests that recorded it, for example average time to first token = sum(ttft_seconds) / sum(ttft_n). That keeps averages correct under any filter. Averaging the rows themselves would give an hour with one request the same weight as an hour with two million.
SELECT DATE '2025-04-11' + dnum AS date, hod AS hour, family, model_type, call_type, streaming,
sum(reqs) AS requests,
sum(sum_it) AS input_tokens, sum(n_it) AS input_n,
sum(sum_ot) AS output_tokens, sum(n_ot) AS output_n,
sum(sum_ttft) AS ttft_seconds, sum(n_ttft) AS ttft_n
FROM chutes.parquet GROUP BY ALL
The charts query this file in your browser with DuckDB-WASM and are drawn with D3. Mosaic links them, so every filter you set re-runs each chart's SQL against the same data. The family-month and month-hour reels keep the whole period and dim months outside the chosen dates; the hour reel and the numbers in the top bar follow every filter. A family filter applies everywhere.
Reading the charts
- Three reels of scenes. In each reel one set of marks (a family's month, a month's hour, an hour) moves from chart to chart, so you can follow every mark. Each chart is complete on its own: stacked areas and bars start at zero, the 3D boxes have three data axes (month, family, requests) with height from zero and values on hover, heat cells print their value, radial bars grow from the inner ring.
- Heat-cell colors run from zero to the slowest family-month in view; the legend gives the scale.
- Months marked * are partial (the trace starts on 11 April 2025 and ends on 12 April 2026).
Definitions
- Request: one API call, of any kind (chat, completion, embedding, health check…).
- Model family: the model's lineage, inferred from its name. Fine-tunes count with their base model. Other covers 9 smaller families, unnamed top models, and the long tail outside the 200 busiest models.
- Model type: chat / instruct, reasoning, coding or vision, inferred from the model name. Unclassified means the name gives no signal.
- Time to first token: seconds from the request until the first output token arrived.
Limitations
- Dates are reconstructed. The released file numbers days from 0 and has no calendar dates. Day 0 is placed on 11 April 2025, following the span the paper states (11 April 2025 to 12 April 2026). Model launches agree: 18 of 20 well-known models first appear on or after their release day.
- Hours are on the trace's own clock. They are not a known timezone, so the hour-of-day chart shows the shape of the day, not local business hours.
- Time to first token exists only for streaming calls, 49% of requests. A non-streaming call returns everything at once. The latency chart and KPI therefore describe streaming traffic only, and the streaming share itself changed over the year: about 65% in September 2025, 36% from November onward.
- Token counts are missing for 7.5% of requests. Averages divide only by requests that recorded a count, so the gaps don't pull them down.
- Latency is not adjusted for model mix. Families differ widely (Gemma averages 0.9 s, GLM 3.1 s), so a change over time can reflect which models were used as much as how fast the platform was.
- Outage. 13 January 2026 recorded 1.6 million requests against about 15 million on a typical day then, consistent with a service interruption of about 30 hours.
- The first and last days are partial. Family and type labels come from model names, not from the trace itself.
Source: Chutes LLM serving trace, 11 Apr 2025 – 12 Apr 2026 (arXiv:2608.13573). Averages are per request that recorded the value. Built with DuckDB-WASM, Mosaic, D3 and d3-3d; icons from Tabler.