Training Economics

Training-cost decomposition by FLOP budget × hardware × procurement model, evaluated against substrate-priced cloud rates (127 provider×hardware×region rows from compute pricing) and a 3-year on-prem amortization. The methodology trail is exposed inline next to the calculator; cross-validation pulls from frontier_model_runs (34 rows · 24 unique models · 13 labs · 10 multi-source pairings; methodology) and shows the per-row gap between the calculator and each lab’s primary disclosure or Epoch’s estimate. The EU AI Act 10²⁵ FLOP trigger sits at the calculator’s default budget; BIS designations bind on the upper FLOP curve.

Training an AI model has three priced dimensions: FLOP budget, hardware mix, procurement model. The calculator below estimates a run’s cost from cloud pricing plus 3-year on-prem amortization; the cross-validation table reads from frontier_model_runs (34 disclosure records across 13labs) and surfaces per-row gaps against each lab’s primary disclosure or Epoch AI’s central estimate. Frontier-cost trajectories and the cost of replicating a prior generation’s capability live in the deep-dive chart at the bottom.

127

Cloud-price rows

live from compute_pricing as of 2026-06-11

Source: getCloudPricing() — provider × hardware × region × pricing-tier

GPU generations modelled

+ 2 placeholders (R100 Rubin, Maia 200)

Source: GPU_MODELS — Epoch ML Hardware reference, dense BF16 spec

Cross-validated runs

24 models in frontier_model_runs · 10 multi-source pairings

Source: frontier_model_runs substrate (Epoch ≥1e25 + lab-primary disclosures)

23 T1+T2

Authority distribution

of 34 rows · T1 = 2 · T2 = 21 · T4 = 11

Source: Per-column tier on training_compute_flop; 50% bar met at 68%

Cross-validation · disclosed runs

Cross-validation against publicly-disclosed runs

Each row reproduces a published cost figure inside the calculator above; Δ is the reproduction-versus-disclosed gap, with the ±20% band marking the coarse external-validity threshold. The table reads from frontier_model_runs (multi-source per cell, best-tier-available picked per column); where both Epoch and the lab disclose, the higher-authority figure leads with the source labelled.

Model	FLOP	Hardware	Reported $	Source	Scrutica calc	Δ
Grok 4 xAI · Jul 2025	5.0e+26 T2	H100 SXM5 × 200,000 (Colossus full)	$387.8M	Epoch AI (T2)	$527.2M $252.2M–$1.1B	+36%
GPT-4.5 OpenAI · Feb 2025	3.8e+26 T2	undisclosed	$340.0M	Epoch AI (T2)	$400.7M $191.7M–$813.1M	+18%
Grok 3 xAI · Feb 2025	3.5e+26 T2	H100 SXM5 × 80,000 (Colossus subset)	$217.8M	Epoch AI (T2)	$369.0M $176.5M–$748.9M	+69%
Llama 4 Behemoth (preview) Meta Platforms, Inc. · Apr 2025	5.2e+25 T2	H100 SXM5 × 32,000 (FP8)	$44.6M	Epoch AI (T2) + 1 paired source	$54.7M $26.1M–$110.9M	+23%
Gemini 1.0 Ultra Google Deepmind · Dec 2023	5.0e+25 T2	TPUv4 × ~57,000 (Epoch derivation)	$30.7M	Epoch AI (T2) + 1 paired source	$52.7M $25.2M–$107.0M	+72%
Llama 3.1-405B Meta Platforms, Inc. · Jul 2024	3.8e+25 T1	H100 SXM5 80GB × 16,384	$52.9M	Epoch AI (T2) + 1 paired source	$40.1M $19.2M–$81.3M	-24%
Grok-2 xAI · Aug 2024	3.0e+25 T2	H100 SXM5 (count undisclosed)	$31.6M	Epoch AI (T2)	$31.2M $14.9M–$63.3M	-1%
Claude 3.5 Sonnet Anthropic · Jun 2024	2.7e+25 T2	undisclosed	$25.9M	Epoch AI (T2) + 1 paired source	$28.5M $13.6M–$57.8M	+10%
GPT-4 (Mar 2023) OpenAI · Mar 2023	2.1e+25 T2	A100 SXM4 × ~25,000	$37.3M	Epoch AI (T2) + 1 paired source	$37.1M $17.7M–$75.3M	-1%
Nemotron-4 340B Nvidia · Jun 2024	1.8e+25 T2	H100 80GB SXM5 × 6,144	$21.3M	Epoch AI (T2) + 1 paired source	$19.0M $9.1M–$38.5M	-11%
Mistral Large Mistral · Feb 2024	1.1e+25 T2	H100 SXM5 (count undisclosed)	$14.1M	Epoch AI (T2)	$11.8M $5.6M–$24.0M	-16%
Inflection-2 Inflection Ai · Nov 2023	1.0e+25 T1	H100 × 5,000	$13.5M	Epoch AI (T2) + 1 paired source	$10.5M $5.0M–$21.4M	-22%

Grok 4. Reproduction: On-prem · 3-year amortized at MFU 0.40 · interconnect 0.85. xAI Grok 4 announcement attributes 200K-H100 cluster. Calculator runs against Epoch central 5.0e26.

GPT-4.5. Reproduction: On-prem · 3-year amortized at MFU 0.40 · interconnect 0.85. OpenAI does not disclose hardware. Calculator runs against Epoch central 3.8e26.

Grok 3. Reproduction: On-prem · 3-year amortized at MFU 0.40 · interconnect 0.85. xAI Memphis Colossus is on-prem; Grok 3 attributed to a Colossus subset. On-prem 3-year amortization is the appropriate reproduction baseline.

Llama 4 Behemoth (preview). Reproduction: On-prem · 3-year amortized at MFU 0.40 · interconnect 0.85. Meta blog discloses 32K-GPU cluster + FP8 training + 390 TFLOPs/GPU throughput; Epoch attributes H100 SXM5. On-prem 3-year amortization.

Gemini 1.0 Ultra. Reproduction: On-prem · 3-year amortized at MFU 0.40 · interconnect 0.85. Google paper discloses TPUv4 hardware class only; chip count not stated. Epoch derives ~57,000 TPUv4. Calculator runs on H100-equivalent throughput as proxy — figure indicative, not directly comparable.

Llama 3.1-405B. Reproduction: On-prem · 3-year amortized at MFU 0.40 · interconnect 0.85. Llama 3 paper §3.3.1 discloses 3.8×10²⁵ FLOPs and 16,384 H100 SXM5; Meta runs internal cluster (no retail rate published). On-prem 3-year amortization is the right baseline.

Grok-2. Reproduction: On-prem · 3-year amortized at MFU 0.40 · interconnect 0.85. xAI Memphis Colossus pre-Grok-3 scaling. Calculator runs against Epoch central 3.0e25.

Claude 3.5 Sonnet. Reproduction: On-prem · 3-year amortized at MFU 0.40 · interconnect 0.85. Anthropic does not disclose compute on system cards. Anthropic compute runs on AWS Trainium and GCP TPU under multi-year contracts at heavily-discounted rates; on-prem 3-year amortization is the closest defensible baseline.

GPT-4 (Mar 2023). Reproduction: On-prem · 3-year amortized at MFU 0.40 · interconnect 0.85. OpenAI Tech Report explicitly does not disclose hardware or compute. Microsoft built a dedicated supercomputer for OpenAI; the effective price is capex amortization, not retail cloud reserved. Cottier & Rahman (arXiv:2405.21015) total dev cost $90–100M.

Nemotron-4 340B. Reproduction: On-prem · 3-year amortized at MFU 0.40 · interconnect 0.85. NVIDIA paper discloses 768 DGX H100 nodes × 8 GPUs/node = 6,144 H100 80GB SXM5. FLOP not stated; calculator uses Epoch central estimate 1.8e25.

Mistral Large. Reproduction: On-prem · 3-year amortized at MFU 0.40 · interconnect 0.85. Epoch attributes H100 SXM5 to the run; Mistral does not disclose count. Calculator reproduction uses Epoch central estimate 1.1e25 against Epoch-derived $14.1M.

Inflection-2. Reproduction: On-prem · 3-year amortized at MFU 0.40 · interconnect 0.85. Inflection blog: "5,000 NVIDIA H100 GPUs in fp8 mixed precision for ~10²⁵ FLOPs". Lab disclosure of compute scale at the EU threshold.

Reading the Δ column

Δ is the calculator-vs-published gap under one procurement assumption per row. When the calculator runs lower than the disclosure, the cause is typically MFU 0.40 acting as a conservative default against an estimate that folded in R&D or assumed a higher implicit price; when it runs higher, the cause is typically a final-run-only published estimate against the cloud-pricing overhead the calculator carries. The column is a per-row audit point, not a scorecard.

Source: Source: Epoch AI, CC BY 4.0 · lab-primary rows from each lab’s canonical paper / system card / announcement (URLs in the Source column above). Substrate: frontier_model_runs.

Training Economics

Cross-validation against publicly-disclosed runs

Sensitivity — what would change the answer

Deep dive: cost vs capability for 20 historical models