Facilities written 2026-10
Capacity
Estimator
Estimate daily compute
Use a disclosed chip count, power capacity, or construction budget.
Default-parameters derivation (Hardware path)
10,000 H100 SXM5 GPUs at the documented defaults (interconnect efficiency 1, MFU 0.4). This derivation is fixed; the calculator below, once its scripts load, exposes every parameter as an adjustable slider.
| Step | Formula | Result |
|---|---|---|
| Peak FP16 | gpu_count × per_gpu_fp16_tflops | 9.89 × 1018 FLOP/s |
| Effective FP16 | peak × interconnect_efficiency (1) | 9.89 × 1018 FLOP/s |
| Training Throughput | effective × mfu_assumption (0.4) | 3.96 × 1018 FLOP/s |
| Daily FLOP Budget | training × 86,400 | 3.42 × 1023 FLOP/day |
At this daily budget, the EU AI Act 10²⁵ FLOP threshold is reached in approximately 29.2 days. Adjust GPU count, model, interconnect efficiency, and MFU in the interactive calculator below.
None of the 315 facilities with a GPU count and a power figure qualifies for a paired comparison
Evidence reasons for 315 withheld comparisons
Each line counts the withheld records by the first requirement they fail.
- 173: The power figure is not recorded as whole-facility power.
- 58: Facility status is announced, not operational.
- 31: NVIDIA V100 is not identified as the V100 SXM2, the only V100 the estimator supports.
- 11: No dense FP16 rate for NVIDIA GH200 has passed evidence review.
- 5: No dense FP16 rate for Google TPU v4 has passed evidence review.
- 4: AMD Instinct MI300A includes unsupported hardware.
- 4: Facility status is decommissioned, not operational.
- 2: Google TPU v3 includes unsupported hardware.
- 2: NVIDIA Tesla K20X includes unsupported hardware.
- 2: NVIDIA Tesla K40c includes unsupported hardware.
- 1: AMD Instinct MI210 includes unsupported hardware.
- 1: Google TPU v1 includes unsupported hardware.
- 1: Google TPU v2 includes unsupported hardware.
- 1: GPU inventory total 1,872 does not equal total GPU count 3,704.
- 1: GroqChip LPU v1 includes unsupported hardware.
- 1: Intel Data Center GPU Max 1550 is unsupported.
- 1: Intel Habana Gaudi2 includes unsupported hardware.
- 1: Maia 100 (M100) is unsupported.
- 1: No dense FP16 rate for Google TPU v5e has passed evidence review.
- 1: No dense FP16 rate for Google TPU v5p has passed evidence review.
- 1: No dense FP16 rate for Huawei Ascend 910B has passed evidence review.
- 1: No GPU inventory is recorded.
- 1: NVIDIA A40 PCIe includes unsupported hardware.
- 1: NVIDIA H200 141GB does not identify an H200 form factor.
- 1: NVIDIA L40 includes unsupported hardware.
- 1: NVIDIA P100 includes unsupported hardware.
- 1: NVIDIA Quadro RTX 5000 is unsupported. 2 of 2 partitions are held.
- 1: NVIDIA Tesla K40m includes unsupported hardware.
- 1: NVIDIA Tesla P100 PCIe 16GB includes unsupported hardware.
- 1: NVIDIA Tesla P100 SXM2 includes unsupported hardware.
- 1: NVIDIA Tesla V100 DGXS 32 GB is not identified as the V100 SXM2, the only V100 the estimator supports.
- 1: NVIDIA V100 is not identified as the V100 SXM2, the only V100 the estimator supports. 2 of 2 partitions are held.
- 1: PEZY-SC2 is unsupported.
Capacity Thresholds lists facilities with a GPU count or a power figure, and gives the reason for each estimate it withholds.
2 corrections · latest 26 Sep 2026
Published hardware and compute for frontier training runs
These 10 frontier-class training runs (≥ 10²⁵ FLOP) provide reference figures for interpreting an estimate. The table pairs available hardware disclosures with Epoch’s compute estimates, or a lab’s compute figure where no Epoch record is available.
| Model | Reported hardware | Training compute (FLOP) | Epoch confidence | Source provenance |
|---|---|---|---|---|
Grok 4 xAI · Jul 2025 | 200,000 GPUs (class undisclosed) via Epoch derivation | 5.0e+26 FLOP T2 | Speculative | |
Grok 3 xAI · Feb 2025 | NVIDIA H100 SXM5 80GB × 80,000 via Epoch derivation | 3.5e+26 FLOP T2 | Likely | |
Llama 4 Behemoth (preview) Meta Platforms, Inc. · Apr 2025 | 32,000 GPUs (class undisclosed) | 5.2e+25 FLOP T2 | Likely | |
Gemini 1.0 Ultra Google DeepMind · Dec 2023 | Google TPUv4 | 5.0e+25 FLOP T2 | Speculative | |
Llama 3.1-405B Meta Platforms, Inc. · Jul 2024 | NVIDIA H100 80GB × 16,384 | 3.8e+25 FLOP T2 | Confident | |
GPT-4 (Jun 2023) OpenAI · Jun 2023 | NVIDIA A100 SXM4 40 GB × 25,000 via Epoch derivation | 2.1e+25 FLOP T2 | Likely | |
GPT-4 (Mar 2023) OpenAI · Mar 2023 | NVIDIA A100 SXM4 40 GB × 25,000 via Epoch derivation | 2.1e+25 FLOP T2 | Likely | |
Nemotron-4 340B Nvidia · Jun 2024 | NVIDIA H100 80GB SXM5 × 6,144 | 1.8e+25 FLOP T2 | Confident | |
Pangu Ultra Huawei Technologies · Apr 2025 | Huawei Ascend NPUs × 8,192 | 1.1e+25 FLOP T2 | Confident | |
Inflection-2 Inflection AI · Nov 2023 | NVIDIA H100 × 5,000 | 1.0e+25 FLOP T2 | Confident | Lab disclosure Source unavailable when checked 2026-09-02 (HTTP 404). Flagged citation |
Source: Epoch AI, CC BY 4.0 · Lab-reported figures come from each lab’s paper, system card or announcement, linked in the table.
Shared assumptions
5 parameters · methodology v1.4.0
The calculator and Thresholds view use these five default parameters. Adjust them with the calculator’s sliders. Methodology version 1.4.0, dated 2026-09-26.
Interconnect EfficiencyDefault 1
Scrutica modeling assumption
Model FLOP Utilization (MFU)Default 0.4
Power Usage Effectiveness (PUE)Default 1.15
Scrutica modeling assumption
GPU Fraction of IT LoadDefault 0.49
Scrutica modeling assumption
GPU Fraction of CapexDefault 0.45
Scrutica modeling assumption
The full derivation chain, with every formula, is in the methodology chapter.
The infrastructure behind seven frontier models
Published evidence on the hardware and training locations behind these models.
DeepSeek · December 2024
DeepSeek-V3
Keys 1–7 for model · ↑↓ for evidence views · Esc for training cluster
- Training hardware
- NVIDIA H800
- Reported · Model run
- Eight GPUs per node.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
- Run allocation
- 2,048
- Reported · Model run
- GPUs in the reported training cluster.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
- Compute cost
- ~$5.576M
- Estimate · Model run
- GPU rental estimate at $2/hour; official training only.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
- Pretraining duration
- Under 2 months
- Reported · Model run
- Context extension and post-training are separate stages.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
01 / Model · DeepSeek-V3
671B parameters; 37B active per token
- Architecture
- Mixture of experts
- Reported · Model run
- Multi-head Latent Attention and DeepSeekMoE.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
- Expert routing
- 8 of 256
- Reported · Model run
- Each MoE layer also has one shared expert.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
- Pretraining data
- 14.8T tokens
- Reported · Model run
- Followed by context extension, supervised fine-tuning and reinforcement learning.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
- Cost scope
- ~$2 per H800 GPU-hour
- Estimate · Model run
- The $5.576M estimate excludes prior research and ablation experiments. It is not total programme spending.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
02 / Training cluster · DeepSeek-V3
NVLink within nodes; InfiniBand between them
- Node layout
- 8 H800 GPUs
- Reported · Model run
- NVLink and NVSwitch connect the GPUs within each node.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
- Official training
- 2.788M H800 GPU-hours
- Reported · Model run
- 2.664M pretraining; 119K context extension; 5K post-training.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
- Elapsed time
- Pretraining under 2 months
- Reported · Model run
- The report gives this bound and no exact full-training duration.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
- Run power
- —
- Not established · Model run
- No measured whole-facility power figure is established by this report.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
03 / Data centre · DeepSeek-V3
The physical training site is not identified here
- Facility
- —
- Not established · Model run
- The cited report identifies the GPU cluster without establishing a named building or location.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
- Facility operator
- —
- Not established · Model run
- No physical-site operator is established by this source.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
- Run power
- —
- Not established · Model run
- No measured whole-facility power figure is established by this report.
- DeepSeek technical report§1, Table 1; §3.1; §4.2
DeepSeek-V3 · Training cluster
Continue
Thresholds presents reported facility evidence and the requirements for threshold estimates. Its reverse calculator solves for a hypothetical chip count or duration.