Compute verification

Inspection Sampling

How many chips must inspectors check?

An inventory lists each chip and where it should be. Inspectors check a random sample of the entries: is each chip there, and is it the chip listed?

A hypothetical inventory · change any number

Of 7,200 chips on the list, at least 25 are missing or misidentified, and the host keeps 10 off limits. Inspectors want a 90% chance that a random pick includes a missing or misidentified chip.

Inspectors must check 1,023 chips picked at random from the 7,190 they can reach.

Off limits: chips the host withholds from inspection, as this page models it. In the worst case 10 of the 25 are among them, leaving at least 15 within the inspectors’ reach. How a host can limit an inspection

The chance counts chips on the list: an unlisted chip can never be picked, and a picked chip still has to be recognised.

The chance, by chips checked. The axis stops at 3,000 of the 7,190; past it the worst-case chance stays above 99.9%.

10 off limits, worst case
None off limits
Shaded: the chance lost to the 10 chips off limits

Check 1,023 chips and the chance is 90.0%, the smallest sample that reaches 90%.

How the chance is computed

Chips are picked uniformly at random without replacement, and each chance is an exact hypergeometric probability. The calculation takes the stated number of missing or misidentified chips as a minimum and assumes the worst case: as many of them as the off-limits count allows are among the chips kept off limits, which inspectors cannot pick. The chart’s axis stops where the worst-case chance passes 99.9%, or further out when the smallest sample or a typed sample needs the room.

The question, how many chips to pick for a chosen chance of catching one, is Shavit’s (2023, §3.2 and Appendix B). His Equation 1 treats each pick as independent, which asks 1,103 chips here; picking without replacement needs 1,023. Chips kept off limits are this page’s addition.

The check this page models: Scher and Thiergart (2024)

In the first, low-tech, approach to tracking AI chips, the following components are necessary: countries self-report the location of their data centers that house AI chips, identify and count AI chips in those data centers, and keep an updated registry of this information; reported centers are physically inspected to ensure the presence of a random sample of the expected chips.

Mechanisms to Verify International Agreements About AI Development, p. 15 · primary

The cost of chips off limits

With 10 chips off limits inspectors need 1,023, where 633 would do with none; with 25 off limits no sample can include a discrepancy.

The smallest sample for a 90% chance at each count off limits, in the worst case.

Your setting: 10 off limits
Other counts

The smallest sample at each count off limits (26)

Chips off limits	Smallest sample
Chips off limits	Smallest sample
Chips off limits	Smallest sample
0	633
1	658
2	685
3	715
4	747
5	782
6	821
7	863
8	911
9	963
10	1,023
11	1,090
12	1,166
13	1,254
14	1,357
15	1,477
16	1,621
17	1,796
18	2,013
19	2,288
20	2,649
21	3,142
22	3,846
23	4,908
24	6,459
25 or more	No sample is enough

Who can stop a challenge inspection

Under the Machine Intelligence Research Institute’s draft, a challenge inspection requires approval by consensus of the Executive Council (X.3(b)), initially the US and China (III.3(a)); on Scrutica’s reading, either could withhold it, including for a site on its own territory. Under the Chemical Weapons Convention, a three-quarter majority of all Council members can stop one (IX.17).

The draft’s first Executive Council (III.3(a))

The inspected and requesting Parties both approve the inspectors (X.3(d)); a Party’s territory sees at most 20 challenge inspections a year, a limit the Council can change by consensus (X.3(e)).

The Convention’s Executive Council, all members (IX.17)

Preparations begin once the request meets the Convention’s formal requirements (IX.14); the Council may stop them if it considers the request frivolous, abusive or clearly beyond the Convention’s scope (IX.17).

Deadlines

The draft: 24 hours from the Council’s call, for the host to grant access; meanwhile the site may be watched, and people or vehicles leaving it inspected (X.3(c)).

The Convention: 12 hours from the request, for the Council to stop it; otherwise it proceeds, and the Council’s deliberations cannot delay it (IX.16–17).

Who receives the findings

The draft: The technical body gives the US and China all it receives or generates, inspection results included (X.4(b)); inspectors pass the Council only what is pertinent to the agreement (X.3(f)).

The Convention: The final report, with the inspection team’s assessment of the access and cooperation granted, goes to every State Party (IX.21).

How a host can limit an inspection

Under the Convention’s managed access the inspected state negotiates how far inspectors go and what they do, and may shroud sensitive equipment such as computer systems (Verification Annex X.47–48). It may also use random selective access, in which inspectors select a given percentage or number of buildings of their choice, so which buildings go unchecked is theirs to choose (Annex X.48(f)). Both exist to protect what is unrelated to chemical weapons (Annex X.46, 48); this page models a host that chooses which chips to keep off limits, as shrouding lets it choose what equipment to cover (Annex X.48(c)).

The 14 provisions quoted, from two documents

An International Agreement to Prevent the Premature Creation of Artificial SuperintelligenceScher, Abecassis, Barnett and Abeyta, v3, 8 May 2026 (arXiv; v1 13 Nov 2025) · primary

The Executive Council initially consists of the United States of America and the People’s Republic of China.Article III, paragraph 3(a)
A challenge inspection requires approval by consensus of the Executive Council.Article X, paragraph 3(b)
Access to a suspected site must be granted by the nation in which the site is present within 24 hours of the Executive Council calling for a challenge inspection. During this time, the site may be surveilled, and any people or vehicles leaving the site may be inspected by officials from a Party.Article X, paragraph 3(c)
The challenge inspection will be conducted by a team of inspectors who are approved by both the Party being inspected and the Party that requested the inspection. The CTB is responsible for working with Parties to maintain lists of approved inspectors for this purpose.Article X, paragraph 3(d)
Challenge inspections may be conducted in a given Party’s territory at most 20 times per year, and this limit can be changed by consensus of the Executive Council.Article X, paragraph 3(e)
Inspectors will take absolute care to protect the sensitive information of the inspected Party, passing along to the Executive Council only what information is pertinent to the Agreement.Article X, paragraph 3(f)
The Coalition Technical Body provides all information it receives or generates to the U.S. and the PRC to support their independent verification efforts. This includes but is not limited to: declarations, reports, monitoring data, inspection results, registers, and any other information relevant to compliance with this Agreement. The CTB does not filter or withhold information from the U.S. and PRC.Article X, paragraph 4(b)

Chemical Weapons Convention, Article IX and Verification Annex Part XOrganisation for the Prohibition of Chemical Weapons · primary

The Director-General shall immediately ascertain that the inspection request meets the requirements specified in Part X, paragraph 4, of the Verification Annex, and, if necessary, assist the requesting State Party in filing the inspection request accordingly. When the inspection request fulfils the requirements, preparations for the challenge inspection shall begin.Article IX, paragraph 14
After having received the inspection request, the Executive Council shall take cognizance of the Director-General’s actions on the request and shall keep the case under its consideration throughout the inspection procedure. However, its deliberations shall not delay the inspection process.Article IX, paragraph 16
The Executive Council may, not later than 12 hours after having received the inspection request, decide by a three-quarter majority of all its members against carrying out the challenge inspection, if it considers the inspection request to be frivolous, abusive or clearly beyond the scope of this Convention as described in paragraph 8. Neither the requesting nor the inspected State Party shall participate in such a decision. If the Executive Council decides against the challenge inspection, preparations shall be stopped, no further action on the inspection request shall be taken, and the States Parties concerned shall be informed accordingly.Article IX, paragraph 17
The final report shall contain the factual findings as well as an assessment by the inspection team of the degree and nature of access and cooperation granted for the satisfactory implementation of the challenge inspection. The Director-General shall promptly transmit the final report of the inspection team to the requesting State Party, to the inspected State Party, to the Executive Council and to all other States Parties.Article IX, paragraph 21
The inspection team shall take into consideration suggested modifications of the inspection plan and proposals which may be made by the inspected State Party, at whatever stage of the inspection including the pre-inspection briefing, to ensure that sensitive equipment, information or areas, not related to chemical weapons, are protected.Verification Annex, Part X, paragraph 46
The inspected State Party shall designate the perimeter entry/exit points to be used for access. The inspection team and the inspected State Party shall negotiate: the extent of access to any particular place or places within the final and requested perimeters as provided in paragraph 48; the particular inspection activities, including sampling, to be conducted by the inspection team; the performance of particular activities by the inspected State Party; and the provision of particular information by the inspected State Party.Verification Annex, Part X, paragraph 47
In conformity with the relevant provisions in the Confidentiality Annex the inspected State Party shall have the right to take measures to protect sensitive installations and prevent disclosure of confidential information and data not related to chemical weapons. Such measures may include, inter alia: (a) Removal of sensitive papers from office spaces; […] (c) Shrouding of sensitive pieces of equipment, such as computer or electronic systems; […] (f) Using random selective access techniques whereby the inspectors are requested to select a given percentage or number of buildings of their choice to inspect; the same principle can apply to the interior and content of sensitive buildings; […]Verification Annex, Part X, paragraph 48, its opening and items (a), (c) and (f)

Proposals that pick chips at random

Fist and Grunewald would pick chips from an ownership registry to confirm their owners still hold them; Shavit would pick them to read what they logged during training.

Fist (Center for a New American Security) and Grunewald (Institute for AI Policy and Strategy) · 2023

Checked on each picked chip: An ID match and evidence of tampering; if chips are mailed in, inspectors also check the package receipts to confirm each chip came from its expected origin (annex items 2–3).
The method depends on: A registry of chip ownership, updated when chips are resold, destroyed or lost (annex item 1).
A picked chip can show: A failed check, visible tampering or a chip not sent in time, after which the Bureau of Industry and Security (BIS) may choose to investigate the owner or any previous owner, check the owner’s facilities, or issue a temporary denial order (annex item 5).

Shavit (Harvard University) · 2023

Checked on each picked chip: A serial-number match, the hashes of the model’s weights in its logs, and its logging hardware for tampering (§3.1).
The method depends on: Chips that log hashes of the weights during training, and the operator’s reported transcripts to check them against (§3.1, Appendix B).
A picked chip can show: A violation, only if it logged such a hash during the prohibited run, so the odds also depend on how often chips log one and how often inspectors sample (§3.2, Appendix B).

The 7 passages quoted, from two documents

Preventing AI Chip Smuggling to ChinaFist and Grunewald, 24 Oct 2023 · primary

Deploy a centralized registry of chip ownership, with reporting requirements when chips are resold, destroyed, or lost.Technical annex, Program steps, item 1
Every 2 weeks, BIS selects a random set of IDs from the registry and notifies the chips’ owners that they must submit to a short-notice inspection. Inspections could take several forms. One cost-minimizing approach is short-notice mail-in of chips to regional U.S. Commercial Service/Export Control Offices.Technical annex, Program steps, item 2
BIS personnel inspect chips for ID matches and evidence of tampering and check package receipts to ensure the chip came from the expected origin (if a mail-in program is used).Technical annex, Program steps, item 3
If a chip owner fails to send a chip that it is supposed to own within the allowable time frame, or if a chip has been visibly tampered with or fails to pass inspection, BIS can choose to take further actions: 1. Investigations of the chip owner, and/or any of the chip’s previous owners. 2. End-user checks at the owner’s facilities. 3. Issuing a temporary denial order, if the chip owner may intend to purchase more controlled chips.Technical annex, Program steps, item 5

What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute MonitoringShavit, 20 Mar 2023 (v2 30 May 2023) · primary

A chip inspection involves several steps. First, the Verifier confirms the chip’s serial number (often burned into chips and accessible both digitally and physically) matches the serial number they requested from the Prover. This confirms that this is the correct randomly-sampled chip, and also that the Prover still owns the chip and has not secretly resold it to an untrusted party. Second, the Verifier checks the on-chip logs, from which they extract the weight snapshot hashes which they will check against the Prover’s reported training transcripts (Section 5). Finally, the Verifier checks that the chip’s logging mechanisms […] have not been physically tampered with.§3.1, p. 7
Sections 4, 5, and 6 will together guarantee that if the Verifier observes even a single ML chip that logged a weight snapshot hash during a rules-violating training run, then the Verifier can use that weight snapshot to prove that the Prover violated the rules. For the Verifier to make sure they’d observe at least one such snapshot from a large training run with high probability (or alternatively, catch a chip that’d been tampered with), they would need to inspect at least s of the Prover’s C chips every T_m days.§3.2, p. 7
It is not enough that a chip involved in the training run be sampled by the Verifier; the chip needs to have also logged a weight-snapshot from this particular training run […]. The probability of a chip logging a weight-snapshot is uniform over time (Section 4).Appendix B, p. 18