Standards Body · Foundation paper, public edition · Released July 17, 2026
Canonical record: https://standardsbody.ai/library/foundation-paper/independent-expert-review/
Standards Body is an independent research and institutional-design project. It is not currently a regulator, accreditation body, certification body, or governmental authority. This document is research; it is not an adopted standard.
Series: Foundations for Frontier AI Evaluation Infrastructure
Version: 1.0
Status: Canonical working white paper
Project: Standards Body
Primary domain: standardsbody.ai
Core line: Foundations for Frontier AI
Research basis reviewed through: July 16, 2026
Document owner: Standards Body
Review cycle: Annual review, with event-triggered revision after material changes in evaluator access, review practice, institutional governance, or frontier AI capabilities
This paper defines the Standards Body position on independent expert review for frontier artificial intelligence.
It is intended to serve as:
This paper is not an accreditation standard for evaluator organizations. That subject belongs primarily in FOUNDATION_05_THIRD_PARTY_AUDITOR_ECOSYSTEM.md and EVALUATOR_ACCREDITATION_FRAMEWORK.md.
This paper is not a claim that external reviewers are inherently correct, neutral, or free from incentives.
It establishes the conditions under which expert review can add credible, decision-relevant evidence to frontier AI evaluation.
Frontier AI systems are commonly developed, configured, evaluated, secured, and released by the same organizations.
Those organizations possess knowledge that outside reviewers often do not.
They understand:
Developer participation is therefore indispensable.
But developer expertise and developer independence are different properties.
The same organization may be responsible for:
Even highly responsible organizations operate under incentives that can affect these judgments.
Relevant pressures can include:
These pressures do not prove misconduct.
They create a structural reason for independent challenge.
Independent expert review adds value when qualified people or institutions can:
Independent review should not be understood as a ceremonial external endorsement.
A reviewer is not meaningfully independent merely because they:
Independence is multidimensional.
It includes at least:
Organizational independence
The reviewer is not governed by the organization whose claims are under review.
Financial independence
The reviewer is not so dependent on the reviewed party that unfavorable conclusions threaten its survival or future access.
Methodological independence
The reviewer can challenge the evaluation design, propose additional tests, and reject unsupported interpretations.
Informational independence
The reviewer has enough access to form a judgment rather than merely validating a curated presentation.
Publication independence
The reviewer can communicate material conclusions under agreed security and confidentiality constraints.
Decision independence
The reviewer is not required to align its conclusion with the developer's preferred outcome.
Intellectual independence
The reviewer is willing to revise their own assumptions and is not selected only because their prior views are convenient.
Security independence
The reviewer can protect sensitive material without relying entirely on controls administered by the reviewed party.
No reviewer will maximize every dimension.
The objective is not perfect detachment.
The objective is sufficient independence for the claim and decision at issue.
Independence is also insufficient on its own.
An external reviewer may be:
Independent review must therefore be paired with:
Standards Body adopts the following core position:
No organization should be the sole authority over the evaluation of its own most consequential AI systems. Developer evidence should be complemented by independent expert review that is sufficiently competent, informed, resourced, secure, contestable, and free from controlling conflicts to improve the quality of consequential decisions.
A mature review ecosystem should recognize multiple forms of review:
These functions should not be collapsed into one generic category.
The level of review should increase with:
The strongest review systems should also preserve dissent.
A review is less credible when every reviewer is expected to produce consensus.
Unresolved disagreement can be decision-relevant evidence.
The purpose of independent review is not to create a new class of unquestionable authorities.
It is to create a disciplined system in which important claims face informed challenge before institutions, markets, governments, or the public rely on them.
Developer evaluation is necessary, but consequential frontier AI claims should not depend solely on developer judgment.
Independence should be assessed across multiple dimensions and relative to the specific claim, access arrangement, funding model, and decision.
Independent but unqualified review can reduce rather than increase trust.
Review quality is constrained by what reviewers can inspect, test, question, and reproduce.
Independent reviewers should themselves be subject to transparent methods, conflict controls, quality review, correction, and appeal.
No single professional community, institution, laboratory, government, or worldview should define the complete evaluation agenda for frontier AI.
The depth and formality of review should be proportionate to the potential consequence and evidentiary uncertainty of the decision.
This paper covers independent expert review of:
This paper does not fully specify:
Those topics are addressed elsewhere in the Standards Body architecture.
Expert review evaluates technical or institutional evidence using relevant competence.
Public consultation enables broader participation, legitimacy, local knowledge, and affected-party input.
Both matter.
Neither should substitute for the other.
A review may:
An audit normally assesses conformity with defined criteria through a more formal evidence process.
Some activities combine both.
The term used should match the function.
Red teaming seeks to identify weaknesses through adversarial testing.
Independent review can include red teaming but also covers:
Certification is a third-party attestation that specified requirements are fulfilled.
Independent review may inform certification but should not be described as certification unless the full scheme supports that claim.
Academic peer review normally evaluates research for publication.
Frontier AI review may require:
Traditional peer review alone is not enough.
An expert is a person with demonstrable knowledge, skill, judgment, or experience relevant to the specific review question.
Prestige, title, institutional affiliation, or public visibility is not sufficient evidence of expertise.
A reviewer is a person or body responsible for examining evidence, methods, claims, or decisions and producing a reasoned judgment.
An independent reviewer is sufficiently free from controlling interests and organizational authority to form and communicate a genuine judgment.
An external reviewer is outside the reviewed organization.
Externality is one component of independence, not proof of it.
A third party is separate from the supplier or developer and the immediate user or purchaser whose interests are directly represented.
In conformity-assessment practice, first-party, second-party, and third-party activities are distinguished according to the relationship of the assessor to the object and interested parties.[^iso-casco-bodies]
A subject-matter expert possesses specialized domain competence, such as:
A review panel is a group assembled to provide complementary expertise, deliberation, and judgment.
A conflict of interest is a relationship, incentive, obligation, or commitment that could impair, or reasonably appear to impair, impartial judgment.
Impartiality is the presence of objectivity and the management of conflicts so that decisions are not improperly influenced.
Competence is the demonstrated ability to apply knowledge and skills to achieve intended results.
A review mandate defines the question, scope, authority, access, outputs, constraints, timeline, and decision relationship of a review.
Review independence is the degree to which the reviewer can select methods, interpret evidence, communicate findings, and resist improper influence.
Review access is the combination of model access, system information, artifacts, personnel, time, tools, and permissions available to the reviewer.
A review finding is a supported conclusion concerning evidence, process, conformance, risk, capability, safeguard effectiveness, or uncertainty.
Dissent is a reasoned disagreement with the majority or final institutional conclusion.
A minority report is a documented dissenting assessment issued alongside or in connection with the primary review conclusion.
The right of reply allows the reviewed party to respond to factual or procedural findings before publication without obtaining veto power over the conclusion.
Reviewer capture occurs when a reviewer becomes materially influenced by the organization, funder, professional community, government, or ideology it is expected to scrutinize.
Access asymmetry occurs when one reviewer or participant receives materially different information, model access, time, or support from another.
Epistemic diversity is the inclusion of different forms of relevant knowledge, methods, assumptions, and professional experience.
Frontier developers possess essential private information.
This creates a tension.
Outside reviewers need access to evaluate claims, but developers often control access.
Independent review helps convert private evidence into more credible public or institutional assurance.
Internal teams can be affected by:
NIST's AI Risk Management Framework notes that independent review can improve testing effectiveness and reduce internal bias and conflicts of interest.[^nist-rmf-core]
The choice of what to evaluate can be as important as the result.
Independent reviewers can ask:
Developers may optimize for known threat models.
Independent experts can introduce:
External testing can confirm, refine, or challenge internal findings.
AISI has described third-party evaluation as a source of independent verification that can complement developer testing and reduce real or perceived conflicts.[^aisi-lessons]
Trust should not depend solely on self-description.
Independent scrutiny can increase confidence when:
Independent review can improve:
External reviewers may identify:
Institutions gain legitimacy when important judgments can be challenged by qualified parties outside the immediate decision chain.
Independent review cannot guarantee:
It is one layer of assurance.
Questions:
Questions:
Financial ties do not automatically invalidate review.
They should be disclosed and managed.
Questions:
Questions:
Questions:
Questions:
Questions:
Questions:
Questions:
Questions:
Questions:
Every material review should include an independence profile.
The profile should state:
A credible review system requires cooperation.
Developers may need to provide:
Reviewers need enough separation to question that information.
The desired relationship is neither:
It is structured cooperation under independent judgment.
Some review tasks benefit from collaboration:
Others benefit from adversarial independence:
A review mandate should identify the mode.
"Trust but verify" is incomplete when verification is weak.
The relevant principle is:
Cooperate enough to understand the system, verify enough to form an independent judgment.
All reviewers should demonstrate:
May include:
Examples:
May include:
May include:
Reviewers handling sensitive systems may need:
Expertise in the subject is not the same as expertise in reviewing.
Reviewers should know how to:
A panel can possess collective competence that no single member has.
The mandate should identify:
Possible evidence:
Titles should not substitute for evidence.
Every reviewer should state:
Examines:
Examines:
Examines:
Examines whether capability was sufficiently revealed.
Examines controls against representative and adaptive attacks.
Examines:
Examines:
Examines:
Examines:
Examines:
Coordinates scope, process, deliberation, dissent, and reporting.
The chair should not dominate technical conclusions outside their competence.
An observer may monitor process without voting or accessing all content.
Occurs before active evaluation.
Questions:
Examines readiness:
Reviewer observes or participates during testing.
Useful for:
Examines:
A separate party repeats the evaluation.
Examines the developer's integrated reasoning, evidence, assumptions, and mitigation decisions.
Anthropic's Responsible Scaling Policy now authorizes and, in specified circumstances, requires external review of risk reports under governance arrangements involving its Long-Term Benefit Trust.[^anthropic-rsp]
Examines whether a structured argument adequately supports a defined deployment.
Examines whether controls reduce risk under representative attack.
Examines:
Examines real-world evidence and drift.
Triggered by:
Examines proposed technical or institutional standards.
Assesses adherence to a defined framework or requirement.
Examines the quality and consistency of earlier reviews.
Ongoing access and evidence collection support repeated assurance.
This is more resource-intensive and belongs at higher maturity levels.
Select reviewers for the mandate, not for prestige.
Potential selectors:
The reviewed party may propose reviewers but should not control final selection for high-consequence review.
Maintain a pool with metadata on:
Rotation can reduce:
Excessive rotation can lose institutional memory.
Use staggered terms.
Repeat reviewers can gain valuable system knowledge.
Risks should be managed through:
Diversity should include relevant differences in:
Diversity should not be reduced to symbolic demographic representation.
Serious critics can improve review.
They should not be excluded merely for prior disagreement.
They should also be held to evidence and professional standards.
The reviewed party may identify:
Objection should be adjudicated independently.
It should not become veto power over rigorous reviewers.
For some panels, public nomination can expand the pool.
Final selection should remain competence-based.
Open and closed development create different:
A review system dominated by frontier laboratories may misunderstand decentralized development.
A review system dominated by open-source advocates may underestimate security and deployment concerns.
Both perspectives should face evidence.
Open-source representation should include authority to:
Representatives should demonstrate:
Public visibility alone is insufficient.
One representative should not be expected to speak for all open communities.
Open-source reviewers can also have conflicts:
Some open-source experts may require controlled access to review closed systems.
The process should not assume that community affiliation is incompatible with security.
Open-source expertise is particularly valuable for:
People affected by deployment may understand:
Affected parties can contribute through:
Affected-party participation should not be asked to certify technical claims outside competence.
Participation processes should avoid exposing individuals to:
Community expertise should not be extracted without fair compensation where resources allow.
Every review should begin with a written mandate.
What must the review determine?
What is included and excluded?
Can reviewers:
Specify:
What evidence is sufficient?
Possible outputs:
Who receives the review?
What decision can it influence?
Define grounds and process.
State how long the conclusion remains current.
Review quality depends on access.
Possible levels:
Possible information:
Reviewers need time for:
Reviewers may need interviews with:
May include:
Reviewers may need:
Every review report should state its access profile.
Research on external frontier-model evaluation proposes separating model access, model information, and evaluation timeframe because each changes what conclusions are justified.[^external-access]
If access is insufficient, the conclusion should be narrowed.
Greater access can improve rigor while increasing:
Use proportional controls rather than automatic denial.
Different reviewers may have different access.
Material asymmetries should be disclosed.
Translate the mandate into review questions.
Identify:
Select representative:
Compare:
Where possible, rerun or reproduce critical evidence.
Attempt to falsify important claims.
Use structured interviews and preserve notes appropriately.
Confirm that the reviewed evidence applies to the actual system.
Record:
Classify findings by:
Preserve:
The reviewer should not simply restate the developer's framework.
They should evaluate whether that framework is itself adequate.
No rigid hierarchy fits every question.
Reviewers should distinguish:
Where practical, review:
Assertions should be supported by verifiable evidence.
Confidentiality does not reduce the need to assess:
Absence of a finding may reflect:
Each major conclusion should include confidence and rationale.
High-consequence conclusions should ideally receive more than one independent line of evidence.
Where the reviewer participated in creating the evidence, that dual role should be disclosed.
The developer explains:
Reviewers should be able to request:
Developer support may be needed to avoid false negatives caused by poor integration.
Support should not become task coaching.
Before publication, the developer may review for:
The developer should not suppress an unfavorable but supported conclusion.
The final report may include:
Reviewers can assess corrective actions without becoming responsible for operating them.
Repeated review can improve understanding.
It can also increase dependence.
Manage through rotation, disclosure, and governance.
Disclosure should be specific enough to evaluate materiality.
Examples may include:
An organization can be conflicted even if individual reviewers are not.
Perceived conflict matters because legitimacy depends on justified public interpretation.
Appearance alone should not become a tool for arbitrary exclusion.
Maintain:
Serious review requires time and expertise.
Unpaid systems can:
Direct developer payment can create dependence.
Developer pays reviewer.
Strength:
Risk:
Developers contribute to a fund administered independently.
Strength:
Risk:
Government funds review.
Strength:
Risk:
Strength:
Risk:
Can reduce dependence if governed carefully.
Payment should not depend on:
Reviewer organizations should disclose material client concentration.
Early model access, prestige, and relationships can be nonfinancial benefits that influence reviewers.
Disclose:
Review may involve:
May include:
Not every panel member requires access to every artifact.
Some deliberation may remain private to support candor.
Material reasoning should still be represented in the report.
Redaction should protect legitimate interests without removing the basis of the conclusion.
A separate security reviewer can assess proposed disclosure.
The reviewed party may raise concerns but should not be the sole decision-maker.
Reviewers should report:
OpenAI's 2026 third-party evaluation playbook highlights the importance of early model access, appropriate technical integration, clear communication, and secure handling for credible frontier evaluations.[^openai-playbook]
Reviewers should not promise confidentiality beyond their technical, legal, and institutional capacity.
Forced consensus can hide:
A panel should:
A minority report should include:
Anonymous dissent may protect individuals but weaken accountability.
Use only with justified safeguards.
Reviewers should not lose access or future eligibility merely for evidence-based dissent.
The chair should ensure dissent is represented fairly.
Should include qualified people not responsible for the original decision.
Material findings may be:
The approach should reflect consequence and urgency.
Maintain:
Appeals should improve methods, not merely resolve individual disputes.
A public review should disclose:
May include:
The report should state what reviewers could and could not access.
Include:
State what the review does not establish.
Publish a concise response where appropriate.
Avoid language that implies:
Independent reviewers need evaluation.
A second independent party can assess the review process.
Reviewers can assess common cases and compare judgments.
Where methods are standardized, reviewers can demonstrate performance on known or controlled problems.
Collect feedback from:
Feedback should not become client control.
Maintain:
Reviewers may be suspended for:
Formal accreditation, surveillance, and recognition systems will be developed in Foundation 5 and the accreditation framework.
Strengths:
Risks:
Strengths:
Risks:
Strengths:
Risks:
The UK AI Security Institute has conducted frontier-model evaluations and published lessons about third-party evaluation, access, and the evolving science of testing.[^aisi-lessons][^aisi-trends]
Strengths:
Risks:
Strengths:
Risks:
Strengths:
Risks:
Strengths:
Risks:
Strengths:
Risks:
A strong ecosystem may combine:
Frontier AI is not identical to established conformity-assessment domains.
It can still learn from them.
ISO/CASCO standards emphasize competence, consistent operation, and impartiality across testing, inspection, validation, verification, and certification activities.[^iso-casco-bodies][^iso-building-trust]
The relationship of the assessor to the object matters.
Conformity-assessment competence can be recognized through:
ISO/CASCO describes several recognition routes for conformity-assessment bodies.[^iso-recognition]
Useful lessons include:
Frontier AI differs because:
Standards Body proposes five review levels.
Characteristics:
Use:
Characteristics:
Use:
Characteristics:
Use:
Characteristics:
Use:
Characteristics:
Use:
The review level should match the claim and consequence.
A review begins because of:
Define question, scope, authority, access, and outputs.
Assess competence, independence, security, and conflicts.
Define technical, informational, personnel, and time access.
Map claims to evidence.
Developer and reviewers establish shared factual understanding.
Reviewers test, inspect, reproduce, interview, and challenge.
Reviewers compare findings and identify dissent.
Include evidence, confidence, limitations, and recommendations.
Reviewed party identifies errors and security concerns.
Reviewer retains conclusion authority within mandate.
Issue public and restricted outputs.
Resolve material challenges.
Assess corrective action where relevant.
Re-review after system or evidence change.
List claims that should not rely solely on developer judgment.
Identify required roles and competence.
Define conflicts, funding, selection, and publication rights.
Recruit across:
Standardize scope, access, evidence, and outputs.
Test different model, information, and time access arrangements.
Compare advisory and high-stakes reviews.
Support controlled evidence and model access.
Make disagreement operational.
Measure quality and revise process.
Make rigor legible.
Use lessons to develop Foundation 5.
Independent Expert Review of a Dynamic Held-Out Evaluation Protocol
A pilot protocol developed under:
FOUNDATION_01_DYNAMIC_EVALUATION_PROTOCOLS.mdFOUNDATION_02_HELD_OUT_EVALUATIONS.mdFOUNDATION_03_HIGH_STAKES_CAPABILITY_EVALUATION.mdThe preferred initial domain is autonomous cyber capability.
Include:
Review:
Provide:
The pilot succeeds if it:
Failure: External names are used to legitimize a developer-controlled process.
Safeguard: Independence profile, selection transparency, publication rights, access statement.
Failure: Famous reviewers are selected without relevant competence.
Safeguard: Mandate-specific competence evidence.
Failure: Reviewers validate curated evidence without sufficient model or information access.
Safeguard: Access profile and narrowed claims.
Failure: Reviewer depends financially on repeated developer contracts.
Safeguard: Revenue disclosure, pooled funding, rotation, peer review.
Failure: National interests distort technical findings.
Safeguard: international participation, technical-policy separation, dissent.
Failure: Reviewers approach evidence with fixed pro- or anti-AI conclusions.
Safeguard: methodological pluralism and evidence standards.
Failure: Everyone shares the same discipline, institution, or assumptions.
Safeguard: complementary panel design.
Failure: Open-source, public-interest, or affected-party members are present without authority.
Safeguard: defined role, access, vote, and dissent rights.
Failure: Disagreement is removed to create a clean conclusion.
Safeguard: minority reports and unresolved-issue register.
Failure: Factual review becomes conclusion negotiation.
Safeguard: time limit, issue log, reviewer final authority.
Failure: Security is used to conceal weak evidence.
Safeguard: controlled independent scrutiny and public methods.
Failure: Reviewer speaks beyond mandate or competence.
Safeguard: claims boundary and peer review.
Failure: Sensitive models, tasks, or incidents leak.
Safeguard: security qualification, least privilege, incident response.
Failure: Review becomes slow, costly, and disconnected from capability pace.
Safeguard: tiered review levels and clear triggers.
Failure: High-stakes decisions rely on informal consultation.
Safeguard: mandatory mandate, records, conflicts, and report.
Failure: Reviewer fears losing future model access.
Safeguard: access agreements, pooled infrastructure, public disclosure.
Failure: A developer seeks reviewers until receiving a favorable finding.
Safeguard: reviewer registry, disclosure of prior reviews, governance selection.
Failure: Multiple reviews produce incompatible conclusions without explanation.
Safeguard: shared metadata, meta-review, explicit methodological differences.
Failure: Advisory findings are marketed as proof of safety.
Safeguard: precise terminology and public claims controls.
Failure: Old findings are applied to changed systems.
Safeguard: expiration and event-triggered renewal.
Often true.
Response:
Residual concern:
Some knowledge remains tacit and difficult to transfer.
It can.
Response:
Residual concern:
High-quality review requires time.
It can expose:
Response:
Residual concern:
Risk cannot be eliminated.
Correct.
Response:
Residual concern:
No reviewer is value-free.
Direct payment creates a conflict, but it does not automatically invalidate review.
Response:
Residual concern:
Future business can still influence behavior.
Government review can carry public authority.
It can also face:
A mixed ecosystem is preferable.
Community affiliation does not imply security incapacity.
Use competence and controls rather than categorical exclusion.
Response:
Assign roles according to competence.
Not every participant should decide every technical question.
This is a serious risk.
Response:
Correct.
Independent review broadens challenge but cannot guarantee completeness.
It may reduce superficial confidence.
It can increase justified trust by making uncertainty visible.
Several frontier developers use external testing or external review.
This is valuable.
It does not eliminate the need for common standards governing independence, access, method, and claims.[^openai-external][^anthropic-rsp]
Which measures predict genuinely independent reviewer behavior?
What minimum access is required for different claim types?
Which selection models reduce capture while preserving competence and speed?
Which funding models best balance sustainability and independence?
How does disciplinary and institutional diversity affect findings?
When does formal dissent improve decision quality?
How often does external review identify material issues missed internally?
Can reviewers be meaningfully calibrated across frontier evaluations?
Which controls enable deep access without unacceptable leakage risk?
Which mechanisms support meaningful and secure participation?
How should competing national interests be managed?
Do external reviews change deployment and safeguards in beneficial ways?
How long should findings remain current?
How should reviewers adapt if models can recognize and manipulate evaluation?
Develop a standardized multidimensional independence disclosure.
Validate access categories across real evaluations.
Develop task-specific competence models and proficiency exercises.
Compare direct fees, pooled funds, public funding, and mixed structures.
Pilot independent committees, random selection, and rotating pools.
Develop common procedures for evidence mapping, reperformance, challenge, and dissent.
Design secure, meaningful pathways for open and decentralized communities.
Clarify how sociotechnical evidence should connect to technical evaluation.
Develop secure access environments and reviewer security standards.
Evaluate consistency and rigor across independent reviews.
Track whether reviews detect problems and improve decisions.
Study ongoing rather than one-time review.
Develop review methods resistant to model manipulation and sabotage.
Create requirements for cross-border recognition of review findings.
Have reviewers assess the same claim under limited, moderate, and deep access.
Compare single-discipline and multidisciplinary panels.
Compare developer-selected and independently selected reviewers.
Test how different payment structures affect perceived and actual independence.
Run a panel with a formal minority-report mechanism.
Include qualified open-weight experts in a controlled review and evaluate their distinct contributions.
Test a structured developer response process that separates correction from conclusion negotiation.
Have a second panel assess the first panel's methodology.
Compare one-time review with ongoing evidence access.
Give reviewers common cases and measure judgment consistency.
Simulate sensitive-evidence compromise during review.
Track which recommendations are accepted and whether outcomes improve.
A future independent expert review standard could require:
A written question, scope, authority, and decision relationship.
Documented reviewer and panel competence.
Organizational, financial, methodological, informational, publication, and intellectual independence.
Disclosure, assessment, mitigation, recusal, and register.
Transparent and proportionate reviewer-selection process.
Defined model, information, personnel, artifact, and time access.
Evidence map, sampling, testing, reperformance, interviews, and uncertainty.
Controls for sensitive models, tasks, incidents, and reports.
Rules for panel judgment and dissent.
Factual and security review without conclusion veto.
Public mandate, access, methods, findings, limitations, funding, conflicts, and status.
Independent challenge and correction process.
Reviewer performance, meta-review, and suspension.
Time and event triggers for renewal.
Such a standard should be developed through the future STANDARDS_DEVELOPMENT_PROCESS.md.
Independent reviewers help determine when evaluations should change, expire, or retire.
Protected content requires independent scrutiny so confidentiality does not shield weak methods.
The consequence of the capability should determine review depth and panel competence.
Foundation 4 defines the review function and principles. Foundation 5 will define how organizations scale, qualify, compete, and remain accountable.
Independent review can mature from voluntary challenge into procurement, certification, or formal assurance.
Recognition should reward rigorous, honest review, including correction and dissent.
Review findings need compatible metadata, competence, access, and recognition across jurisdictions.
Standards Body adopts the following working positions.
Developer evaluation is essential but insufficient for the most consequential claims.
No organization should be the sole authority over the evaluation of its own most consequential systems.
Externality is not the same as independence.
Independence is multidimensional and claim-specific.
Independence without competence does not create trustworthy review.
Competence without sufficient access does not support broad conclusions.
Every material review should disclose its mandate, access, funding, conflicts, and limitations.
Review depth should increase with potential consequence and uncertainty.
Reviewer selection should be based on mandate-specific competence rather than prestige.
The reviewed party may contribute expertise and correct factual errors but should not control the final conclusion.
Reviewers should be paid fairly, but payment should not depend on a favorable outcome.
Repeat client relationships require additional safeguards.
Independent reviewers should themselves be accountable, correctable, and reviewable.
Dissent is legitimate evidence and should not be erased for appearance of consensus.
Open-source and open-weight expertise should receive meaningful representation where relevant.
Public-interest and affected-party knowledge should complement technical review without replacing technical competence.
Security restrictions should be proportionate and should not prevent qualified scrutiny.
A review should state what reviewers could not access.
Advisory review should not be marketed as certification or proof of safety.
Reviews should expire after material system, protocol, or deployment changes.
Multiple independent institutions are preferable to one permanent review monopoly.
International recognition should depend on competence, independence, access, method, and accountability.
Public transparency alone cannot replace secure expert review of legitimately confidential evidence.
Confidential evidence alone cannot replace public accountability.
The review ecosystem should be evaluated for capture, concentration, exclusion, and effectiveness.
Independent review should be required when:
A review should not be described as independent when:
A reviewer should be replaced or supplemented when:
A review should be renewed when:
Reviewer or organization:
Reviewed party:
Mandate:
Date:
Reviewer:
Organization:
Review:
Finding identifier:
Review:
Category:
Severity:
Confidence:
Review:
Minority reviewer or group:
Date:
| Dimension | Core Question |
|---|---|
| Mandate | Is the question, scope, and authority explicit? |
| Competence | Does the reviewer or panel cover the required expertise? |
| Organizational independence | Is the reviewer outside the reviewed authority chain? |
| Financial independence | Are payment and client dependence controlled? |
| Methodological independence | Can reviewers challenge methods and add tests? |
| Informational independence | Is access sufficient and not excessively curated? |
| Operational independence | Can reviewers verify and run relevant evidence? |
| Publication independence | Can material findings be communicated? |
| Intellectual independence | Is the panel capable of genuine disagreement? |
| Selection | Was reviewer choice legitimate and transparent? |
| Conflicts | Are interests disclosed and managed? |
| Access | Are model, information, artifacts, personnel, and time adequate? |
| Evidence | Are findings supported by appropriate evidence? |
| Reperformance | Can critical evidence be reproduced or verified? |
| Security | Are sensitive materials handled competently? |
| Deliberation | Is panel reasoning documented? |
| Dissent | Can minority views be recorded? |
| Developer response | Can factual errors be corrected without conclusion veto? |
| Reporting | Are methods, access, funding, limitations, and findings legible? |
| Appeal | Can material error be challenged independently? |
| Quality assurance | Is reviewer performance evaluated? |
| Open-source participation | Is relevant decentralized expertise included meaningfully? |
| Public interest | Are deployment and affected-party considerations represented? |
| Expiration | Is the review tied to time and system version? |
| Decision utility | Does the review improve an actual decision? |
Independent expert review is often described as a trust mechanism.
That description is incomplete.
Trust should not be produced by the presence of an external name on a report.
It should be produced by a process that makes scrutiny real.
A credible reviewer needs:
Frontier AI creates a difficult asymmetry.
Developers possess the strongest access.
Outsiders may possess greater independence.
Neither access without independence nor independence without access is enough.
The institutional task is to combine them.
This requires cooperation without control.
It requires confidentiality without opacity.
It requires expertise without prestige theater.
It requires compensation without purchased conclusions.
It requires pluralism without loss of technical rigor.
It requires public accountability without unsafe disclosure.
It also requires humility.
Independent reviewers can be wrong.
Panels can be captured.
Governments can be political.
Nonprofits can depend on funders.
Academics can chase publication.
Commercial evaluators can chase contracts.
Critics can become ideological.
Experts can overreach.
For that reason, independent review should not create a new unquestionable authority.
It should create a more contestable evidence system.
The fourth foundation of Standards Body is therefore not external approval.
It is informed challenge under conditions that make judgment genuinely independent.
[^nist-rmf-core]: National Institute of Standards and Technology, AI RMF Core, including guidance on independent review and internal bias. https://airc.nist.gov/airmf-resources/airmf/5-sec-core/
[^nist-tevv]: National Institute of Standards and Technology, AI Test, Evaluation, Validation and Verification. https://www.nist.gov/ai-test-evaluation-validation-and-verification-tevv
[^nist-conformance]: National Institute of Standards and Technology, Overview of Conformance Testing. https://www.nist.gov/itl/ai/applied-ai-research-group/overview-conformance-testing
[^nist-global]: National Institute of Standards and Technology, A Plan for Global Engagement on AI Standards, 2024. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-5.pdf
[^aisi-lessons]: UK AI Security Institute, Early Lessons from Evaluating Frontier AI Systems, October 24, 2024. https://www.aisi.gov.uk/blog/early-lessons-from-evaluating-frontier-ai-systems
[^aisi-trends]: UK AI Security Institute, Frontier AI Trends Report. https://www.aisi.gov.uk/frontier-ai-trends-report
[^aisi-qa]: UK AI Security Institute, Early Insights from Developing Question-Answer Evaluations for Frontier AI, September 23, 2024. https://www.aisi.gov.uk/blog/early-insights-from-developing-question-answer-evaluations-for-frontier-ai
[^aisi-research]: UK AI Security Institute, Research Agenda. https://www.aisi.gov.uk/research-agenda
[^openai-external]: OpenAI, Strengthening Our Safety Ecosystem with External Testing, November 19, 2025. https://openai.com/index/strengthening-safety-with-external-testing/
[^openai-playbook]: OpenAI, A Shared Playbook for Trustworthy Third-Party Evaluations, May 29, 2026. https://openai.com/index/trustworthy-third-party-evaluations-foundations/
[^openai-early]: OpenAI, Early Access for Safety Testing, December 20, 2024. https://openai.com/index/early-access-for-safety-testing/
[^anthropic-rsp]: Anthropic, Responsible Scaling Policy, Version 3.2, April 29, 2026. https://www.anthropic.com/responsible-scaling-policy
[^anthropic-rsp3]: Anthropic, Responsible Scaling Policy Version 3.0, February 24, 2026. https://www.anthropic.com/news/responsible-scaling-policy-v3
[^deepmind-warning]: Google DeepMind, An Early Warning System for Novel AI Risks, 2023. https://deepmind.google/blog/an-early-warning-system-for-novel-ai-risks/
[^external-access]: Jacob Charnock, Alejandro Tlaie, Kyle O'Brien, Stephen Casper, and Aidan Homewood, Expanding External Access to Frontier AI Models for Dangerous Capability Evaluations, 2026. https://arxiv.org/abs/2601.11916
[^frontier-auditing]: Miles Brundage et al., Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies, 2026. https://arxiv.org/abs/2601.11699
[^compliance-review]: Aidan Homewood et al., Third-Party Compliance Reviews for Frontier AI Safety Frameworks, 2025. https://arxiv.org/abs/2505.01643
[^dangerous-capabilities]: Mary Phuong et al., Evaluating Frontier Models for Dangerous Capabilities, 2024. https://arxiv.org/abs/2403.13793
[^iso-casco-bodies]: International Organization for Standardization, CASCO Toolbox: Bodies. https://casco.iso.org/bodies.html
[^iso-recognition]: International Organization for Standardization, CASCO Toolbox: Recognition of Conformity Assessment Bodies. https://casco.iso.org/recognition-of-cabs.html
[^iso-building-trust]: International Organization for Standardization, Building Trust: The Conformity Assessment Toolbox. https://www.iso.org/iso/casco_building-trust.pdf
[^iaf-accreditation]: International Accreditation Forum, What Is Accreditation? https://iaf.nu/en/faq/what-is-accreditation/
Date: July 16, 2026
Change type: Complete foundational edition
Summary: Establishes the fully developed canonical working white paper for Foundation 4. Defines the rationale for independent expert review, multidimensional independence, competence, reviewer roles, review types, reviewer selection, open-source and public-interest participation, mandates, access, methodology, evidence, developer interaction, conflicts, funding, security, dissent, appeals, transparency, reviewer quality assurance, institutional models, conformity-assessment lessons, review levels, lifecycle, implementation pathway, a Standards Body pilot, metrics, failure analysis, objections, evidence gaps, research agenda, standards implications, operational templates, scorecard, and primary-source research basis.
Status: Ready for internal review and future expert critique.