The White House recently brought together some of the world’s largest AI companies around an important commitment: independent oversight of the safeguards designed to keep their systems safe. The accord, released on Sept. 29, calls on developers to monitor risks, establish internal controls and bring in outside auditors or evaluators to assess whether those controls work. It’s an encouraging development for an industry where companies have largely been responsible for evaluating their own technology.
But the agreement leaves a critical gap. It calls for independent assessments without defining who qualifies to conduct them, what independence requires or how conflicts of interest should be handled. Without those standards, two developers could fulfill the same commitment through assessments with vastly different levels of rigor and objectivity, leaving businesses and regulators uncertain about how much confidence to place in the results.
At Schellman, we provide accredited AI assurance, and we’ve seen firsthand why the credibility of that work depends on independence, clear scope and qualified professionals. The assurance profession has spent decades establishing standards for competency, independence and quality oversight across financial reporting, cybersecurity and privacy. Those standards exist because an independent opinion is only as credible as the people conducting the assessment and the evidence supporting their conclusions. Independence isn’t something you simply declare. It’s something you have to demonstrate.
Independence has to hold up in practice
Consider an AI developer hiring a firm to evaluate its safety controls. The evaluator agrees to a narrow scope, performs testing and issues a favorable report. If that same firm helped design the controls under review, its objectivity deserves scrutiny. If major risks fall outside the scope, customers might mistake limited findings for a broader endorsement of the system.
For executives choosing AI vendors, these distinctions shape purchasing and deployment decisions. They need to know who performed the assessment, whether they had the right expertise, which safeguards they examined and what they excluded. The evaluator also needs access to sufficient evidence and the freedom to report unfavorable findings. A report is useful only when readers understand both its conclusions and its limits.
Assessors should disclose relevant commercial relationships before work begins and document how they manage conflicts. They should use defined criteria, explain their methods and face independent quality review. Accreditation, where appropriate, adds another way to verify competence and impartiality.
Independence matters most when problems emerge
The real test of independent oversight comes when an assessment uncovers something a developer would rather not hear. The accord calls for internal teams to address identified risks, with an independent board committee overseeing findings and responses. But those safeguards depend on assessors having the independence to report concerns without pressure to soften their conclusions.
Consider an evaluator who identifies a serious cybersecurity vulnerability in a frontier model. If the developer disputes the finding or pushes to narrow its scope, the evaluator must be able to stand behind the evidence and communicate the risk clearly. The developer then needs to determine whether additional testing, corrective action or release restrictions are warranted. Without an independent assessment that accurately captures the original risk, even a well-defined remediation process can fall short.
Credible AI assurance also requires the right expertise. Technical evaluators understand model behavior, emerging capabilities and attack scenarios, while assurance professionals bring experience evaluating governance, internal controls and supporting evidence. Both perspectives are necessary, but neither automatically qualifies someone to assess every aspect of an AI system. As AI assurance evolves, the industry needs clearer expectations for the qualifications, methodologies and oversight required of those conducting assessments.
We’re already seeing progress in how AI assurance is conducted. The AICPA and AIUC recently announced a joint offering pairing SOC 2 examinations with AIUC-1 certification, combining established assurance practices with technical testing of AI agents, including quarterly retesting as risks evolve. It’s a promising step toward more rigorous assessments, but stronger methodologies alone cannot guarantee credible results. The assessors applying them must also demonstrate the expertise, independence and accountability necessary to earn trust.
Make oversight useful to the people relying on it
The companies signing the accord have an opportunity to establish clearer expectations for independent AI assurance. That should include minimum qualifications for assessors, transparent assessment methodologies, conflict-of-interest disclosures and independent quality oversight. Reports should clearly communicate their scope, limitations and material findings so boards, regulators and customers can understand what was evaluated and how much confidence to place in the results.
Standards also need to be practical for smaller developers. AI adoption extends beyond the largest technology companies, and an assurance process so expensive that smaller firms cannot participate would leave gaps across the market. Expectations should match the risks involved without compromising independence.
The White House has taken an important step by putting independent oversight at the center of its AI safety commitments. But for that oversight to carry weight, the industry needs clear standards for who can conduct assessments, how their independence is verified and what accountability looks like when those standards aren’t met. Without those expectations, developers may be able to demonstrate that an assessment took place without proving it was credible. As AI systems take on greater responsibility, the people evaluating them must be held to equally rigorous standards.
The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.
This story was originally featured on Fortune.com
