Comment on NIST CAISI RFI 2025-0035: AI Agent Security
Box Commons · 30 N Gould St Ste N, Sheridan WY 82801
- Behavioral safety is a distinct security domain — a secure agent can still cause catastrophic harm through behavioral failures.
- NIST should expand agent security standards to include a behavioral safety layer suitable for insurance underwriting.
- Third-party credentialing provides the verification mechanism that self-assessment cannot.
+ Jump to Section
I. Introduction
Box Commons is a 501(c)(6) standards body focused on developing third-party credentialing standards for autonomous AI agent operations. Our credentialing model maps to the NIST AI Risk Management Framework and draws on the HITRUST approach to certifiable, technology-agnostic standards.
This comment responds to the CAISI Request for Information on AI Agent Security, arguing that behavioral safety constitutes a distinct domain requiring independent, third-party assessment — not merely an extension of existing cybersecurity frameworks.
II. Behavioral Safety as a Distinct Security Domain
The RFI correctly identifies agent security as a priority but treats it as an extension of cybersecurity. We argue that behavioral safety is a separate, orthogonal domain.
An agent with perfectly legitimate, cryptographically authenticated access to a financial trading API can execute a catastrophic, hallucination-driven sequence of trades without violating a single cybersecurity access control. These are not security failures — they are behavioral failures on an independent axis.
The distinction matters because the evaluation methods, monitoring approaches, and credentialing standards for behavioral safety differ fundamentally from those for cybersecurity.
III. The Case for Third-Party Credentialing
Self-assessment creates a structural conflict of interest. The entity deploying an AI agent has financial incentives that can conflict with rigorous behavioral evaluation. Third-party credentialing resolves this by interposing an independent evaluator.
The HITRUST model provides a proven precedent: a technology-agnostic certification framework that multiple regulatory regimes recognize as evidence of compliance, without requiring each regulator to build its own evaluation infrastructure.
IV. Recommendations
1. Expand the definition of "agent security" to include behavioral safety as a distinct domain alongside cybersecurity.
2. Develop evaluation standards specifically designed for behavioral safety assessment, distinct from penetration testing and vulnerability analysis.
3. Establish a credentialing pathway that third-party organizations can use to certify agent behavioral safety, analogous to the FedRAMP 3PAO model.
V. Conclusion
NIST has the opportunity to establish behavioral safety as a formal domain within the AI security landscape. The credentialing infrastructure to support this already has proven precedents in healthcare (HITRUST) and federal IT (FedRAMP). We urge CAISI to include behavioral safety credentialing in its agent security standards development.
Contact:
Brice Love, Acting Executive Director
Box Commons
[email protected]
Content Integrity Notice: This comment was authored by the Box Commons Policy Working Group. Generative AI was used for research synthesis and drafting support. All policy positions, recommendations, and normative claims were formulated and reviewed by human authors.
Related Filings
Comment on NCCoE AI Agent Identity and Authorization
Recommends that agent identity credentials be interlocked with behavioral safety verification — no credential without verified behavioral certification. Identity without behavioral verification provides incomplete assurance to downstream systems.
NISTComment on NIST AI 800-2: Evaluation Practices for Language Models
Five observations on strengthening NIST's evaluation framework as results increasingly inform third-party credentialing, regulatory compliance, and insurance underwriting. Argues that behavioral safety is a distinct evaluation domain requiring independent assessment.
IMDAComment on the Model AI Governance Framework for Agentic AI
Five recommendations on strengthening Singapore's agentic AI governance framework through behavioral safety credentialing and international interoperability. First international filing by Box Commons.