Navigating the AI Security Landscape: A Deep Dive into MITRE's SAFE-AI Framework for Compliance

The rapid integration of Artificial Intelligence (AI) into Information Technology (IT) systems is fundamentally changing how we approach cybersecurity. While AI offers transformative capabilities, it also introduces new vectors for adversarial actions that greatly expand the attack surface of IT systems. For cybersecurity and AI professionals tasked with securing information systems and ensuring compliance, understanding and mitigating these novel risks is paramount.

This is where MITRE's SAFE-AI framework comes in. Developed by authors such as J. Kressel and R. Perrella, SAFE-AI is a comprehensive guide for securing AI-enabled systems. It addresses the urgent need for AI-focused security guidance, especially given that traditional IT security approaches and current risk frameworks do not comprehensively address the unique risks posed by AI systems.

The Unique Challenges of AI Security

Why is AI security different? AI's inherent dependency on data and corresponding learning processes significantly contributes to an expanded attack surface. Unlike traditional IT systems, AI systems can exhibit unique and often non-deterministic behaviors.

Cloud Security Configuration Checker | Security Assessment Tool

Assess your cloud security posture across AWS, Azure, GCP, and other providers. Map security controls to compliance frameworks like CSA CCM, ISO 27001, NIST CSF.

Security Assessment Tool

Key AI-specific security concerns include:

• Adversarial inputs: Maliciously crafted inputs designed to manipulate AI models.
• Poisoning: Attacks that perturb AI model inputs or modify AI models to undermine their reliability, integrity, and availability, often by making changes to code, objective functions, model parameters, or training data.
• Exploiting automated decision-making: Leveraging AI's automated processes for malicious ends.
• Exploiting model biases: Manipulating or leveraging inherent biases within models.
• Exposure of sensitive information: Such as Personally Identifiable Information (PII), Protected Health Information (PHI), or Federal Tax Information (FTI), which can be unintentionally embedded into a model if it changes after exposure to sensitive data.
• Supply chain vulnerabilities: A significant concern stemming from the unclear provenance of AI models, tools, and data. AI systems often rely on third-party libraries, frameworks, and pre-trained models, which may contain hidden vulnerabilities or malicious code. The high cost of training Large Language Models (LLMs), for instance, means most organizations acquire them from open-source or proprietary sources with little method for determining risks.

Introducing SAFE-AI: A Unified Approach

To tackle these challenges, SAFE-AI strengthens the processes of security control selection and assessment by ensuring that AI-specific threats and concerns are systematically identified and addressed. It aims to make it possible to develop and secure trustworthy AI-enabled systems.

The framework is built upon established and widely recognized standards:

• National Institute of Standards and Technology (NIST) standards: Specifically, NIST Special Publication 800-53, Revision 5 (which provides the control catalog for federal information systems), the NIST AI Risk Management Framework (RMF), and the NIST Risk Management Framework (RMF).
• MITRE Adversarial Threat Landscape for Artificial Intelligence Systems (ATLAS)™ framework: A globally accessible knowledge base of adversary tactics and techniques against AI-enabled systems, based on real-world observations and demonstrations from AI red teams. SAFE-AI leverages ATLAS™'s taxonomy for identifying AI-specific concerns and determining relevant mitigation controls.

Key Pillars of SAFE-AI

SAFE-AI provides guidance for securing AI-enabled systems by systematically addressing AI-specific threats and concerns across four distinct system elements:

• Environment: The operational setting of an AI-enabled system, encompassing infrastructure, network, compute, and storage.
• AI Platform: The application software, AI software, and the operating system.
• AI Model: The software program and its algorithms that process input data to produce outputs (e.g., predictions, recommendations, generated content). This includes various AI technologies like Large Language Models (LLMs) and Generative AI (GenAI).
• AI Data: Data utilized for training and tuning AI models.

The framework also augments three key steps of the NIST Risk Management Framework (RMF) to incorporate AI-specific considerations:

1. Prepare: This step involves identifying AI subject matter experts, reassessing risk tolerance levels for AI-related risks, incorporating SAFE-AI into organization-wide risk assessments, and adjusting common controls for AI-enabled systems. Assessors must plan to address each AI concern, mapping MITRE ATLAS™ Threats to the four system elements and enumerating relevant NIST SP-800-53 Controls.
2. Select: This involves choosing, tailoring, and allocating appropriate security controls. For AI-enabled systems, this means augmenting the selection, tailoring, and allocation tasks to address AI-specific threats, drawing from the hundred NIST SP-800-53 controls identified as potentially AI-affected.
3. Assess: This step focuses on conducting security control assessments. SAFE-AI provides supplemental assessment criteria through question-and-answer sets for assessors to use during interviews with system owners and stakeholders, ensuring that AI-specific risks are considered across the four system elements.

AI Security Risk Assessment Tool

Systematically evaluate security risks across your AI systems

AIRiskAssess.comAIRiskAssess Team

Addressing AI-Specific Threats for Compliance Professionals

For compliance professionals, understanding the granular threats covered by SAFE-AI is crucial. The framework delves into specific AI threats and concerns, outlining potential residual risks and related ATLAS™ identifiers. Examples of critical threats include:

• Model Poisoning and Data Poisoning: Attacks that can embed vulnerabilities or biases into an AI-enabled system, which may be difficult to detect. Compliance requires stringent preprocessing, validation, and continuous testing of data and models.
• Insecure APIs: APIs, especially inference APIs, are vulnerable to unauthorized access and manipulation, leading to incorrect predictions or compromised decision-making. Robust authentication, authorization, and input validation are essential.
• Sensitive Data/Information Exposure: Accidental or malicious disclosure of sensitive data (like PII) or proprietary algorithms, potentially through AI memorization during training or prompt engineering. This necessitates strong access controls, encryption, and data sanitization techniques.
• Supply Chain Infiltrations: A pervasive risk where unvetted changes or compromises in external AI models, tools, data, or environment components can introduce malicious code or vulnerabilities. This highlights the need for thorough vetting, documentation of provenance, and careful change management for all AI components.
• Prompt Injection (Direct and Indirect): Malicious prompts designed to bypass defenses, generate harmful content, or issue privileged commands, often exploiting the non-transparent logic of AI models. This requires strong input validation and continuous monitoring of AI responses.
• AI Bias: Biases in data or models leading to inaccurate or discriminatory outcomes. Compliance requires careful attention to data quality, statistical properties, and vigilant monitoring of system performance for bias detection.
• Excessive Agency and Insecure Plugin Design: When AI components have unnecessary permissions or poorly designed plugins, leading to unintended and potentially damaging consequences like data exfiltration or privilege escalation. Strict control over functionality, permissions, and diligent vetting of plugins are critical.

Why This Matters for Your Compliance Program

The guidance presented in SAFE-AI is designed for cybersecurity and AI professionals with technical responsibilities for securing information systems, including those developing system security plans (SSPs), planning and performing security control assessments (SCAs), or developing system architectures that defend against adversarial AI.

By systematically applying SAFE-AI, organizations can:

• Identify new and unique AI-specific threats for which assessors may not be prepared.
• Bolster existing security assessment criteria to meet the needs of AI-enabled systems.
• Ensure the thorough evaluation of risks introduced by AI technologies.
• Facilitate the selection of security controls that align with the level of risk posed by AI advancements.

In essence, SAFE-AI provides the necessary framework to address the complex and evolving security challenges of AI, allowing organizations to deploy and leverage AI technologies with greater confidence and in adherence to robust security and compliance standards.