6
Beyond foundational risk categories, firms must address performance-specific
challenges. Overfitting occurs when models perform well on training data but fail to
generalize in new market conditions. Model drift (e.g., gradual performance
degradation) and data drift (e.g., changing input patterns) require ongoing monitoring.
Black box opacity in complex models creates validation challenges, especially for deep
learning and ensemble methods. Continuous learning models pose unique validation
risks as they evolve post-deployment.
Agentic AI Systems that orchestrate multiple tools pose unique validation challenges,
particularly when outputs are sent externally without human review.
• Operational &Technology Risks
AI system failures can disrupt critical business operations. System downtime, latency
issues, and performance degradation affect reliability. Integration failures with existing
infrastructure creates operational risk. Dependency failures from third-party APIs, data
feeds, or cloud services can cascade through AI systems. In addition, edge case failures
can occur when AI systems encounter scenarios outside their training distribution
• Vendor Risks
Many AI tools enter the firm through third-party software. These systems may perform
key forecasting or reconciliation tasks but offer little visibility into model architecture or
training data. Governance practices should include clear protocols for evaluating vendor
tools that rely on AI, especially when they impact financial or regulatory outputs.
• Cybersecurity &Security Risks
AI systems face significant cybersecurity threats that can compromise model integrity,
data confidentiality, and system availability. Adversarial attacks include data poisoning
(corrupting training data), model evasion (fooling deployed models), and model
extraction (stealing proprietary models). Prompt injection and jailbreaking pose risks for
LLMs.
API vulnerabilities, data breaches, and unauthorized access threaten system security.
Firms should implement robust access controls, input validation, security monitoring,
and incident response protocols specific to AI systems.
Understanding and categorizing these risks is essential for building resilient AI Risk
Management Frameworks. By proactively identifying exposures across these categories, firms
can tailor controls, oversight mechanisms, and strategic responses to ensure safe and effective
AI deployment.
3. Risk Management Framework
The CCRO recommends companies adopt a robust AI Risk Management Framework built on
five pillars: governance, recognition, measurement, management, and integration.
Beyond foundational risk categories, firms must address performance-specific
challenges. Overfitting occurs when models perform well on training data but fail to
generalize in new market conditions. Model drift (e.g., gradual performance
degradation) and data drift (e.g., changing input patterns) require ongoing monitoring.
Black box opacity in complex models creates validation challenges, especially for deep
learning and ensemble methods. Continuous learning models pose unique validation
risks as they evolve post-deployment.
Agentic AI Systems that orchestrate multiple tools pose unique validation challenges,
particularly when outputs are sent externally without human review.
• Operational &Technology Risks
AI system failures can disrupt critical business operations. System downtime, latency
issues, and performance degradation affect reliability. Integration failures with existing
infrastructure creates operational risk. Dependency failures from third-party APIs, data
feeds, or cloud services can cascade through AI systems. In addition, edge case failures
can occur when AI systems encounter scenarios outside their training distribution
• Vendor Risks
Many AI tools enter the firm through third-party software. These systems may perform
key forecasting or reconciliation tasks but offer little visibility into model architecture or
training data. Governance practices should include clear protocols for evaluating vendor
tools that rely on AI, especially when they impact financial or regulatory outputs.
• Cybersecurity &Security Risks
AI systems face significant cybersecurity threats that can compromise model integrity,
data confidentiality, and system availability. Adversarial attacks include data poisoning
(corrupting training data), model evasion (fooling deployed models), and model
extraction (stealing proprietary models). Prompt injection and jailbreaking pose risks for
LLMs.
API vulnerabilities, data breaches, and unauthorized access threaten system security.
Firms should implement robust access controls, input validation, security monitoring,
and incident response protocols specific to AI systems.
Understanding and categorizing these risks is essential for building resilient AI Risk
Management Frameworks. By proactively identifying exposures across these categories, firms
can tailor controls, oversight mechanisms, and strategic responses to ensure safe and effective
AI deployment.
3. Risk Management Framework
The CCRO recommends companies adopt a robust AI Risk Management Framework built on
five pillars: governance, recognition, measurement, management, and integration.
