11
4.5 Implementation
Implementation should follow formal development and change management protocols,
including rollback plans and compatibility checks with existing systems. Shadow mode
deployment and mirrored systems are recommended to simulate AI behavior before live
execution, allowing firms to assess performance in a controlled environment.
A rigorous, multi-disciplinary approval process should be in place so that all stakeholders can
review and sign off on model design, development and testing results, sensitivities and limits,
compliance and regulatory issues, cybersecurity concerns, and any necessary control changes.
A single point of accountability should be assigned as part of the approval process prior to first
use.
Admin roles for trading software should be distributed among IT, traders, Risk, and
Compliance, ensuring independent oversight and the ability to intervene via kill switches or
similar controls.
Initial deployment of AI trading tools should be constrained by small volumetric limits,
drawdown limits, and regular monitoring of key risk indicators (KRIs) and performance
indicators (KPIs) until the company is comfortable with the tools’ performance.
4.6 Usage and Monitoring
Usage controls should include access restrictions, user training, and supervision to prevent
misuse and ensure responsible operation. Monitoring activities should involve regular
performance reviews, back testing, and incident reporting to detect anomalies and maintain
model reliability.
Trading AI tools should track prediction accuracy, trade execution quality, P&L attribution
related to AI decisions, maximum drawdown, VaR limit utilization, position concentration,
correlation breakdown detection, and model confidence scores. Operational AI tools should
monitor processing time reduction, error rate reduction, automation rate, failed transaction
rates, data quality exceptions, and manual intervention frequency.
To further support responsible AI deployment, firms should implement additional safeguards
across the lifecycle. These include shadow mode deployment and mirrored systems to simulate
behavior before production, kill switches with tested escalation paths for anomalies, and trader
certification programs that define permissible AI usage and prohibit unauthorized model
development.
Segregation of duties is critical, with IT managing administrative roles and Risk/Compliance
overseeing kill switch authority. Firms should also consider volumetric and product limits,
explainability checks, version control, and rollback mechanisms to ensure safe and transparent
implementation.
4.7 Revalidation
Periodic revalidation should be scheduled based on risk tier.
Updates to documentation, performance metrics, and assumptions are necessary to maintain
relevance and accuracy over time. AI Tools, especially those used for trading, are prone to data
overfitting, so ongoing revalidation and recalibration will be critical to continued performance
quality. Backtesting should occur on a regular basis.
4.5 Implementation
Implementation should follow formal development and change management protocols,
including rollback plans and compatibility checks with existing systems. Shadow mode
deployment and mirrored systems are recommended to simulate AI behavior before live
execution, allowing firms to assess performance in a controlled environment.
A rigorous, multi-disciplinary approval process should be in place so that all stakeholders can
review and sign off on model design, development and testing results, sensitivities and limits,
compliance and regulatory issues, cybersecurity concerns, and any necessary control changes.
A single point of accountability should be assigned as part of the approval process prior to first
use.
Admin roles for trading software should be distributed among IT, traders, Risk, and
Compliance, ensuring independent oversight and the ability to intervene via kill switches or
similar controls.
Initial deployment of AI trading tools should be constrained by small volumetric limits,
drawdown limits, and regular monitoring of key risk indicators (KRIs) and performance
indicators (KPIs) until the company is comfortable with the tools’ performance.
4.6 Usage and Monitoring
Usage controls should include access restrictions, user training, and supervision to prevent
misuse and ensure responsible operation. Monitoring activities should involve regular
performance reviews, back testing, and incident reporting to detect anomalies and maintain
model reliability.
Trading AI tools should track prediction accuracy, trade execution quality, P&L attribution
related to AI decisions, maximum drawdown, VaR limit utilization, position concentration,
correlation breakdown detection, and model confidence scores. Operational AI tools should
monitor processing time reduction, error rate reduction, automation rate, failed transaction
rates, data quality exceptions, and manual intervention frequency.
To further support responsible AI deployment, firms should implement additional safeguards
across the lifecycle. These include shadow mode deployment and mirrored systems to simulate
behavior before production, kill switches with tested escalation paths for anomalies, and trader
certification programs that define permissible AI usage and prohibit unauthorized model
development.
Segregation of duties is critical, with IT managing administrative roles and Risk/Compliance
overseeing kill switch authority. Firms should also consider volumetric and product limits,
explainability checks, version control, and rollback mechanisms to ensure safe and transparent
implementation.
4.7 Revalidation
Periodic revalidation should be scheduled based on risk tier.
Updates to documentation, performance metrics, and assumptions are necessary to maintain
relevance and accuracy over time. AI Tools, especially those used for trading, are prone to data
overfitting, so ongoing revalidation and recalibration will be critical to continued performance
quality. Backtesting should occur on a regular basis.
