Client Profile
The client is a U.S. registered Futures Commission Merchant and member of the National Futures Association. Through multiple branches, the firm offers trade execution, brokerage, clearing, electronic trading, foreign exchange, and managed futures services to customers. Its operations span listed derivatives execution, OTC FX pricing and trading, clearing and margin management, and managed account programs.
Governance Challenge
AI and machine learning had entered the firm unevenly. Algorithmic execution tools, margin and clearing risk models, FX pricing engines, managed futures allocation models, and client onboarding screening tools had been deployed by individual business lines without central oversight. There was no AI specific policy, no consolidated model inventory, and no named owner for AI failures. At the same time, NFA and CFTC examination attention was increasingly focused on model risk, electronic trading supervision, and recordkeeping. The firm needed a structured way to assess its AI governance maturity and close the gap before it became a regulatory liability.
Baseline Assessment
A baseline assessment using the five dimension AI governance maturity matrix was completed within 30 days.
| Dimension | Current Level | Evidence |
|---|---|---|
| Strategy and Leadership | Level 1 | No named executive sponsor for AI; AI initiatives launched independently by trading, FX, clearing, and managed futures desks. |
| Policy and Ethics | Level 1 | No AI acceptable use policy; existing compliance manual did not cover machine learning models or automated decision systems. |
| Risk Management | Level 2 | Traditional model validation existed for margin models, but AI systems were not risk tiered; residual risk after controls was not measured. |
| Data Governance | Level 1 | No unified model registry; no data lineage for AI training inputs; data fragmented across clearing, FX, and managed futures platforms. |
| Monitoring and Observability | Level 1 | Manual trade surveillance; no automated drift detection for execution, pricing, or risk models; no governance dashboards. |
Key Findings and Risk Classification
The discovery phase identified 47 AI and machine learning systems across the firm, more than double the number leadership initially believed existed. These included algorithmic execution models used by customers in electronic trading, margin and counterparty credit risk models supporting clearing, FX pricing and hedging models, managed futures allocation and rebalancing models, client onboarding and AML/KYC screening tools, sanctions screening tools, and customer service and trade support chatbots.
Using a risk tiered approach aligned with the NIST AI RMF, the firm classified the highest risk systems as those affecting financial exposure, clearing obligations, or regulatory compliance. Algorithmic execution and margin and credit risk models were designated high risk because model failure could cause customer harm, market disruption, or NFA and CFTC rule violations. FX pricing and AML screening were also high risk due to quote integrity and anti money laundering obligations. Chatbots and internal support tools were classified as lower risk.
Governance Roadmap
The firm set a target of Level 3 across all five dimensions within 12 months, with a plan to reach Level 4 for high risk areas within 24 months. The roadmap prioritized completing the model inventory and assigning owners, adopting an AI acceptable use policy and model risk management addendum, deploying automated monitoring for the highest risk systems, establishing RACI accountability for AI decision points, and building an audit ready evidence repository for NFA and CFTC examinations.
120 Day Pilot Governance Sprint
The 120 day structure extends the NIST style first 90 days assessment by adding 30 days for production control deployment and formal compliance sign off, essential for a regulated FCM where changes to trading and clearing systems require change advisory board approval.
The firm selected two high priority systems for the sprint: a customer algorithmic execution model and a clearing margin and credit risk model.
| Phase | Duration | Focus |
|---|---|---|
| Baseline and inventory | Days 1 to 30 | Complete model inventory, risk classification, and gap analysis. |
| Pilot design | Days 31 to 60 | Select systems, map controls, draft RACI, and align with risk and compliance. |
| Implementation | Days 61 to 105 | Configure monitoring, document lineage, apply policy, and deploy controls via change advisory board. |
| Review and sign off | Days 106 to 120 | Validate with risk and compliance, adjust controls, and hand off to production. |
Sprint Activities
Inventory metadata was captured for both systems, including data sources, model type, owner, deployment environment, and risk tier. Formal risk assessments were conducted against the NIST AI RMF. Both systems were confirmed as high risk. A new AI acceptable use policy was drafted, and the compliance manual was updated to include AI model risk and monitoring requirements. Drift detection thresholds, data integrity checks, and model accuracy metrics were configured and deployed into production with change approval.
A RACI was applied to each decision point.
| Decision Point | Responsible | Accountable | Consulted | Informed |
|---|---|---|---|---|
| Model onboarding | Quant/Technology | Chief Information Officer | Compliance, Data Governance | Business line head |
| Risk assessment sign off | Model Risk Management | Chief Risk Officer | Legal, Compliance | Executive sponsor |
| Data access approval | Data Governance lead | Chief Data Officer | Compliance | AI team |
| Production authorization | Change advisory board | Chief Technology Officer | Risk, Compliance | Model owner |
| Incident escalation | Quant/Technology | Chief Information Officer | Legal, Compliance | Board risk committee |
Outcomes
Within 120 days, the firm moved from Level 1 to a measurable Level 2 on the governance dimensions most exposed to regulatory scrutiny, with a clear path toward Level 3. Specific results included the following:
- Model inventory coverage increased from 20 known systems to 47 documented AI and machine learning assets.
- Two high risk models were under continuous monitoring for drift, data integrity, and accuracy.
- Named owners were assigned to all high risk AI systems.
- AI deployment approval cycles shortened from weeks to days for low risk systems, while high risk systems followed a structured review path.
- The firm established an evidence repository showing control design, testing results, and remediation actions, directly supporting NFA and CFTC recordkeeping and supervision expectations.
- The additional 30 days allowed deployment of drift detection into production trading systems and completed compliance review that would have been impractical in a 90 day window.
Lessons Learned and Next Steps
Governance did not slow AI deployment at this firm, it sped it up. Once roles and risk tiers were explicit, approval for low risk systems dropped from weeks to days, and high risk systems moved through a structured, defensible review instead of an ad hoc one.
Next steps include automating controls within CI/CD pipelines, expanding continuous monitoring to all high risk systems, scheduling quarterly governance reviews, and conducting a full maturity reassessment annually. The firm is now positioned to treat AI governance as a competitive differentiator with institutional customers and a core part of its NFA and CFTC compliance posture.
