Fire Mountain Labs

Talk · April 4, 2026

B-Sides San Diego 2026: Breaking the Black Box.

Chris M. Ward Founder · Chief Executive Officer Dr. Josh Harguess Founder · Chief Technology Officer

Originally delivered at BSides San Diego 2026 by Chris Ward and Dr. Josh Harguess of Fire Mountain Labs, this session walks security and engineering leaders through the AI governance frameworks that actually matter: NIST AI RMF, the EU AI Act, ISO 42001, GDPR, and MITRE’s AI Maturity Model

Prefer a PDF? Download the deck

Workshop Agenda

  • Intro (10 min)
  • Part 1: AI Governance Fundamentals (20 min)
  • Part 2: Tabletop Exercise (30 minutes)
  • Part 3: Wrap-Up & Q&A (5 minutes)
68% of business leaders still call AI an opportunity while the number seeing it as a risk has more than doubled; 34% of senior executives list hallucinations as a top concern; 77% of enterprises have already experienced an adversarial attack on an AI model
Balancing opportunity with unprecedented risk.

AI Governance Fundamentals

What is AI Governance?

  • “Bureaucracy and approval gates”
  • “Something only regulated industries need”
  • “A 50-page policy no one reads”

Goverwhat?

  • Most organizations lack basic AI oversight
  • No inventory, no review process, no accountability
  • Failures aren’t edge cases: they’re systemic and recurring
  • No governance = no way to control exposure, liability, or impact
Illustration: AI robots running unsupervised through an office, papers in the air and small fires burning

Common Organizational Pitfalls

Slide: six common organizational pitfalls, no clear ownership, pilot paralysis, poor data foundations, talent gaps, hidden risks, and no measurable ROI

Core AI Governance Components

  • Develop comprehensive AI policies
  • Assess and monitor AI risks
  • Ensure accountability and documentation.
  • Align with regulatory frameworks.
Illustration: a dense field of interlocking gears stamped AI, captioned AI Governance

AI Lifecycle: CRISP-ML(Q)

Slide: the CRISP-ML(Q) lifecycle, running from business and data understanding through model engineering, evaluation and deployment to monitoring and maintenance

Governance ≠ MLOps

Governance

  • Determines the what and why for responsible AI.
  • Ensures ethical, legal, and compliant AI use.
  • Directs MLOps to build trustworthy AI systems.

MLOps

  • MLOps focuses on how to deploy AI efficiently.
  • Streamlines AI lifecycle for speed and reliability.
Illustration: two meshed gears, one labelled MLOps and one labelled Governance

Governance ensures that MLOps is doing the right things, not just efficiently. They’re complementary.

Frameworks Overview

  • NIST AI Risk Management Framework (RMF)
  • EU AI Act
  • GDPR
  • ISO/IEC 42001 (AI Management Systems)
  • US AI Governance Landscape
Illustration: documents for the AI governance frameworks, NIST AI Risk Management Framework, the EU AI Act, GDPR and ISO/IEC 42001, linked to one another

NIST AI RMF Overview

  • Purpose: Voluntary, risk-based framework for trustworthy AI
  • Key Functions: GOVERN, MAP, MEASURE, MANAGE AI risks
  • Focuses on trustworthiness characteristics for reliable AI
  • Addresses unique AI risks, like data and complexity issues

EU AI Act: Key Details

  • World’s first binding legal framework for AI
  • Focuses on AI and data protection, complementing GDPR
  • Categorizes AI systems into four risk tiers
  • High-risk AI requires strict rules and assessments
  • Mandates robust risk management and data governance

GDPR & AI: Key Principles

  • Applies to AI processing EU residents’ data.
  • Reinforces strong data protection principles.
  • Mandates lawfulness, fairness, and transparency.
  • Requires data minimization and accuracy.
  • Emphasizes accountability and human oversight.
Illustration: scales of justice and a gavel outside a courthouse, a stream of data curving in behind them

ISO 42001: AI Governance Standard

  • First international AI management system standard
  • Ensures ethical and transparent AI development
  • Covers entire AI system lifecycle
  • Enhances credibility and mitigates AI risks
  • Builds trust and provides competitive advantage
Illustration: a shield stamped ISO 42001 over gears and circuit traces

2026 US AI Governance Landscape

  • Federal actions formalize NIST AI RMF for trustworthy AI.
  • States rapidly adopt AI, establishing governance structures.
  • State initiatives focus on impact assessments and responsible AI use.
Illustration: scales weighing Federal Actions against State Initiatives, labelled in deliberately garbled AI-generated text

AI Governance Ownership

  • Senior leadership sets AI governance direction and accountability.
  • Cross-functional teams ensure enterprise-wide AI policy integration.
  • Providers and deployers have distinct compliance responsibilities.
  • Organizations are accountable for third-party AI risks.
Illustration: a boardroom presentation in front of a whiteboard of unreadable AI governance diagrams

AI Approval Workflows: Key Steps

  • Executive leadership drives AI governance & accountability.
  • Cross-functional teams identify & assess AI risks early.
  • Rigorous testing verifies technical quality.
  • Human oversight & secure design principles are crucial.
  • Continuous monitoring ensures ongoing improvement & compliance.
Illustration: an operations centre of holographic AI dashboards quoting invented statistics

AI Governance: Key Takeaways

  • AI governance requires enterprise-wide accountability.
  • Adopt a proactive, risk-based AI approach.
  • Leverage frameworks for continuous AI adaptation.
Illustration: interlocking gears built from AI icons, charts and documents

Checklist: Is Your AI Governable?

  • Develop a comprehensive model inventory
  • Include Model Cards, Data Cards, and AI-BOMs
  • Conduct thorough risk analysis for AI systems
  • Gather forensic evidence for incidents
  • Implement robust logging and response mechanisms

Data Provenance

  • AI data provenance documents the data lifecycle.
  • Key aspects are origin, processing, and training.
  • It ensures transparency, accountability, and compliance.
  • Provenance manages risks like bias and privacy.
  • It is critical for trustworthiness and human oversight.

AI Bill of Materials

  • Do any of our models come from near-peer adversaries?
  • Is any of our AI built on legacy models that are no longer supported?
  • Do we have the legal right to use these models and datasets in our industry?
  • Does any of our AI ship with software vulnerabilities?
  • Where’s the complete inventory of all our models and datasets?

Governance Board Setup

  • Diverse, cross-functional experts should form the internal AI Board.
  • Establish a review cadence for AI policies.
  • The board needs authority to make and overturn decisions.
  • Senior leadership drives accountability for AI systems.
Illustration: a team around a table reviewing an AI governance presentation

Cultural Enablement

  • Governance does not equal bureaucracy.
  • Effective communication fosters good AI governance.
Illustration: a group gathered around a glowing orb labelled AI

Culture eats AI governance for breakfast. Make it easy to do the right thing.

MITRE AI Maturity Model

The MITRE AI Maturity Model: six pillars covering ethical, equitable and responsible use; strategy and resources; organization; technology enablers; data; and performance and application

Thank you

Closing slide: thank you, with a QR code to the Breaking the Black Box tabletop guide and contact addresses for both speakers

Need help running this workshop, or putting these ideas into action?

We build AI assurance programs for enterprise and government: model inventories, AI bills of materials, governance boards, and the controls that turn the frameworks above into something operational.

Talk to the people who'll do the work.

Book a consultation