Skip to main content

Why AI Governance Starts with Your Data

You can't govern AI outputs without governing data inputs. A practical framework for data governance as the foundation of responsible AI.
8 September 2024·7 min read
Dr Tania Wolfgramm
Dr Tania Wolfgramm
Chief Research Officer
compliancedataai-strategy
Most AI governance frameworks start with the AI: model risk, output monitoring, bias detection. That's starting at the wrong end. If you can't account for what goes into the system, you can't meaningfully govern what comes out. Data governance isn't a prerequisite for AI governance. It is AI governance.

What You Need to Know

  • AI governance without data governance is theatre. You can't audit AI decisions if you can't trace them back to the data that informed them.
  • Data lineage is the foundation of AI explainability. When a regulator or customer asks "why did the AI make this decision?", you need to show the data path, not just the model architecture.
  • Most organisations have data governance policies. Few enforce them. The gap between policy and practice is where AI risk lives.
  • Start with the data your AI actually uses, not a boil-the-ocean data governance programme. Scope it to the specific data sources feeding your AI systems.
61%
of organisations with AI governance policies report they cannot trace AI decisions back to source data
Source: MIT Sloan Management Review, 2024

The Governance Gap

Here's what typically happens. An organisation decides to deploy AI responsibly. They write an AI ethics policy. They establish an AI governance committee. They create review processes for new AI deployments.
All good steps. But when you ask "Can you show me exactly which data sources fed this AI decision, when that data was last validated, and who has access to modify it?", the room goes quiet.
The governance gap isn't at the AI layer. It's at the data layer. And no amount of AI-layer governance compensates for it.

Why Data Governance Is AI Governance

Bias Enters Through Data

AI bias is a data problem first and a model problem second. A recruitment AI trained on historical hiring data inherits every bias in that history. A credit scoring model trained on data from a period of discriminatory lending practices reproduces those practices.
Governing the model (testing for bias in outputs) is necessary but insufficient. You also need to govern the data: understand its provenance, its limitations, its historical context, and its representativeness.

Compliance Requires Traceability

The EU AI Act, Australia's emerging AI framework, and New Zealand's Algorithm Charter all point in the same direction: organisations must be able to explain AI decisions. Explanation requires traceability. Traceability requires data lineage.
When an affected person asks "Why was my application declined?", the answer needs to trace from the decision, through the model's reasoning, to the specific data points that influenced the outcome. Without data governance, that chain breaks at the first link.

Data Quality Determines AI Quality

We've written about this before, but it bears repeating in the governance context. Poor data quality isn't just a performance issue. It's a governance risk. An AI system making decisions based on outdated, incomplete, or incorrect data isn't just inaccurate. It's ungovernable, because you can't distinguish between a model error and a data error.
82%
of AI incidents in the AIAAIC Repository involve data-related root causes
Source: AI, Algorithmic, and Automation Incidents and Controversies Repository, 2024

A Practical Framework

You don't need a perfect data governance programme before you deploy AI. You need fit-for-purpose data governance for the specific data your AI uses.

Step 1: Map Your AI Data Sources

For each AI capability, document every data source: databases, document stores, APIs, third-party data, and user inputs. This is your AI data inventory.

Step 2: Assess Each Source

For each data source, answer:
  • Provenance: Where does this data come from? How was it collected? What consent was obtained?
  • Quality: How complete, accurate, and current is it? When was it last validated?
  • Access: Who can read and modify it? Are access controls enforced?
  • Sensitivity: Does it contain personal information, cultural information, or commercially sensitive data?
  • Lineage: Can you trace a specific AI output back to the specific data that informed it?

Step 3: Close the Gaps

Prioritise gaps by risk. Personal data without clear consent? High risk, fix immediately. Stale reference data? Medium risk, schedule a refresh cycle. Missing access logs? Low-to-medium risk, add monitoring.

Step 4: Build Ongoing Monitoring

Data governance isn't a one-time exercise. Data changes. Sources are added. Quality degrades. Build regular review cycles into your AI operations, monthly at minimum for production systems.
The Lineage Test
Pick any AI output from your production system. Can you trace it back to the specific data sources that informed it within 30 minutes? If not, your data governance isn't ready for AI at scale.

What This Looks Like in Practice

In our governance work with enterprise clients, we've found that the data governance conversation surfaces issues that the AI governance conversation misses. A knowledge base AI that retrieves policy documents sounds low-risk until you discover that the document repository contains three versions of the same policy with no clear indication of which is current. A claims processing AI seems well-governed until you realise the training data includes claims from a period when a particular demographic was systematically under-served.
These aren't model problems. They're data problems. And they're only visible when you govern the data layer with the same rigour you apply to the AI layer.
The AI governance frameworks that actually work share one characteristic: they start with data. If you can't govern the data, you can't govern the AI.
Dr Tania Wolfgramm
Chief Research Officer