What is Explainable AI (XAI)? Explainability in Governance
Explainable AI (XAI) is the set of methods that make an AI system's outputs understandable to people. For a regulated decision, the explanation that counts is specific to the case: what decided it, and on which facts.
Global and local explanations
Global methods describe how a model behaves overall; local methods, such as SHAP or LIME, estimate which inputs mattered for one prediction. Both describe a model; neither is the rule a decision followed.
Explanation by rule
When a decision is taken by sealed rules, its explanation is exact: the rule, the facts that met or failed it, and the source of the rule. The EU AI Act requires high-risk systems to be transparent enough for deployers to interpret their output and use it appropriately (Article 13), and oversight requires people to understand what they are overseeing (Article 14).
Example
A loan is declined.
The explanation says income was the most important factor at 42%, followed by credit history at 31%.
The explanation says: debt-to-income 48% (limit 50%, met); employment 8 months (minimum 12, not met); declined under rule EMP-03, section 3.1 of the credit policy.
Frequently asked questions
Related concepts
See it in practice
Upload a policy, confirm its rules, seal them: any system can then check an action against them.