What is Causal Verification in AI Governance?
Causal verification tests whether a pattern in decisions, such as approving cases a rule would decline, actually changed outcomes, after accounting for the other factors that influence them. It separates departures that did harm from ones that did not.
How Studio does it
Studio estimates the effect with DoWhy, adjusting for up to five covariates that predict the outcome, requires a sample large enough for the observed effect, corrects for testing many rules at once (Benjamini-Hochberg, q ≤ 0.10), and runs refutation tests (placebo, random common cause, data subset). A result that does not meet those tests is reported as inconclusive.
Realised outcomes
When per-decision outcomes are uploaded, such as defaults or charge-offs joined to decisions, the estimate uses what actually happened rather than a proxy, provided at least 20 decisions are joined to outcomes.
Example
Officers approve some applications above a debt-burden limit.
The exceptions are judged by opinion: some think they are prudent, others reckless.
The estimate shows whether the exceptions changed default rates, with its uncertainty, and whether it survived the refutation tests.
Frequently asked questions
Related concepts
See it in practice
Upload a policy, confirm its rules, seal them: any system can then check an action against them.