Governance is treated in most Australian enterprises as a compliance exercise. A policy is written, a committee is stood up, a register is maintained, and the assumption is that the AI system underneath will bend itself to fit. That assumption is wrong. Governance is an architectural property. If it is not designed into the system, it cannot be added later without rebuilding the parts that matter.
This piece sets out a working reference model for AI governance that has been used across regulated Australian environments, including federal government, financial services and critical infrastructure. It is deliberately practical. The goal is a system a board can defend, an auditor can inspect, and an engineering team can actually operate.
## Start with the decision, not the model
The unit of governance is the decision the AI system produces, not the model that produces it. A recommendation to approve a loan, flag a transaction, prioritise a case, or route a citizen enquiry is the thing that has real world consequences. Everything in the reference model hangs off that decision.
For every decision class the system emits, four things must be recorded and queryable:
1. The inputs the model saw, including any retrieved context. 2. The output it produced, with the confidence signal it used. 3. The version of the model, prompt, retrieval index and business rules in force at the time. 4. The human or automated action taken as a result.
If any of the four is missing, the decision is not governable. It can still be made, but it cannot be reviewed, reproduced or defended.
## Five layers that carry governance
The reference model has five layers. Each has an owner, a set of controls, and a small number of measurable properties.
**Data layer.** Sources are catalogued with provenance, sensitivity classification and permitted use. Personally identifiable data is separated from training and evaluation corpora unless a lawful basis is documented. Access to production data by developers is time bound and logged. The measurable property is coverage: what proportion of the data feeding the system has a documented owner and lawful basis.
**Model layer.** Every model in production has a model card that names the training regime, the eval regime, known failure modes, and the human sign off that promoted it. Third party models are treated as suppliers, not commodities, and are versioned inside your own registry with a hash you control. The measurable property is traceability: given any production decision, can you name the exact model artifact that produced it.
**Prompt and retrieval layer.** Prompts and retrieval configurations are code. They live in version control, they are reviewed, and they are deployed through the same pipeline as the model. Ad hoc prompt changes in production are the single most common source of silent behavioural drift I have observed in Australian deployments. The measurable property is change control: what percentage of prompt and retrieval changes in the last quarter went through review.
**Decision layer.** Business rules that wrap the model, thresholds that convert probabilities into actions, and human in the loop workflows sit here. This is where regulatory obligations bite. A model that produces a score is not making a decision. The rule that turns the score into a refusal, an approval or an escalation is. The measurable property is explainability: for any decision, can a suitably trained reviewer reconstruct why it was taken in under fifteen minutes.
**Assurance layer.** Continuous evaluation, red team exercises, incident response, and periodic external review live here. The measurable property is dwell time: how long between a material behavioural change and the moment your assurance function detects it.
## Roles that keep the model honest
Three roles are non negotiable. The model owner is accountable for the technical behaviour of the system. The decision owner is accountable for the business outcomes of the system. The assurance owner is accountable for detecting when either of the first two is wrong. In smaller organisations one person can hold two of these roles, but never all three. If a single person owns model, decision and assurance, there is no independent check and the governance function is theatre.
## What to build first
Most Australian enterprises I work with already have a governance policy and a model inventory. What they do not have, in almost every case, is a decision log with the four fields above, and a working assurance layer that can detect behavioural drift within a defined window. Those are the two things to build first. Everything else is easier once decisions are captured and drift is visible.
The reference model is not exotic. It is the minimum viable structure for running AI in an environment where the regulator, the board or the public may reasonably ask you to explain a specific decision. If you cannot answer that question today, the governance you have on paper is not the governance you actually operate.
