By hodo_admin
The Footnote That Broke a Policy
South Africa’s Department of Communications and Digital Technologies had, by all accounts, produced something historic. After months of drafting, the country had finalised a national AI policy that would establish a National AI Commission, an AI Ethics Board, a National AI Safety Institute, an AI Ombudsperson, and an AI Regulatory Authority. On Cabinet approval, the Government Gazette published it on 10th April. It would have made South Africa the first African nation to formally place AI oversight inside a legal and institutional framework.
Everything was progressing smoothly – until a journalist checked the footnotes. Six of the document’s 67 academic citations led nowhere. The journals named were real, and the researchers credited with the work were real people. But the papers themselves had never existed.
The Parliamentary Portfolio Committee Chair’s response sums up the general reception of this debacle succinctly – for them to “skip using ChatGPT this time” when redrafting. What makes this moment particularly vertiginous and slightly comical is the subject matter here. At its core, this was a policy for governing AI, built partly with AI, containing AI-generated citations that AI could not catch, reviewed by professionals who didn’t seem to catch them either. The recursive quality of the irony is hard to shake, the ouroboros quality of it is almost too neat to be real, but unfortunately it is. And it’s more rampant in the exalted offices than we think.
This is Not a South Africa Story
The South Africa incident made headlines because it was dramatic; an historic document, a continent-first juncture, undone by a computerised model. But the truth is that across governments and high-stakes consulting engagements, the same failure has appeared with unsettling regularity in the past year alone:
Australia and Deloitte (August 2025): Academics flagged suspected AI use in a Deloitte report commissioned by Australia’s Department of Employment and Workplace Relations, alleging fabricated academic references and invented quotes. Deloitte confirmed generative AI had produced inaccurate outputs. The corrected report was republished. The firm refunded $290,000 of its $440,000 fee. A similar problem surfaced months later in Canada in another Deloitte government engagement, suggesting this was not an isolated failure.
ENISA, the EU’s cybersecurity agency: Europe’s apex body for cyber threat intelligence admitted that two of its 2025 threat reports were laced with hallucinated sources. In one report alone, 26 of 492 footnotes were incorrect. Chiara Gallese, an AI law and ethics researcher, put it with some precision: “ENISA let AI touch the one layer it should never touch unguarded: the truth layer.”
These are the bodies governments trust to tell them where the dangers are, behaving like fringe actors or underfunded agencies. If they can’t verify their own sources, isn’t the warning system itself compromised?
Why AI Keeps ‘Hallucinating’
Understanding this requires dispelling a common misconception of how AI functions. Large language models do not have the ability to retrieve information or to look things up like a search engine does. When you ask one to write a policy document with academic citations, it generates text by predicting the most statistically probable next token given everything that preceded it. Because the model knows what that genre looks like, it can construct plausible details in plausible citation formats. And so fake citation, or data, arrives dressed in the same lexical clothing as the real one. This is what researchers like Megana Nataranjan mean when they call it confabulation rather than hallucination. The model cannot be lying because lying requires knowing the truth.
Governments Going Full Speed, No Map
It is clear that no governing body or public-facing institution possesses the will – or, frankly, the way – to simply stop. The answer to confabulation might not even be as simple as abstinence.
New Zealand’s Parliamentary Counsel Office, the body that drafts the country’s laws, ran a proof of concept using AI to generate first-draft explanatory notes for legislation. Policy staff across Canada, the US, the UAE, and parts of Europe are already using AI as a drafting partner in some form. This is happening whether or not there are guidelines for it, and whether or not the people doing it have thought carefully about where the tool fails.
The harder problem is that in most large organisations, there is no way to know where AI touched a document and where it didn’t. A policy passes through fifteen hands over three months. Anyone in the middle can run a section through ChatGPT. Nobody flags it for review, because surely someone else will if it’s needed. By the time the minister signs, the provenance of every paragraph is effectively invisible and individual accountability close to impossible.
Estonia is the country that seems to have deliberated the most careful balance. It amended its core legislative drafting rules so that every time a new law is prepared, policymakers are now formally required to assess whether AI or automation could be part of the process. That requirement took effect in May 2026. The key word is “part.” Estonian policy treats AI as an early-stage generative tool that surfaces options and builds rough drafts that humans then interrogate, verify, and deliberate over.
India has perhaps more riding on this than most. The IndiaAI Mission, approved with an outlay of ₹10,371 crore, stands as one of the most ambitious sovereign AI programmes in the Global South. India hosted a major AI governance summit in February 2026. The country is building frameworks at considerable speed, precisely because the opportunity is real and the window appears mercurial. But the South Africa case, and others before it, should be a direct admonition to any country in this position. The IndiaAI Mission is one of the most watched governance experiments in the world right now. There’s no question about whether India will use AI to build its frameworks. It will. We need to hope that it builds the infrastructure to verify before the first gazette goes out.
Across organisations globally, 75% report using AI tools. Only 36% have formal policies governing that use. Considerably more than half are using it but less than half have thought through the rules for it, making the adoption-governance gap tangible. Unfortunately, most governments are not Estonia. Most governments are Deloitte, and show no signs of stopping.
The Trust Deficit
Across 11 of 28 countries in Edelman’s latest trust barometer, governments are already more distrusted than trusted as upstanding, effective institutions. There is a strong air of distrust for governments as competent stewards of much of anything. In the United States, only 31% of people trust their government to regulate AI effectively. In the UK, just 29% of citizens trust their government to use AI accurately and fairly.
Meanwhile, three in four people globally say they want regulation to take precedence over unfettered AI innovation. Which is interesting. The public is asking for guardrails, asking loudly, while also doubting that the people meant to build those guardrails can be trusted with the task.. That it’s a reasonable read of the evidence so far. Every incident pulls the trust deficit wider and consumes whatever remaining credibility those governments had.
The Word Nobody is Using : Assurance Literacy
There is a vital part of the conversation that only exists in a nascent form. And that isn’t better AI or stricter bans. It’s a clearer account of what humans need to be able to do when AI is irrefutably inside the workflow.
Here is where assurance literacy comes into play, a term that sits outside mainstream discourse and that lives mostly in research circles and policy papers . Not AI literacy in the generic sense of knowing what a large language model is, but the specific operational knowledge of when to trust AI output, when to verify it, and how. The understanding that confident-sounding prose is not evidence of accuracy and a well-formatted citation is not evidence of a real paper. That no matter how authoritative a document looks, its foundations need to be checked.
The Estonian model points at the answer: AI as a first-draft mechanism, and human expertise as the verification layer. The tool’s fluency, here, is treated as a starting point and nothing more, taken with a pinch of salt. The truth layer — factual claims, citations, empirical grounding — is the one place where AI’s characteristic felicity turns into a liability. Human sentience and EQ are core requirements in this layer.
Assurance literacy is that skill, made institutional. This is a very learnable skill. It is also, evidently, not yet a requirement. And until it is, things aren’t going to change.
Who is Governing Whom?
This is a slightly dizzying question buried in all of this that deserves to sit with the reader for a moment. If the institutions writing the rules for AI cannot reliably verify what AI produces — if they lack the assurance literacy to catch a fabricated citation before it reaches a government gazette — then the governance relationship is inverted. If the people writing the rules can’t reliably verify what the tool produces, then in some fugacious but real sense, the tool is shaping the rulebook more than the rulemakers are. That is a rather strange arrangement to have arrived at so quickly and quietly.
South Africa’s minister said this “should not have happened”. He was right. But it did happen, in Australia, and again in Canada, and inside the EU’s own cybersecurity apparatus. At some point the question stops being whether this was a lapse and starts being whether the institutions now racing to govern AI can be equipped to do so. The evidence, so far, suggests not.
