Shared Test Harnesses: A Government Approach

There’s a term floating around the AI world called P(doom). Basically, it’s someone’s estimate of the chance that advanced AI eventually causes a catastrophic outcome, possibly even something as extreme as human extinction. The numbers are all over the place. Some people think the risk is tiny, while others think it’s frighteningly high. I find the debate interesting, but I also think we’re spending a lot of time arguing over a number nobody can actually prove.

From predictions to practical tests

Whatever your P(doom) is, we still have the same practical problem: how do we know whether the next generation of AI models is safe enough to release? My answer is pretty simple. Test them.

If a company wants to release one of the world’s most powerful AI models in Australia, then it should have to provide its safety framework, its testing results and controlled access to an independent testing body. Australia would have a clear set of safety tests that the model needs to pass. If it passes, great, release it. If it fails, fix the issue and try again.

I’m not suggesting we do this for every little AI application or every Australian startup using a language model. That would be a nightmare and would probably hurt innovation more than it helped. This should only apply to the most powerful frontier models, where the potential risks are much higher and where a failure could have much broader consequences.

The interesting thing is that we’re already halfway there. Australia now has an AI Safety Institute, and its role includes looking at emerging AI models, testing them and helping government understand the risks. The big AI companies are also already doing their own safety testing, including red teaming, capability testing and looking for dangerous behaviours. So the pieces already exist. What I think is missing is a proper shared testing system.

Share the tests, strengthen the evidence

This is where the idea gets really interesting. Let’s say Australia develops 100 tests for a new frontier model. Canada finds a new way to test whether an AI can carry out sophisticated cyberattacks. Britain comes up with a better way to test deceptive behaviour. Japan discovers a new weakness, and researchers in Europe find a jailbreak nobody had thought of. Australia doesn’t need to reinvent any of that. It simply adds those tests to its own harness.

Now Australia has 104 tests. Then someone finds another problem, and now it has 105. Another country improves one of the existing tests, and now the test harness gets stronger again. Over time, the system becomes harder and harder to fool because every country is contributing what it learns.

That, to me, makes far more sense than every country trying to write huge amounts of legislation predicting what AI might do in the future. AI is moving too quickly for regulation to keep up in the traditional way. A law can take years to design and pass. A new AI model can appear in weeks. A shared test harness can evolve much faster.

Australia is already working with other countries on AI safety and evaluation, so this isn’t some wild idea with no foundation. The relationships are already there. What we need is a stronger mechanism for sharing the actual tests, methods and lessons so each country benefits from what the others discover.

It also gives us a way out of the two extremes we seem to hear all the time. On one side, there are people who think we should simply trust the AI companies to sort it out themselves. On the other side, there are people who want to regulate AI so heavily that we risk slowing it down, blocking access and hurting innovation. There has to be a better middle ground, and I think shared testing could be part of it.

Independent evaluation, protected findings

Of course, there are problems to solve. Testing can’t take six months when new models are coming out every few weeks. It can’t cost so much that only the biggest companies in the world can afford it. Governments also shouldn’t be the only ones deciding what “safe” means. The testing should involve government, independent organisations, researchers and industry, with enough separation to make sure nobody is marking their own homework.

I also don’t think all of this information should be made public. If researchers discover a serious vulnerability, publishing every detail could actually make things worse. Some information should remain confidential and only be shared with trusted organisations. The important thing is that the model is being independently tested, rather than everyone simply taking the developer’s word for it.

Retest as capabilities and risks change

Passing the test once also shouldn’t mean passing forever. AI systems change. New vulnerabilities are discovered. Models get connected to tools, databases and autonomous agents. Something that looked safe six months ago might look very different today. The testing system itself needs to keep changing, and models may need to be retested when major capabilities or risks emerge.

This is also why I think the whole P(doom) debate becomes more useful when we move away from hypothetical percentages and towards things we can actually measure. Can the model carry out advanced cyberattacks? Can its safeguards be bypassed? Can it help someone do something genuinely dangerous? Can an autonomous AI hide what it is doing, manipulate people or take actions without meaningful human oversight? These are much more useful questions because they can actually be tested.

Global knowledge, accountable decisions

That’s really the key idea. We don’t need one government claiming it has solved AI safety. We need a system where governments, researchers, universities and AI companies are constantly finding weaknesses, building new tests and sharing what they learn. Australia could start with its own test harness, Canada could have one, Britain could have one, Japan could have one, and over time those systems start sharing more and more with each other.

The result is that the tests get better, the models get tested harder, and hopefully the safety improves without putting a giant handbrake on innovation. The technology is global, so the safety knowledge should be global too.

Instead of asking an AI company, “Is your latest model safe?”, maybe the better question is simply this: show us it passed the tests.