Instead of spending millions on consultants to check public programs are working, the Labor government is rolling out an in-house model it believes will be more effective.
Assistant Minister for Competition, Charities and Treasury Andrew Leigh set out his plan in an address to the National Press Club on Tuesday.
In 2021/22 the Commonwealth spent $52 million on evaluations by private consultants, which Dr Leigh reckons is likely a conservative estimate.
In his speech, he said consultants were not particularly well-incentivised to undertake high-quality evaluations.
That's because it's very difficult to actually improve social programs and a well-designed evaluation is more likely than not to reveal shortcomings in new policy interventions - a phenomenon known as Rossi's Law, as coined by sociologist Peter Rossi.
"The fact that failure is more common than success does not suggest that program designers are foolish or careless, but that they're grappling with problems that are really, really difficult," Dr Leigh said.
Based on Rossi's Law, a well-designed evaluation is actually unlikely to show a program is working.
"Which may make it harder for (consultants) to win the next contract," he said.
He said that was one of the reasons the government had set up the Commonwealth Evaluation Unit, a Labor government election promise.
The unit, which will be available as a resource for agencies, received $10 million in the May budget but is intended to save millions more in private consultancy fees.
Asked how vulnerable the unit would be to policymakers gaming the system, Dr Leigh said the body would be "more doctor than cop".
"It's looking to improve the health of the evaluation ecosystem."
With the unit in place, he said norms would start to shift but this would take time.
"You're able to move people from a culture that says 'I can come up with a bright idea and implement it in the world', to an approach that says 'I need to test my ideas, I'm not that sure what will work and I'm willing to produce new ideas if these ones fail'," he said.
Evaluation techniques such as randomised trials have been used to fine-tune public policy to deliver better outcomes around the world.
In Africa, for example, some economists argued that free anti-malarial bed nets would not be valued by villagers. But a randomised trial found the free bed nets were more popular and just as likely to be used, prompting the World Health Organisation to change its policy.
"The results of the experiment save thousands of lives every year," Dr Leigh said.
