The ex-Liberal MP, who lost his West Australian seat at the last election, said he had taken a report to cabinet on two separate occasions when he was in office on what an Indigenous voice would look like.
Mr Wyatt said the argument by Nationals MPs that they did not have enough detail about the voice was being used as an excuse for not backing the referendum.
"To my mind, it offers up a level of immaturity around a very complex issue," he told ABC Radio on Wednesday.
But Nationals leader David Littleproud defended the move, saying the party wanted to "close the gap" and didn't believe the voice would have a tangible impact on the ground.
"Let's take all the vitriol out of this. We want to close the gap," he said adding the voice would only add another layer of bureaucracy.
"We think the way to close the gap is to empower these local communities."
Some Nationals, including the party's WA branch and federal MP Andrew Gee, have already broken rank with the federal party saying they would campaign for the voice.
Mr Littleproud backed their decision to do so, saying people were free to disagree.
"That's hardly earth-shattering in the Nationals. Our party room is a diverse one," he said.
But Mr Wyatt said he was extremely disappointed with the party's stance.
"I challenge every federal member to get out of their offices, go to the Aboriginal organisations within their electorates, sit and listen to the issues, see firsthand what Aboriginal people are talking about," he said.
The 2022 Closing the Gap report to be handed down in parliament on Wednesday will show many of key targets are not on track.
It will be the first report since the national agreement on Closing the Gap took effect.
In 2020, an agreement between the federal government, the Coalition of Peaks, all state and territory governments and the Australian Local Government Association (ALGA) was struck.
It aimed to renew ways of working together to improve outcomes for Indigenous Australians.
The groups agreed to improve 18 socio-economic outcomes across health, education, employment, housing, justice, safety, land and waters, culture, language and connectivity.
But two years later only four of the targets are currently on track, while four are getting worse and others have insufficient data to assess their progress.
Worsening targets include the number of children who are school-ready, adult incarcerations, children in out-of-home care and deaths by suicide.
Targets currently on track are the number of babies born at a healthy weight and children enrolled in preschool.
The Minister for Indigenous Australians, Linda Burney, said the latest annual report told a story of mixed success, and that it was disappointing to see a lack of progress in a number of areas.
"The Closing the Gap architecture can only work when all parties are invested and there is a coordinated effort from all jurisdictions in partnership with First Nations peoples," she said.
"We have to work more closely with Aboriginal and Torres Strait Islander people to make real and much-needed progress."
Lifeline 13 11 14
beyondblue 1300 22 4636
24°C