An audit launched by Liberal senator James Paterson found almost 1000 surveillance cameras and other recording devices - some of them banned by the United States and Britain - installed across government buildings.
Defence Minister Richard Marles is investigating where the devices are in use.
"We're doing an assessment of all the technology for surveillance within the defence estate and where those particular cameras are found they'll be removed," he said.
Mr Marles said the issue was important but should not be overstated.
"It's a significant thing that's been brought to our attention and we're going to fix it," he said.
"It's important that we go through this exercise and make sure that our facilities are completely secure."
Prime Minister Anthony Albanese denied removing the cameras would harm the Australia-China relationship.
"We act in accordance with Australia's national interest. We do so transparently. That's what we'll continue to do," he told reporters in Canberra.
Senator Paterson said the spread of Chinese surveillance devices was a serious issue of national security.
"We have no way of knowing whether images, audio or other data collected by these devices are being sent back to China and handed over to Chinese intelligence agencies," he said.
The Liberal senator said the devices also posed a moral concern.
"These companies have been implicated in what the United Nations has called crimes against humanity, what others call genocide, against the Uighur people in Xinjiang, and I don't think any Australian taxpayer dollars should be going to companies involved in these things," he said.
Senator Paterson first requested the information from the home affairs department in September, at which time he was told there was no way of knowing how many of the cameras were in operation across government.
Australian War Memorial chairman Kim Beazley this week confirmed several Chinese-made security cameras would be removed from the shrine out of an "abundance of caution".
The cameras were manufactured by Hikvision, which is part-owned by the Chinese government and is one of the world's largest CCTV suppliers.
Eleven devices will be removed from the memorial by July.