An audit launched by Liberal senator James Paterson found 1000 surveillance cameras and other recording devices, some of which have been banned in the United States and Britain, had been installed across government buildings.
Mr Marles said the government was assessing surveillance technology in use.
"We're doing an assessment of all the technology for surveillance within the defence estate and where those particular cameras are found they'll be removed," he told ABC's Radio National on Thursday.
Asked how widespread the issue was, Mr Marles said it "shouldn't be overstated".
"It's a significant thing that's been brought to our attention and we're going to fix it," he said.
"It's important that we go through this exercise and make sure that our facilities are completely secure."
Asked if there was concern the removal of the cameras would negatively impact the relationship between the two countries, Prime Minister Anthony Albanese said "no".
"We act in accordance with Australia's national interest. We do so transparently. That's what we'll continue to do," he told reporters in Canberra.
Senator Paterson told reporters he was encouraged by Mr Marles' comments as it was a "serious issue of national security".
"We have no way of knowing whether images, audio or other data collected by these devices are being sent back to China and handed over to Chinese intelligence agencies," he said.
"But I also have a moral concern. These companies have been implicated in what the United Nations has called crimes against humanity, what others call genocide, against the Uighur people in Xinjiang, and I don't think any Australian taxpayer dollars should be going to companies involved in these things."
Senator Paterson first requested the information from the home affairs department in September, at which time he was told there was no way of knowing how many of the cameras were in operation across government.
25°C