Home Affairs Minister Clare O'Neil on Friday met face to face with three western Sydney mayors concerned their areas were being treated as a "dumping ground" for the former ISIS affiliates.
While scant on details due to national security concerns, Ms O'Neil said there was quite "a lot of misinformation" about where members of the group were being resettled.
"The people are coming back to where they left from," Ms O'Neil told reporters on Friday.
"We have thought about this very carefully. We have worked on this matter over a number of months."
Cabinet colleague Chris Bowen, whose federal electorate of McMahon covers some of the mayors' local government areas, defended the government's actions.
He said he only learned this year a similar resettlement had occurred in 2019 under Scott Morrison's Liberal-National government.
"I was utterly kept in the dark. The mayors were utterly kept in the dark. The community did not know," Mr Bowen said.
Liverpool Mayor Ned Mannoun, Fairfield Mayor Frank Carbone and Campbelltown Mayor George Greiss received a security briefing from Ms O'Neil and ASIO agents regarding the returnees on Friday.
"There's no doubt it was an open and honest discussion," Cr Carbone said.
"Hopefully they've learnt a lot about ... the concerns we have here in this region.
"The minister took on board ... that the real victims are the refugees, those people who actually fled ISIS."
The federal government last month repatriated four Australian women and 13 children stranded in Syria's al-Roj refugee camp since the defeat of ISIS.
None were being resettled in Fairfield, Cr Carbone said.
"It's clear to me after this discussion that western Sydney will not be used as a dumping ground," he added.
"If the families (of the returnees) are in Melbourne, they will be repatriated in Melbourne. They will be repatriated in Perth and Queensland (if that's where they travelled from)."
The talks on Friday come after the mayors told Prime Minister Anthony Albanese there were community concerns about the returnees.
Their resettlement has raised fears that refugee communities in the area, such as Yazidis and Assyrians who were targeted by ISIS, could be re-traumatised.
Cr Mannoun said many recently arrived refugees from Iraq and Syria had fled the ISIS militia.
"You would not put a victim of a crime next to the perpetrator or the person who is an accessory to the perpetrator," Cr Mannoun told AAP ahead of the meeting.
Labor minister Jason Clare echoed Mr Bowen's remarks, saying earlier that the former coalition government had also repatriated people from the camps to his Blaxland electorate, also in Sydney's west.
Earlier this week, Australian Federal Police commissioner Reece Kershaw confirmed investigations were underway into whether the returned wives had broken laws on travelling to war zones, including to Syria.
The women had co-operated with police, he said.
ISIS was ousted in 2019 from the last of the territory it held across Syria and Iraq. At its peak in 2014, the group controlled large swathes of both countries.
The violent movement attracted thousands of foreign fighters, including Australian citizens, around 50 of whom were killed in the conflicts in Syria and Iraq.
