Mr Abe was rushed to hospital after the attack on Friday but he was later reported to have died.
The 67-year-old was shot while delivering a stump speech near a train station in the Japanese city of Nara. A man was arrested at the scene.
Prime Minister Anthony Albanese said Mr Abe's death was devastating and tragic and offered his sympathies to the former prime minister's family and friends, and to the people of Japan.
"Mr Abe was one of Australia's closest friends on the world stage," he said in a statement.
"It was his vision that helped elevate our bilateral relationship to a Special Strategic Partnership in 2014. Under his leadership Japan emerged as one of Australia's most like-minded partners in Asia - a legacy that endures today.
"Just a few days after being sworn in as prime minister, I had the privilege of travelling to Japan for the Quad meeting. In many ways, this grouping is a legacy of Mr Abe's diplomatic activism."
Former prime minister Tony Abbott said Japan had lost its most significant post-war leader in the shocking act of violence.
"Under Abe, Japan assumed its rightful place as the leading democracy of the western Pacific. He was the father of the Quad, a friend to Taiwan, and a staunch defender of liberal values," he tweeted.
Earlier, Japan's ambassador to Australia Yamagami Shingo said he was shocked by news of the shooting in his home country.
"Such an outrageous act should never be condoned," he said.
Several of Mr Albanese's other predecessors also showed their support, with former prime minister Scott Morrison saying he was deeply distressed by the attack.
Malcolm Turnbull called Mr Abe "one of the great leaders of our times", while fellow ex-prime minister Kevin Rudd expressed solidarity with Japan.
"An attack on any democratically elected political leader is an attack on supporters of democracy everywhere," he said.
"Shinzo Abe has been a dynamic political leader of Japan over many years."
20°C