The partnership between the three countries will see a new air missile defence network developed as part of efforts to deter Chinese aggression in the Indo-Pacific.
The pact was announced during a joint press conference in Washington between US President Joe Biden and Japanese Prime Minister Fumio Kishida.
"Today we announce our vision to co-operate on a networked air defence architecture among the United States, Japan, and Australia to counter growing air and missile threats," the two leaders said in a joint statement.
"We will explore enhanced co-operation, including missile defence information sharing to counter growing air and missile threats."
Prime Minister Anthony Albanese said the agreement would see closer collaboration between the three countries.
"This is a deal between Australia, Japan and the United States to enhance our capability. We want to cooperate with countries in the region in particular," he told Nine's Today program on Thursday.
"It's about increasing our defence capability, and that's in Australia's national interest."
However, Mr Albanese said while the agreement came following rising tensions in the Indo-Pacific due to China, positive talks with Australia's largest trading partner were still ongoing.
"We'll always engage in a constructive manner, and that's what we're continuing to do," he said.
"This is about our national defence, and all nations have a right to ensure that their defence capability is maximised, that's precisely what Australia is doing."
The missile agreement comes after the prime minister described Japan joining AUKUS under a technology sharing arrangement was a natural evolution of the trilateral security pact.
While Japan will not become a fourth member of AUKUS alongside Australia, the US and UK, the three countries have said they are considering working with Japan as part of advanced capability projects.