The inquiry announced on Thursday excludes an investigation of any "unilateral" state and territory decisions.
This means state-imposed lockdowns, school closures and mask mandates are not expected to fall under the spotlight.
Opposition Leader Peter Dutton has claimed Labor premiers are being shielded from scrutiny, saying the inquiry should be able to compel states and territories to participate.
Speaking at the Victorian Liberal party's state council meeting on Saturday, he said the Albanese government was running a "protection racket" for Victorian Premier Daniel Andrews and Queensland Premier Annastacia Palaszczuk.
"That the prime minister of our country would side with Daniel Andrews and Annastacia Palaszczuk over the people of Victoria, or indeed the people of our country, is a shameful act," he said.
"If there is to be a pandemic into the future, we should and we deserve to know the answers as to what went right and what went wrong."
The opposition leader noted that Victoria recorded more deaths and more time under lockdown than any other part of the country.
Health and childcare experts want the inquiry to examine the pandemic's impact on children, notably school shutdowns.
National Children's Commissioner Anne Hollonds said the inquiry should "put a spotlight" on the needs of kids as the country grapples with youth mental health, school refusal and academic performance.
"School is about much more than just academic learning that can be replaced by Zoom, and that social learning environment was absent," she told ABC News on Saturday.
Ms Hollonds said she would be personally writing to the inquiry to recommend it learn from how other countries cared for their children during the pandemic.
"I'll be recommending that perhaps we should look at a child wellbeing council of multidisciplinary experts who could provide advice to governments at critical times like this," she said.
"And ideally, that there be a minister for children whose whose accountability it is to pay attention to the unique needs of children and young people."
NSW Premier Chris Minns said extensive data about the impact of COVID on learning and education over the last two years was available and remedial action for children who were left behind was needed.
A final report by an inquiry panel will be handed down by September 30, 2024.
