And it's not the only defence Mr Dutton has been forced into, as a serious challenge brews in his own electorate.
Mr Dutton was courting voters at an event hosted by The West Australian on Friday when the paper's editor-in-chief Chris Dore delivered an unconventional introduction that contrasted a "match-fit, super confident" prime minister against a "punch-drunk" opposition leader.
But Mr Dutton did not take that lying down and used the comments from the Perth gathering - which included the West's owner billionaire businessman Kerry Stokes, 84 - to show he could handle challenges.
"You'll deal with all the slings and arrows and the derogatory comments and editors trying to be funny and not succeeding," he told reporters in Perth on Friday.
"That has steeled me for anything this job has thrown at me - or what could be thrown at me if I'm given the immense pleasure of being prime minister.
"I don't need to attack the character of the prime minister to win the next election ... what I want to offer the Australian people is a much more positive future."
Mr Dutton started the year with the wind at his back, driven by Australia's cost-of-living crisis amid a worldwide turn against incumbent governments.
But since the May 3 election was announced, the coalition has been bleeding support.
Fresh YouGov polling released to AAP reveals Labor has forged ahead, 52.5 per cent to 47.5 over the coalition in the two-party preferred vote.
The result is Labor's best in months and slightly higher than its polling of 52.1 per cent at the 2022 election, putting the party in pole position for a majority government rather than a widely forecast minority.
By contrast, the coalition's primary vote is now down to 33.5 per cent - lower than at the 2022 election.
Mr Dutton's work-from-home policy had sparked the fall and taken his party from "being in the box seat to win the federal election in February to struggling to hold onto the seats they won in 2022", YouGov director of public data Paul Smith said.
"The coalition's support has fallen so far that they now risk losing seats," he said.
While Mr Dutton walked back his work-from-home stance, Mr Smith said it had done "enormous damage" because voters now believed the coalition failed to understand their working lives or support people's workplace rights.
"There have been only two prime ministers who have lost their seats - John Howard and Stanley Melbourne Bruce - and that was because they went against Australians' rights at work," he said.
Mr Dutton currently holds the north Brisbane electorate of Dickson on a 1.7 per cent margin, but both Labor and an independent are hoping to seize on his constituents' discontent and push him out.
"I don't like him, he's slimy, he's not honest, he looks after the big cats," Petrie local Brett Middlebrook told AAP
"He does nothing for the Australian worker - that's what he's all about, that's what that party's all about."
Meanwhile, Prime Minister Anthony Albanese tried to capitalise on the polling momentum and lean into Labor's perceived strengths on Friday when he visited a Northern Territory urgent care clinic to reveal $60 million for an aged care program and $10 million for CareFlight.
Mr Albanese said Labor was still eyeing a majority government.
"I want people to get that pencil and the ballot paper and put a 'one' next to their Labor candidate," he told reporters in Darwin.
"That is the way you elect a majority Labor government, that's my objective, that's what we're aiming for."
The YouGov poll of 1515 people was carried out between April 4 to 10, with a 3.3 per cent margin of error.
16°C