Mr Lehrmann spoke to the Seven Network's Spotlight program on Sunday night, two years after the accusations against him were made public, saying it "simply never happened".
Ms Higgins alleged Mr Lehrmann raped her inside a Parliament House office of then coalition defence industry minister Linda Reynolds, who they both worked for.
Ms Higgins alleged this happened after a night out drinking with colleagues in February 2019. While she initially decided not to go ahead with the complaint and kept working as a Liberal adviser, Ms Higgins asked police to re-open the case in 2021.
She told her story to news.com.au journalist Samantha Maiden and Network 10 journalist Lisa Wilkinson, sparking multiple reviews and eventually an apology from former prime minister Scott Morrison.
On Sunday night, Mr Lehrmann described watching Ms Higgins out him on The Project with "utter outrage" as he watched friends and colleagues block and unfollow him on social media as he watched the program.
He said while he wasn't named, it was obvious he was the one being accused of rape.
"(It was) one of the worst things to see, because I was politically inconvenient to be associated with," Mr Lehrmann said.
"If that's not a clear indication of being identified, without being named formally, I don't know what is.
"Straight away it was like a nuclear bomb going off or your world exploding."
He also denied a witness's report they saw the pair kissing on a couch at a nightclub in Canberra during the night out.
He explained the pair decided to head back to Parliament House because he needed to pick up his apartment keys from his office.
"From my recollection she indicated she also needed to go back to Parliament House - who am I to question that," he said.
"I thought I was being a gentleman in assisting her to do that."
He did agree with Ms Higgins on one thing, that politics was grubby, and maintained he had left politics before the accusations became public because the "shine had worn off" and denied claims he had aspirations to become a politician.
"I had formed a view that ... in many respects, that building is not the best place to work. The culture, irrespective of gender, I mean, 16 hour days ... the shine of it wore off, he said.
"I left because of both sides of politics. It wasn't for me."
Mr Lehrmann faced a criminal trial in the ACT Supreme Court in October 2022, but juror misconduct meant a verdict was never reached.
Prosecutors later dropped the sexual assault charge against him due to fears of the impact a second trial would have on Ms Higgins mental health.
Mr Lehrmann told Seven the allegations made him contemplate taking his own life before seeking professional help for his mental health.
"It all just became too much. I felt I couldn't recover," he said.
"I made arrangements to go. Mum would be okay to get my superannuation. I'll make it easy for everyone. There was no coming back."
Mr Lehrmann launched defamation proceedings against News Life Media, Network 10 and the ABC over their reports of Ms Higgins' allegations.
He reached a settlement with News, but cases against the other two media organisations remain ongoing.
The ACT government also launched an inquiry into how the justice system responded to Ms Higgins' allegations.
A report is being prepared and will be delivered to Chief Minister Andrew Barr by the end of July.
Lifeline 13 11 14
beyondblue 1300 22 4636
