NT Coroner Elisabeth Armitage has been investigating Mr Walker's death since September 2022 and was set to resume her inquiry on October 23.
Mr Rolfe shot the 19-year-old three times while on duty in the remote community of Yuendumu but was acquitted of murder in a five-week trial last year.
Mr Rolfe's lawyer, Luke Officer, filed a submission two weeks out from the resumption of the inquest asking the coroner to consider recusing herself on the grounds of apprehended bias, which has compromised the impartiality of the inquest.
Ms Armitage dismissed the application on Wednesday, saying she would not be stepping aside.
"I am not persuaded that a fair-minded lay observer might reasonably apprehend that I might not bring an impartial mind to the resolution of the issues arising," she said.
"Accordingly, I decline to recuse myself from proceeding with the inquest."
In his application, Mr Rolfe's lawyer said the coroner had colluded with NT police to get his client sacked earlier in the year and the way she engaged with the family of Mr Walker was unnecessary.
He said the coroner could have distanced herself from the grieving families, and particularly from conversations about punishing Mr Rolfe for killing Mr Walker, during a visit to the community in November 2022.
The coroner visited the outback community of Yuendumu, northwest of Alice Springs, last year.
"Some community members made comments in the presence of myself and (counsel assisting Peggy Dwyer) to the effect that Mr Rolfe ought to lose his job as an NT police officer and that there was a need for customary payback," Ms Armitage said.
Mr Rolfe raised concerns about Ms Armitage and Dr Dwyer painting ochre on their foreheads during a ceremony by senior Warlpiri women.
He also questioned whether Dr Dwyer should have told community members their views about the inquest and death of Mr Walker would be taken into account, or "stated that she understood that the community wants justice".
"The two-day visit ran largely consistent with the plan," Ms Armitage said.
"The only significant variation was that I visited House 511, Memory House (where Mr Walker was shot), on the first day instead of the second day."
Coroner Elisabeth Armitage outside house 511, where Kumanjayi Walker was shot, in November 2022.
The coroner rejected Mr Rolfe's assertions, saying the Yuendumu visit was "essentially a listening experience and that is what I did."
Ms Armitage was also critical of the length of time it took for Mr Rolfe to bring his application to the court following her visit to the community.
Mr Walker's family, NT Police and the Yuendumu Parumpurru Committee have previously urged the coroner not to step aside.
The sitting for which Mr Rolfe was set to give evidence was vacated last month and no alternative date has been set.
The inquest into the Warlpiri man's death has been repeatedly disrupted by legal stoushes about whether Mr Rolfe and another officer have the legal right to refuse to provide evidence to the coroner.
Mr Rolfe previously refused to answer questions on the grounds they may expose him to disciplinary action while he was still in the force.
Ms Armitage ruled witnesses could not decline to answer questions and appeals by Mr Rolfe in the NT Supreme Court and the Court of Appeal were unsuccessful.
The rulings meant the coroner could now be able to compel Mr Rolfe to answer questions about racist text messages the inquest was told he sent.
Mr Rolfe has been contacted for comment through his lawyer.
