NT Coroner Elisabeth Armitage has been investigating Mr Walker's death since September 2022, and was set to resume on Monday in Alice Aprings.
Mr Rolfe shot the 19-year-old three times while on duty in the remote community of Yuendumu but was acquitted of murder in a five-week trial.
Mr Rolfe's lawyer, Luke Officer, filed a submission two weeks out from the resumption of the inquest asking the coroner to consider recusing herself on the grounds of apprehended bias.
Mr Walker's grandfather Steven Marshall said the delay meant closure was further out of reach for the family.
"Why is he a special man? He gets to do things differently...we wouldn't get the same chance," he said on Wednesday.
"I feel angry. He has to answer for what he has done".Â
Mr Walker's family, Nohrthern Territory Police and the Yuendumu Parumpurru Committee have urged the coroner not to step aside.
In a statement, the court said the coroner did not have time to sift through the hundreds of pages of submissions in time for Monday's sitting.
"Coroner Armitage will need to take time to carefully consider the matters raised in the many submissions received before ruling on the application, in circumstances where she has other court obligations this week," a spokesman said.
"The further delay to these proceedings is extremely regrettable but necessary to address the application made."
The sitting has been vacated and no alternative date has been set.
Mr Walker's cousin Samara Fernandez said Mr Rolfe's submission showed a "blatant disregard to our healing process".
"Every delay he deliberately creates makes work for us, it is the undoing of months of planning, months of conversations, of building up the strength to be present, only to have to start over again," she said.
In its submission, NT Police also said Rolfe's application was a delaying tactic to avoid giving evidence at the inquest.
Mr Rolfe argued the coroner colluded with police to have him sacked earlier in the year to provide "justice" for Mr Walker's family.
In a reply filed late on Tuesday, Mr Rolfe's lawyer defended the application.
"The description of a serious submission as a forensic tactic is unjustified," Mr Officer said.
The court and Ms Armitage visited the remote community of Yuendumu and met with Mr Walker's family and community members last year.
Mr Rolfe's lawyers said the coroner could have distanced herself from the grieving families, particularly from conversations about punishing Mr Rolfe for killing Mr Walker.
"Discussions had taken place during the proceedings about the need for payback and that there would be no healing until the blood of Mr Rolfe had been split," Mr Officer said.
Mr Officer said the way the coroner and counsel assisting engaged with the community was unnecessary.
"There is a marked distinction between the conduct of a culturally appropriate inquest and the coroner's engagement."
The inquest into the Warlpiri man's death has been repeatedly disrupted by legal stoushes about whether Mr Rolfe and another officer have the legal right to refuse to provide evidence to the coroner.
Mr Rolfe previously refused to answer questions on the grounds they may expose him to disciplinary action, while he was still in the force.
Judge Armitage ruled witnesses could not decline to answer questions, and appeals by Mr Rolfe in the NT Supreme Court and the Court of Appeal were unsuccessful.
The rulings meant the coroner could now be able to compel Rolfe to answer questions about racist text messages the inquest was told he sent.
The coroner is set to make her decision next week as to whether to excuse herself.
