One of them pulled out a kitchen knife and stabbed him as he tried to get away, his partner finding him bleeding profusely on the floor.
The 42-year-old could not be saved and died in hospital from his injuries.
"I haven't gone near an elevator since that night," Mr Sullivan's partner of nine years, Desiree Schmidt, told a court on Wednesday.
His devastated family, who filled Melbourne's Supreme Court for his killer's pre-sentence hearing, called for knife crime laws to be strengthened.
"There must be an end to knife violence ... it's shattering lives, it's taking lives and there needs to be harsher consequences," Mr Sullivan's sister told AAP outside court.
"We can't just act on the precedence of what's been sentenced before because it's rampant.
"The laws need to be changed, they need to be more severe."
Mr Sullivan's killer, Kloud Allen, is facing up to 25 years behind bars after pleading guilty to manslaughter and an additional armed robbery charge over an unrelated incident six days prior.
The 24-year-old was with two teenage boys, on January 16, 2024, seeking "retribution" after an earlier altercation at the Carlton apartment building that had nothing to do with Mr Sullivan, the court was told.
Mr Sullivan arrived home on an electric scooter and got into the elevator to go to his apartment, about 11.30pm, with the three young men.
Allen had a tartan scarf around his face and was armed with a kitchen knife, and the group discussed an earlier fight.
Mr Sullivan tried to calm the young men down, but within minutes Allen stabbed him inside the elevator as he tried to fight back and escape.
Allen then fled the scene and escaped interstate to Queensland, where he was arrested and extradited to Melbourne on February 17.
Allen had suffered from a deprived upbringing and began drinking and taking drugs from a young age, his barrister Adam Chernok said.
"Clearly drugs played a major role here," he told the court.
Mr Chernok read from Allen's police interview, where he admitted having consumed six Xanax, two MDMA pills and "five of these little green pingers" before the killing.
The barrister offered an apology to Mr Sullivan's family in court from Allen, and said it was "the worst thing he has ever done".
He asked for his client to be handed a term of imprisonment with a shorter non-parole period to foster his rehabilitation in the community.
Prosecutor Kristie Churchill called for the court to denounce knife violence in sentencing him for "serious examples of serious offences".
"This type of offending, particularly knife crime that results in the death of somebody, is way too common and way too frequent," she said.
Dozens of Mr Sullivan's family sat in the court room on Wednesday, many of whom had flown from his home of Samoa for the hearing.
"His life ended in such a violent way, when he was the opposite of that," his sister Feuina Sullivan told the court.
"He wouldn't harm a fly, he would do anything for anyone, there was no reason for this, no justification, it was just senseless."
His partner described him as the love of her life.
"Andrew matters, our life together mattered, I need to feel like this court understands that because I carry it with me every day," Ms Schmidt said.
Allen, who was wearing a tracksuit and rosary beads, was taken back to prison by custody officers to await his sentence at a later date.
16°C