He is resigning after four years in the state's top health job to take up a new role as director of health and biosecurity at CSIRO, Australia's national science agency.
Professor Sutton will stay on for several months before departing, though, and help his yet-to-be appointed successor transition into the hot seat.
The demands of leading Victoria's response to the pandemic had taken a toll on him, he said on Friday. At its height he worked 16-hour days for six months, dealing with a "barrage of lies" on social media and death threats.
"On the one hand, it's helped me to grow personally and professionally, on the other hand it nearly crushed me and it certainly weighed upon my family life," Prof Sutton told reporters.
He said current understanding of how the virus is spread showed some things could have been done differently, such as more outdoor freedoms, but that was not evident early in the pandemic.
He declined to go into the controversial lockdown of nine public housing towers in inner Melbourne but said it was a high transmission risk area, likening it to aged and disability care.
Last month the Victorian government offered to settle a $5 million class action over the measure, in which the plaintiffs claimed residents were wrongly detained for up to 14 days and threatened with physical harm if they tried leaving the towers.
"There shouldn't be a person in the world who has been part of the public health response who wouldn't reflect on things that they would do differently, things that they regret, I'm absolutely no different in that regard," Prof Sutton said.
"We acted with the understanding and information that we had at the time with the best of intentions."
Prof Sutton served Victoria and guided it through some of its most difficult times, Premier Daniel Andrews said.
"As CHO, Brett helped keep us informed, and above all, safe," Mr Andrews wrote on Twitter.
"Thank you for your service, and all the very best for your next role."
There were previous reports of clashes between himself and other public health experts and officials over conflicting views around public health orders, however Prof Sutton said he had a great relationship with the premier.
"I don't think the tensions that might exist between, you know, my purist public health view or my role as a statutory officer and cabinet or with the whole of the Victorian government is any different from any other jurisdiction in the world," he said.
"Those tensions will exist, I come with my view and my filter.
"They have broader considerations and they have constituents that they're accountable to but we talked it through and we came to decisions collectively."
Department of Health secretary Euan Wallace hailed Prof Sutton as an invaluable leader and thanked him for his work during the public health crisis.
Prof Sutton was appointed chief health officer in 2019 after joining the Department of Health in 2011.
He became a familiar face during the pandemic, often appearing alongside the premier at press conferences.
At times he was embroiled in controversy, particularly after Melbourne had one of the world's longest lockdowns, which he maintains was necessary.
Prof Sutton was involved in developing parts of the framework supporting the state's disastrous hotel quarantine program and had raised concerns about the use of private guards.
He previously acknowledged failures within Victoria's contact tracing system during an appearance before a state parliamentary inquiry into the matter.
36°C