Breetzke, 26, struck 11 fours and five sixes in his 148-ball innings as South Africa, batting first, amassed 6-304 in their 50 overs on Monday.
New Zealand chased their target down with eight balls to spare thanks to an unbeaten 133 from Kane Williamson to win by six wickets.
Breetzke's score beat the previous best mark of West Indian Desmond Haynes, who made 148 on his debut against Australia in Antigua in 1978. The South African record had been 124 by Colin Ingram against Zimbabwe in 2010.
"It hasn't sunk in yet and I'm a bit tired," Breetzke said. "The pitch was tacky and (New Zealand seamer Will) O'Rourke was really tough to face.
"My mindset was to bat as long as possible and give myself a chance at the end."
Breetzke, who is not part of South Africa's Champions Trophy squad, batted until the 46th over.
He has played 10 Twenty20 internationals and a single Test, making his ODI debut in a weakened line-up missing several leading batters who are not available for the series.
After losing captain Temba Bavuma (20) inside the power play, Breetzke shared 93-run stand with Jason Smith (41). He pushed the scoring rate with some exhilarating hitting, especially on the off side of the wicket, and added further 131 runs with Wiaan Mulder, who made 64 off 60 balls.
Breetzke raised his century off 128 balls with an audacious square cut boundary of fast bowler Will O'Rourke and needed just another 20 balls to surpass Haynes' 47-year record.
He was finally dismissed after hitting 11 boundaries and five sixes when he misread Matt Henry's slower ball and was caught at mid-off.
Williamson and Devon Conway set up the run-chase with a solid second-wicket partnership of 187 runs against a weakened South African pace bowling attack.
Williamson used his feet well against inexperienced bowlers and even experienced wrist-spinner Tabraiz Shamsi could not separate the pair on a flat wicket.
Williamson completed his second fastest century in ODIs off 72 balls with a single, but Conway missed out on his hundred when he fell for 97 while attempting the landmark with a square cut.
The tournament now moves to Karachi, one of the three venues upgraded for the Champions Trophy, with Wednesday's winner of South Africa v Pakistan match taking on New Zealand in the final on Friday.
Karachi will also host Champions Trophy's opening game between Pakistan and New Zealand on February 19.
