The Tahs scored four tries to three, posting 21 unanswered points after trailing 6-0 to lead by 15 at halftime on Saturday night.
But the resilient Brumbies, who played with 14 men for a quarter of the game, closed to within three points with just over 12 minutes left.
Dangerous winger Max Jorgensen soothed the nerves of the home supporters in the 20,572 crowd, when he chipped the ball ahead and then toed it forward, with centre Joey Walton winning the chase.
The anxiety returned within three minutes, as the never-say-die visitors hit back, with winger Andy Muirhead catching a cross-field kick from Jack Debreczeni following great work from Wallabies centre Len Ikitau.
But the Tahs held on to record their first victory over the Brumbies since 2018 and their first at Allianz Stadium in a decade.
"It just gives our group a whole lot of belief and confidence now moving forward," Waratahs and former Brumbies coach Dan McKellar said.
"It's been seven years, 13 in a row they have won. We've got a lot of respect for them as a team but for us it's a real shot in the arm."
Brumbies five-eight Noah Lolesio kicked two early penalties and the visitors had control of the game with a barely believable 97 per cent of territory after 14 minutes.
But things unravelled after a break in play caused by an injury to Waratahs hooker Mahe Vailani, who was a late inclusion the starting side after Dave Porecki (knee/calf) pulled out.
Shortly after play resumed, the Waratahs scored a couple of long-distance breakout tries off turnovers against the run of play, with halfback Teddy Wilson and winger Triston Reilly crossing.
The Tahs scored their third through flanker Rob Leota, when the Brumbies were down to 14 with hooker Billy Pollard in the bin for a high tackle.
The Brumbies started the second half well, keeping the ball in close and appeared to have got five points back when fullback Tom Wright crossed after a 20-phase move.
But the score was rubbed out upon review with Lolesio judged to have knocked on in the build-up.
The Brumbies produced another multi-phase move with captain and prop Allan Alaalatoa barging over from close range.
They suffered another blow, with flanker Tom Hooper binned for a dangerous lifting tackle in the 55th minute.
Fellow flanker Luke Reimer crashed over from a trademark Brumbies drive from a lineout to keep them in touch and set up a thrilling finish.
Brumbies coach Stephen Larkham said his team's record against the Waratahs hadn't played on his mind at any stage.
"It's a tough one. The changing room was very quiet, guys were disappointed in certain aspects of their performance," Larkham said.
Both Larkham and Alaalatoa felt the Brumbies needed to be more composed, with the coach also looking for more consistency in refereeing decisions.
"There were a couple of decisions there that weren't consistent," he said.
"But we had two yellow cards, but that had nothing to do with the referee that was 100 per cent on us."
20°C