The Tahs scored four tries to three, posting 21 unanswered points after trailing 6-0 to lead by 15 at halftime on Saturday.
But the resilient Brumbies, who played with 14 men for a quarter of the game notched 12 straight points to get within three with just over 12 minutes left.
Dangerous winger Max Jorgensen soothed the nerves of the home supporters in the 20,572 crowd, when he chipped the ball ahead and then toed it forward, with centre Joey Walton winning the chase.
The anxiety returned within three minutes, as the never-say-die visitors hit back, with winger Andy Muirhead catching a cross field kick from Jack Debreczeni following great work from Wallabies centre Len Ikitau.
The Tahs held on to record their first victory over the Brumbies since 2018 and their first over them at Allianz Stadium since 2015.
"It just gives our group a whole lot of belief and confidence now moving forward. It's been seven years,13 in a row they have won," Waratahs and former Brumbies coach Dan McKellar said.
"We've got a lot of respect for them as a team but for us it's a real shot in the arm."
Brumbies five-eight Noah Lolesio kicked two early penalties and the visitors had control of the game with a barely believable 97 per cent of territory after 14 minutes.
But things unravelled for the visiting side from that time after a break in play caused by an injury to hooker Mahe Vailani, who was a late inclusion the starting side after Dave Porecki (knee/calf) pulled out.
Shortly after play resumed, the Waratahs scored a couple of long distance breakout tries off turnovers against the run of play, with halfback Teddy Wilson and winger Triston Reilly crossing.
The Tahs scored their third through flanker Rob Leota, when the Brumbies were down to 14 with hooker Billy Pollard in the bin for a high tackle.
The Brumbies started the second half well, keeping the ball in close and appeared to have got five points back, when fullback Tom Wright crossed after a 20-phase move.
But the score was rubbed out upon review with Lolesio judged to have knocked on in the build-up.
The Brumbies produced another multi phase move with captain and prop Allan Alaalatoa barging over from close range.
They suffered another blow, with flanker Tom Hooper binned for a dangerous lifting tackle in the 55th minute.
Flanker Luke Reimer crashed over from a trademark Brumbies drive from a lineout to keep them in touch and set up a thrilling finish.
Brumbies coach Stephen Larkham said his team's record against the Waratahs hadn't played on his mind at any stage.
"It's a tough one, the changing room was very quiet, guys were disappointed in certain aspects of their performance," Markham said.
Both Larkham and Alaalatoa felt the Brumbies needed to be more composed, with the coach also looking for more consistency in refereeing decisions.
"There were a couple of decisions there that weren't consistent. but we had two yellow cards, but that had nothing to do with the referee that was 100 per cent on us," Larkham said.
19°C