Costa found the back of the net with his second on 89 minutes to pull the visitors back to within a goal, before supplying the assist for Jordan Courtney-Perkins to head home a 95th minute equaliser on Friday night.
The Reds had seemingly been in the box seat thanks to a double from Archie Goodwin either side of a Stefan Mauk goal before the late flurry from Sydney.
United had made the perfect start with a goal on five minutes after Ben Folami won possession in the middle of the park and charged forward before passing wide to Dylan Pierias.
The fullback squared for Goodwin, but the pass was less than ideal, forcing Goodwin to shuffle the ball between his feet before eventually slamming it into the back of the net.
Costa then opened his A-League account with a clinical finish from inside the area on 16 minutes to equalise.
Patrick Wood created the opportunity, waltzing into the United penalty area virtually unchallenged, before he was eventually confronted by Reds' defender Bart Vrieds.
The tackle propelled the ball into the path of Costa, who picked out the top corner with a first-time, left-footed strike.
Stefan Mauk (left) had one goal and Archie Goodwin (right) a double for United. (Michael Gorton/AAP PHOTOS)
Nine minutes later Adelaide were in front again. Folami tore forward and cut the ball back to the top of the box.
Zach Clough's shot was blocked, before Pierias was denied by Harrison Devenish-Meares.
However, the keeper could only parry into the path of Mauk, who bundled it over the line from six yards.
Sydney attacker Joe Lolley squandered a golden opportunity in first half injury time after he capitalised on a defensive mix up between Vriends and Panagiotis Kikianis.
The pair went for the same high ball and were caught out, allowing Lolley to race away, but he skied his shot over with just James Delianov to beat.
Following an avalanche of pressure immediately after the restart, Sydney almost equalised from a set piece on 51 minutes.
Costa floated a ball to the back post where Jordan Courtney-Perkins headed across the six yard box for Aaron Gurd, whose header bounced out off the cross bar.
The visitors were left to rue the missed chance as the Reds made it 3-1 on 54 minutes.
Jay Barnett recycled a Sydney clearance out of the penalty area and charged forward, playing a one-two with Folami before attempting to place a shot past Devenish-Meares.
The Sydney shot stopper managed to get a hand to it, but Goodwin pounced and buried his second from close range.
On 89 minutes Costa ensured a dramatic finale, doubling his and the visitors' tally.
The Brazilian cut in from the right and whipped in a cross with his left that bounced in front of Delianov and bent into the far corner.
Costa was involved again when Sydney equalised five minutes into stoppage time. His in-swinging cross was met by Courtney-Perkins, who reached the ball ahead of Delianov to head home from a couple metres out.
16°C