The Edinburgh side looked on course to exit the competition on Monday as they trailed to a first-half goal from Mikael Mandron until Australian midfielder Calem Nieuwenhof, who had not played since March 3 last year due to a hamstring problem, stepped off the bench in the 84th minute to equalise within seconds of entering the fray.
With the tie still deadlocked at 1-1 at the end of extra-time, Neil Critchley's resurgent side - unbeaten since Boxing Day - then secured their progress in the shootout as 42-year-old goalkeeper Gordon saved the Buddies' first two kicks from Oisin Smyth and James Scott.
James Penrice sealed the Jambos' victory after Gerald Taylor had seen his kick - Hearts' fourth - saved by Zach Hemming.
Saints, who had lost five of their previous seven matches, spurned a great chance in the fourth minute when Toyosi Olusanya lobbed the ball over the crossbar after latching on to goalkeeper Hemming's long kick-out.
The visitors' first attempt came in the 13th minute when January signing Elton Kabangu - buoyed by five goals in his previous three outings - saw his ferocious low strike pushed behind by Hemming.
The Buddies broke the deadlock in the 22nd minute when Mandron stabbed home from just inside the six-yard box after Mark O'Hara's low curling shot from the angle of the area was parried out by Gordon.
Hearts offered little in response for the remainder of the first half and were booed off by their 1600 travelling supporters at the break.
Things nearly got worse for the the visitors, with Olusanya and Killian Phillips both missing good chances early in the second half.
The Tynecastle side then lost Lewis Neilson and Socceroos midfielder Cameron Devlin to injuries, with January signings Michael Steinwender and Sander Kartum entering the fray to help salvage the tie.
But it was another substitute who eventually got Hearts on level terms in the 84th minute when Nieuwenhof was perfectly placed to drill home from 10 yards out after Penrice's attempt to get a shot away from James Wilson's low cross broke into his path off Phillips.
It proved a pivotal moment as the Jambos eventually prevailed at the end of a long and gruelling encounter.
