Runaway Premier League leaders Liverpool had survived a barrage of attacks in the French capital on Wednesday night to keep the score at 0-0 going into the final minutes.
Elliott was sent on to replace Mohamed Salah in the 86th and see out the final moments of the match, but became an unlikely hero when sweeping home a low shot delivered to him by another substitute Darwin Nunez.
It was the Reds' first attempt of the night on target, and Elliot's first touch of the match.
"A big moment for him. I can understand that he's sometimes frustrated about the playing time he gets, because he's a good player," Liverpool manager Arne Slot said of Elliott.
"But it wasn't only Harvey, it was also the one that assisted, Darwin, who came from a difficult two games but was absolutely there again tonight."
How Slot's side held on will take some analysing but it came down to sheer force of will, an outstanding performance from goalkeeper Alisson Becker and seizing the moment when it finally arose.
Becker faced 27 shots and saved 10 of them, but at times was the only thing standing in the way of complete annihilation against totally dominant opponents.
The Brazil international told TNT Sports it was "probably the performance of my life" as he broke Pepe Reina's club record with his 22nd Champions League clean sheet.
His nine saves were also the most on record (since 2003) by a Liverpool goalkeeper in a Champions League match.
"We're the lucky ones, that's clear for everyone. I think they were the much better team today," added Slot.
"I have had some very good players as a manager but never the best goalkeeper in the world until now.
"To go away with a win here was probably a bit more than we deserved."
In the all-German tie, Bayern Munich took a big step towards the last eight with a 3-0 win against Bayer Leverkusen.
Harry Kane ended his four-game scoring drought, heading home Michael Olise's cross in the ninth minute.
Jamal Musiala doubled Bayern's advantage on 54 minutes and Kane got his second from the penalty spot in the 75th, the England forward's 30th successfully taken penalty in a row.
Elsewhere, ten-man Barcelona beat Benfica 1-0 and Inter Milan won 2-0 at Feyenoord.
Pau Cubarsi saw red in the 22nd minute, but Barca managed to overcome the setback in Lisbon thanks to Raphinha's winner in the 61st.
Lautaro Martinez became Inter's all-time leading scorer in the Champions League with his 18th goal in European club soccer's most prestigious competition, overtaking club great Sandro Mazzola's record.
Martinez fired high into the corner of the net in the 50th minute in Rotterdam to double Inter's lead after Marcus Thuram struck in the 38th.
The 2023 finalists could have been in an even more commanding position after the first leg, but Piotr Zielinski's penalty in the 65th was saved by Timon Wellenreuther.
With PA & DPA.
20°C