Forest's dream season continued at the City Ground on Monday night, in front of England manager Thomas Tuchel, as they moved one step closer to a first trophy since 1990, while Ipswich now switch their attention to staying in the Premier League.
Five points from safety with 11 games remaining, the result added to a season of woe for Australians Massimo Luongo and Cam Burgess.
Midfielder Luongo was subbed off after 67 minutes while in more concerning news for Socceroos boss Tony Popovic, central defender Burgess couldn't shake an ankle injury suffered in the first half and was replaced at halftime.
It leaves the Scottish-born 29-year-old in doubt for crucial World Cup qualifiers at home to Indonesia and away in China later this month.
George Hirst's third goal of the season put the visitors on track for a first win over any Premier League side this calendar year until Ryan Yates' equalising header sent the match into extra time.
No goals came in the additional 30 minutes and Sels dived low to his left to stop Taylor from the spot to secure a 5-4 shoot-out success, setting up a quarter-final clash at Brighton & Hove Albion.
"It's a good moment, it's important for the club to give it a real go," Forest manager Nuno Espirito Santo said.
"Now we have to focus on the Premier League but the FA Cup is always going to be on our mind."
It was the second straight FA Cup game in which Forest have come out on top in a shoot-out after Sels produced saves in the last round against League One side Exeter City.
The best chance of the first period fell to Forest's Ibrahim Sangare, who glanced a header across the face of goal, before teammate Anthony Elanga had an effort from distance saved by Alex Palmer at the start of the second half.
Ipswich's Sam Morsy smashed an audacious 35-yard effort towards goal but Sels got the finest of fingertips on it to push over the bar.
However, he could not stop the Tractor Boys from breaking the deadlock on 53 minutes from the resulting corner.
Ben Johnson's inswinger picked out an unmarked Hirst at the far post and he nudged a header towards goal which bounced through Sels' legs.
Forest levelled in the 68th after Elanga delivered a teasing ball into the area straight onto the head of Yates who nodded beyond Palmer.
Forest began to put their relegation-threatened visitors under pressure - Murillo's 25-yard bullet was stopped by Palmer before Callum Hudson-Odoi rattled the crossbar from similar range and neither team did enough in extra-time which then forced penalties.
And after nine perfect spot kicks, Taylor missed the only penalty as Sels guessed right to send Forest through to the last eight.
"We can take a lot of positives from tonight but the main outcome we wanted was to go through," Ipswich boss Kieran McKenna lamented.
FA CUP QUARTER-FINAL DRAW:
Fulham v Crystal Palace
Preston North End v Aston Villa
Bournemouth v Manchester City
Brighton and Hove Albion v Nottingham Forest
(Ties to be played weekend of March 29-30)
With AAP.
