The Northern Irishman (11 under, 72, 66, 66, 73) was four shots clear through 10 holes but, as he did 14 years ago at Augusta National, stumbled with five dropped shots as Rose (65, 71, 75, 66) surged to finish level.
A bogey on the last hole of regulation forced a sudden-death playoff and both men had birdie putts on the first extra hole, McIlroy putting his approach to three feet after Rose's had settled 15 feet away.
Rose's effort slid by, before McIlroy tapped in to clinch his maiden Masters title and first major in 11 years.
The 35-year-old fell to the ground in relief after his putt dropped, having watched a five-foot par attempt on the same hole slide by minutes earlier that would have sealed the triumph in regulation.
Tiger Woods was the last man to complete the set of all four majors when he dominated the British Open in his first attempt at the career grand slam in 2000.
"It was 14 years in the making; I feel like I could have got it done there," McIlroy said of the 2011 Masters fadeout that has haunted him.
"There was a lot of pent up emotion that just came out on that 18th green
"A moment like that just makes ... all the close calls worth it."
Gene Sarazen, Ben Hogan, Gary Player and Jack Nicklaus are the only other members of the career grand slam club.
The rollercoaster final day started with McIlroy two shots clear of Bryson DeChambeau (seven under) with the American briefly leading after just two holes before fading as the chasing pack surged.
Leading by four after the 10th, he dropped a shot on the 11th and then inexplicably found water with a wedge despite laying up on 13th.
That fourth double bogey for his tournament is now a quirky record among champions.
When his curling putt on the 14th sat on the lip of the cup but didn't drop and he'd lost the lead.
Rose, who led for a record fifth time after the first round, bogeyed the 17th but then drained a 20-foot putt on the 18th to heap pressure on the leader.
McIlroy found the sand with his wedge approach and then pulled his par putt to fall into the playoff.
He delivered again with an iron, as he had done with scintillating approaches to the 15th and 17th for birdies, to secure victory.
The whirlwind began on Thursday when McIlroy went back to par with a pair of double bogeys and sat seven behind fast-starting Rose.
Only Nick Faldo in 1990 and Woods in 2005 had previously come from that far back after the opening round to win.
But McIlroy launched his comeback with a back nine of 31 and exploded out of the blocks with a record six consecutive threes to start on Saturday.
"All week I responded to setbacks," he said.
Patrick Reed (nine under) holed out for eagle on the 17th to finish outright third.
Scottie Scheffler (eight under) was fourth and Australian Jason Day dropped shots on the last two holes to finish five under and in a tie for eighth.
Rose fell just short of executing the second-biggest Masters comeback after starting the final day seven shots behind.
The former world No.1, US Open and Olympic champion, joins Greg Norman as a three-time Masters runner-up and Ben Hogan as the only loser of multiple playoffs after his 2017 defeat to Sergio Garcia.
Min Woo Lee, Australia's only other cut survivor, was two over on Sunday to finish six over for the tournament in 49th.
CAREER GOLF GRAND SLAMS (when they won each for the first time)
* Gene Sarazen - US Open (1922), PGA Championship (1922), The Open Championship (1932), Masters (1935)
* Ben Hogan - PGA Championship (1946), US Open (1948), Masters (1951), The Open Championship (1953)
* Gary Player - The Open Championship (1959), Masters (1961), PGA Championship (1962), US Open (1965)
* Jack Nicklaus - US Open (1962), Masters (1963), PGA Championship (1963), The Open Championship (1966)
* Tiger Woods - Masters (1997), PGA Championship (1999), US Open (2000), Open Championship (2000)
* Rory McIlroy - US Open (2011), PGA Championship (2012), The Open Championship (2014), Masters (2025)
16°C