Four-time defending champions City had led the Manchester derby 1-0 going into the 88th minute at the Etihad Stadium on Sunday.
But Bruno Fernandes levelled the game from the penalty spot before Diallo flicked the ball past Ederson to score the winner for United.
City fans jeered at the final whistle as their once dominant team's woeful run of form extended to one win and eight losses in 11 games.
Man City boss Pep Guardiola was the picture of frustration during their derby loss. (AP PHOTO)
It was a first derby victory for new United head coach Ruben Amorim, who led Sporting Lisbon to a 4-1 win against City in one of his last games in charge of the Portuguese club in November.
But for a long time it looked like being a third-straight league loss for United after Josko Gvardiol's headed goal in the 36th.
Fernandes was guilty of missing the target when through on goal in the second half and United struggled to create chances against a City team that rarely looked like adding to their lead.
But in a dramatic end to the match, Diallo raced after an underhit backpass from Matheus Nunes.
Nunes then brought down the United forward in the box and referee Anthony Taylor pointed to the penalty spot.
Fernandes made no mistake this time — sending Ederson the wrong way.
Moments later, Diallo was running onto a long pass from Lisandro Martinez, flicked the ball up to take the ball round Ederson and rolled it into the net from a tight angle.
United are now 12th and five points behind fifth-placed City.
Elsewhere, Ismaila Sarr scored twice as Crystal Palace beat Brighton 3-1.
The forward also provided an assist for Trevoh Chalobah in the win at the Amex Stadium, which lifted Palace four points clear of the relegation zone.
Brighton are now ninth and without a win in four matches.
Palace went ahead in the 27th minute through Chalobah, and Sarr doubled the lead for the visitors in the 33rd.
Sarr scored his second in the 82nd before Marc Guehi's own goal - his second in the last four games - pulled one back for Brighton.
29°C