Sharma was out for a record 16th duck, the most for a batsman in IPL history, as Mumbai struggled to get going and posted only 139-8 (20 overs).
Chennai, winless in three games prior, finished at 140-4 (17.4 overs) as Devon Conway led the chase with 44 off 42 balls.
Put into bat, Mumbai changed its batting order to help Sharma get off his poor run. The skipper has failed to reach double digits in his last three innings.
Ishan Kishan (7) and Australian Cameron Green (6) opened the innings, but both fell cheaply.
Tushar Deshpande bowled Green for six, while Kishan skied off Deepak Chahar (2-18) and was out caught.
Sharma, coming in at number three, then chipped to backward point for a three-ball duck as Mumbai was down to 14-3 in 2.5 overs.
Nehal Wadhera played the rescue hand with his maiden IPL half-century off 46 balls. He put on 55 off 46 balls with Suryakumar Yadav (26), before Ravindra Jadeja bowled the latter in the 11th over.
Wadhera scored 64 off 51 balls, hitting eight fours and a six, and put on another 54 off 42 balls runs with Tristan Stubbs (20), as Mumbai reached 123-5 in 17.3 overs.
But the final push for quick runs never came as Chennai bowlers etched out three more wickets in the space of seven deliveries.
Matheesha Pathirana finished with 3-15 in four overs bowling at the death.
In reply, Chennai got off to another solid start with openers Conway and Ruturaj Gaikwad (30) putting on 46 off 25.
Veteran wrist spinner Piyush Chawla (2-25) continued his fine form for Mumbai by picking another couple wickets. First he had Gaikwad caught behind, before trapping Ajinkya Rahane lbw for 21.
Conway, though, held on even as other batsmen hit out. Impact player Ambati Rayudu scored 12, before Shivam Dube helped finish things off in a hurry with 26 not out off 18 balls.
Dube hit three sixes as Chennai beat Mumbai at its home ground for the first time since 2010 and climbed to second in the points' table.
Delhi Capitals host Royal Challengers Bangalore in the second game of the day.
18°C