Christian Pulisic scored one of the easiest goals of his career in the 66th minute to cancel out Dennis Man's second-minute opener in Saturday's match.
But substitute Matteo Cancellieri scored what was to prove the winner for Parma 11 minutes later.
Fonseca, who replaced Stefano Pioli in the offseason, has a lot of work to do. Milan needed two late goals to salvage a 2-2 draw at home to Torino on the opening day.
Inter Milan got their first victory of the season by beating Lecce 2-0. The defending champions were held 2-2 by Genoa in its first match last weekend.
Alvaro Morata had scored on his debut but was out injured and Milan got off to the worst possible start in Parma as they found themselves behind within 90 seconds.
Man, who had netted in Parma's 1-1 draw against Fiorentina on the opening day, was left completely unmarked in the area to sweep home Emanuele Valeri's cross.
Milan offered little until after the colling break midway through the first half. Shortly after play resumed, Parma goalkeeper Zion Suzuki had to push Strahinja Pavlovic's header on to the top off the crossbar.
Minutes later, Pavlovic headed a corner just past the near post.
The Milan debutant was also decisive at the other end, in his natural position, as the centre back rushed in to make a sliding tackle to prevent Man - who had gone clear on goal - from scoring an almost certain second.
The 23-year-old Pavlovic, who joined from Salzburg at the end of last month, made another decisive tackle to deny Valentin Mihaila at the start of the second half.
That was shortly after Tijjani Reijnders hit the crossbar for Milan.
Ange-Yoan Bonny thought he had doubled Parma's lead on the hour mark but it was ruled out for offside and Milan levelled shortly after.
Rafael Leao danced down the left flank and played a one-two with Theo Hernandez before rolling the ball across for Pulisic, who just had to tap into an empty net from six yards.
However, Parma restored their lead with another devastating counterattack. Pontus Almqvist raced from inside his own half and into the box before rolling across for Cancellieri to slot into the bottom left corner.
Inter eased to victory despite being without captain and star forward Lautaro Martinez.
Lautaro, who was the last Inter player to start pre-season training after he was involved in Argentina's Copa America victory, was sidelined with muscular fatigue.
Inter took an early lead through Matteo Darmian's header in the fifth minute and sealed the victory in the 69th with a penalty from Hakan Çalhanoğlu after Marcus Thuram was grappled to the ground by Lecce defender Kialonda Gaspar.
Lazio were one of only four sides to win last weekend but they surprisingly lost their second match 2-1 at 10-man Udinese.
Lorenzo Lucca put Udinese ahead in the fifth minute when he headed in a cross from Florian Thauvin, who doubled his team's advantage early in the second half with a delightful solo effort.
Udinese had to play more than 20 minutes at a numerical disadvantage after Hassane Kamara was sent off following a second yellow card. Gustav Isaksen scored what was to prove a consolation for Lazio in stoppages.
Genoa beat Monza 1-0.
