Aaron Wan-Bissaka's own-goal in the 74th minute was cruel on the visitors and in particular on Potter, who looked for a time like he would exorcise memories of his ill-fated spell in west London that ended with his sacking two years ago.
Jarrod Bowen's goal late in the first half had installed that hope, though it owed to a horror moment from Levi Colwill who had surrendered the ball to his England teammate.
The sound of a restless home support in this ground was a familiar one to Potter, but those audible frustrations were extinguished when Pedro Neto side-footed the equaliser in the 64th minute before Cole Palmer's strike deflected horribly off Wan-Bissaka to win it for Enzo Maresca's side.
It was just a second win in eight in the league for the hosts, whose Champions League challenge looks tentatively back on track following Arsenal's pummelling of Manchester City.
Filip Jorgensen was handed just his second league start in goal over the error-prone Robert Sanchez, the stand-in steadying any early nerves when he shovelled behind Bowen's near-post drive.
When Palmer fired over the bar after 23 minutes, it summed up an opening quarter that had been short of attacking quality.
The game needed a flash of skill to crack it open but what it got was a moment of self-sabotage from Chelsea.
Colwill, facing his own goal, was under pressure from Mohammed Kudus near the touchline but it should not have prompted such a hideous lapse in focus from the defender. Seeking to offload the ball infield to Moises Caicedo he found only Bowen, who with an opportunistic dart latched onto the loose pass and curled low beyond Jorgensen.
Jarrod Bowen celebrates after scoring West Ham's goal in the defeat at Stamford Bridge. (AP PHOTO)
Palmer made an audacious late pitch to level things before the break, his free kick was somehow diverted from its goalward path by an outrageous reach from Alphonse Areola.
Chelsea equalised midway through the second half and the move was finished as it started, by the left boot of Neto. It was his cross to the far post that Marc Cucurella pinged back centrally to where Enzo Fernandez was waiting to shoot at goal. His effort rebounded away off Vladimir Coufal to Neto who did not hesitate in returning it firmly inside Areola's near post to level things.
Kudus battered the post with a header as the visitors almost went back in front within a minute.
Chelsea's goal to win it 15 minutes from time was fortunate - it is doubtful that Palmer's drive would have sailed as it did over Areola but for a huge deflection off Wan-Bissaka.
Just as valuable was a block deep in added time by Tosin Adarabioyo to deny Kudus from close range, as Maresca's side clung on.
34°C