Andrew Gwynne, has apologised for his behaviour and the "badly misjudged" comments.
He lost his ministerial gig and was suspended from the Labour Party, as soon as Starmer learned of the saga.
"I deeply regret my badly misjudged comments and apologise for any offence I've caused. I've served the Labour Party all my life and it was a huge honour to be appointed a minister by Keir Starmer," Gwynne posted on X.
"I entirely understand the decisions the PM and the party have taken and, while very sad to have been suspended, I will support them in any way I can."
Gwynne posted messages in a WhatsApp group called Trigger Me Timbers, which he shares with more than a dozen Labour councillors, party officials and at least one other MP, the Mail on Sunday reported.
He made antisemitic slights and joked about a constituent being "mown down" by a truck, the newspaper said.
He also reportedly wrote in one message that he hoped a 72-year-old woman would soon have "croaked it" after she asked a councillor, who shared the message in the group, about her bins.
"The prime minister is determined to uphold high standards of conduct in public office and lead a government in the service of working people," a government spokesperson said.
"He will not hesitate to take action against any minister who fails to meet these standards, as he has in this case."
A Labour spokesperson said the party was investigating the comments made in the group.
"Swift action will be taken if individuals are found to have breached the high standards expected of them as Labour Party members," the spokesperson said.
