Mr Goff, foreign and defence minister in Helen Clark's government, will leave the role under orders from Foreign Minister Winston Peters.
On Tuesday, the 71-year-old appeared at an event in London with Finland's Foreign Minister Elina Valtonen, when he made a thinly-veiled attack on Mr Trump.
"President Trump has restored the bust of Churchill to the Oval Office. But do you think he really understands history?" he asked.
Made aware of the comments by Wellington newspaper The Post, Mr Peters' office confirmed he would be removed from the job for the slight.
"Phil Goff's comments are deeply disappointing," the spokesman for Mr Peters said.
"They do not represent the views of the NZ Government and make his position as High Commissioner to London untenable.
"We have asked the Secretary of Foreign Affairs and Trade, Bede Corry, to now work through with Mr Goff the upcoming leadership transition at the New Zealand High Commission in London."
Mr Goff is yet to comment on the matter.
On social media, Ms Clark said it was a "a very thin excuse for sacking a highly respected former #NZ Foreign Minister from his post".
However, the move shows the reticence from many foreign governments to be seen as criticising the returned US president, even as he contorts the most commonly understood realities.
Mr Trump has called Ukraine President Volodymyr Zelenskiy a dictator who started the war with Russia despite all evidence to the contrary, and has launched a series of verbal attacks on US allies.
This week, Prime Minister Chris Luxon said he trusted Mr Trump, but would directly challenge the imposition of any tariffs thrust upon New Zealand.
Mr Goff, also a former Labour leader and Auckland mayor, was appointed to the plum diplomatic post by Jacinda Ardern's government in 2022.
His tenure was called into question around the time of King Charles' coronation in 2023, when he disrespected Maoridom by telling an official function that nobody had experienced a coronation before.
Mr Goff apologised after he was informed that for Tuhetia, the Maori King, who was present, it was his third coronation.