Iran had warned of retaliation if the IAEA's 35-member Board of Governors passed a resolution drafted by the United States, France, United Kingdom and Germany criticising Iran for its continued failure to explain uranium traces found at undeclared sites.
The resolution was passed by a crushing majority late on Wednesday.
Iran told the agency overnight it planned to remove equipment including 27 IAEA cameras as of Thursday, which is "basically all" the extra monitoring equipment installed under the 2015 deal going beyond Iran's core obligations to the agency, Grossi told a news conference.
That leaves a window of opportunity of three to four weeks to restore at least some of the monitoring that is being scrapped or the IAEA will lose the ability to piece together Iran's most important nuclear activities, Grossi said.
"This would be a fatal blow (to reviving the deal)," Grossi said of what would happen if that window went unused.
Indirect talks between Iran and the United States on reviving the 2015 deal have been stalled since March.
"You think we would retreat from our positions if you pass a resolution at the (IAEA) Board of Governors? In the name of God and the great nation of Iran, we will not back off a single step from our positions," Iranian President Ebrahim Raisi said in a speech.
Since then-president Donald Trump pulled the US out of the deal and re-imposed sanctions against Iran in 2018, Tehran has breached many of the deal's limits on its nuclear activities.
It is enriching uranium to close to weapons-grade and world powers warn it is getting closer to being able to sprint towards making a nuclear bomb.
Iran denies wanting to.
France, Britain and Germany, the so-called E3, condemned Iran's actions on Thursday and urged it to fully resume co-operation with the watchdog and end its nuclear escalation.
"These actions only aggravate the situation and complicate our efforts to restore full implementation of the JCPoA," the E3 said in a statement.
They also cast further doubt on Iran's commitment to a successful outcome.
Washington issued a separate statement earlier on Thursday, stopping short of condemning Iran's actions and urging Tehran to choose diplomacy and de-escalation.
Iran has been keeping the data recorded by the extra monitoring equipment since February of last year, meaning the IAEA can only hope to access it at a later date.
Grossi said it was not clear what would happen to that data now.
He added that more than 40 IAEA cameras would keep operating as part of the core monitoring in Iran that predates the 2015 deal.
