However, the government remains poised to push reform projects through at next month's budget, which includes new housing, health and climate change measures which will cost billions.
Inflation is on the march in New Zealand, with the consumer price index set to top seven per cent when new figures are released on Thursday.
The Reserve Bank is already fighting inflation by raising the official cash rate from 0.25 in October to 1.50 as of last month.
In an address to the International Monetary Fund on Tuesday, Mr Orr said further rate rises were coming, with the RBNZ predicting an OCR above three per cent within 12 months.
"It's very obvious what we need to do next, which means we are not in a great place now," he said.
"We need to tighten monetary conditions and that's what we've been about."
Mr Orr suggested the NZ government now needed to chip in and help keep inflation within the desired band of one to three per cent.
"Central banks aren't going to achieve their mandates on their own. We are going to need support," he said.
"We're going to have to be very clear with our fiscal authorities ... and how they can assist around more targeted effective fiscal policies."
In response to the address, Finance Minister Grant Robertson ruled out a change of tack from the government, saying certain areas needed investment.
"We need to continue with the building of state houses, for example ... we've still got a housing crisis," he said.
"And in the health system, where COVID-19 has shown us the importance of the kinds of reforms that we're proposing.
"We need to make these long-term investments."
This year's budget comes at the halfway point of the parliamentary term, and represents the best chance Jacinda Ardern's Labour government - which rules in majority - has to implement key planks of its agenda.
In addition to ongoing housing spend, it is centralising the health system at great expense, and implementing its first 'carbon budget' based on an independent assessment of how best to cut greenhouse gas emissions.
Mr Robertson has also pledged to carry on with an income insurance scheme which would see Kiwis paid at 80 per cent of their salary when made redundant.
"There are no costs imposed on anybody until at least the end of 2023," he said of the social insurance.
While the high stakes battle between monetary and fiscal policy is being played out in Wellington, around New Zealand Kiwis are feeling the pinch at the hip pocket.
The surging cost of living is the number one political issue, with the government increasing welfare payments and cutting fuel excise in response.
The opposition links inflation to the government's spending, urging it to tighten up.
Mr Robertson said Kiwis could see through those suggestions.
"New Zealanders understand (inflation) is a global phenomenon. They only need to see the headlines every night," he said.
"Inflation in the US has gone over eight per cent. Inflation in the UK has gone over seven per cent. They can see the war in Ukraine. They've heard about the supply chain constraints.
"These global factors are what is driving inflation."
31°C