Nationals leader David Littleproud said on Monday the party would be against the voice, arguing the body would not improve outcomes for Indigenous people.
"We've been down this road before," he told Nine's Today program on Tuesday.
"What we fear is this will be a voice for Redfern but not for Wilcannia, not for Alice Springs and not for Carnarvon."
While Mr Dreyfus said it was disappointing to hear the opposition from the Nationals, he still thought the referendum would succeed.
"Just because they've now made this announcement does not deal anything like a deathblow to this referendum," he told ABC Radio on Tuesday.
"It's very disappointing to hear one of the major parties in Australia deciding before the campaign has even started that they're going to oppose this really important measure."
The referendum is due to take place during the 2023/24 financial year, although a date has not been set.
Mr Dreyfus said an education campaign will start next year on what would be needed to change the constitution, with the last referendum being held more than 20 years ago.
"If you're under 40, you've never voted in a referendum, we've got a lot of educating to do about what we need to change our constitution and how we change our constitution," he said.
"This is a step that takes Australia forward for everyone. And sadly, there are some in our country who oppose the voice, it seems, and they're holding this country back."
Indigenous coalition senator Jacinta Price had voiced her disapproval for the referendum.
"It's not racist to disagree with a proposal ... that lacks detail and divides us on the lines of race," Senator Price said on Monday.
"Minister (Linda) Burney might be able to take a private jet out to a remote community, dripping in Gucci, and tell people in the dirt what's good for them but they are in the dark and they have been in the dark."
Asked about Ms Price's comments on Ms Burney, Mr Littleproud said she had a lot of passion about the issue but it was important to have a respectful debate.
"It is important we bring respect to this. This should be a conversation we should be able to have," he added.
The government has been calling for a referendum during this term of parliament to set up an Indigenous voice - one of the recommendations from the Uluru Statement from the Heart.
Geoffrey Scott, the spokesman for the Uluru Dialogues, said the Nationals' announcement "will only make us work harder".
"We will continue talking with all Australians including supporters of the Nationals," he said.
"By deciding to do this before a referendum date has even been set, or the detail has been released, it's clear that the Nationals have put internal politics ahead of the interests of First Nations peoples."
The government will release the latest Closing the Gap report on Wednesday, outlining how work on improving Indigenous wellbeing is progressing.
A Productivity Commission report released in July found five targets were not on track, including children being developmentally ready when they commence school, out-of-home care rates, adult imprisonment, deaths by suicide and sea country rights and interests.
17°C