Traditional Credit Union, which provides services for Indigenous people and organisations in remote Australia, told a parliamentary committee the proposal would simply shift people onto the more restrictive basics card.
"We don't agree with broad-based mandatory income management but we do agree with voluntary and targeted income management," TCU chief Tony Hampton said on Wednesday.
"Once the bill goes through then everybody that's on the cashless debit card has to go back to the basics card ... (it) doesn't actually achieve anything as far as we're concerned with moving away from broad-based mandatory income management."
A federal parliamentary committee heard from Northern Territory organisations about an Albanese government plan to abolish the cashless debit card.
The debit cards were introduced by the Liberal-National coalition in 2016 and set up in several communities including Ceduna in South Australia and Cape York in far north Queensland.
Under the scheme, up to 80 per cent of a person's welfare payment was added to a card and the funds could not be withdrawn for cash or spent on gambling or alcohol.
The proposal would shift about 3800 people in the Northern Territory onto the basics card, Social Services Minister Amanda Rishworth said.
"This cashless debit card was effectively privatised welfare and wasn't working for many people," she told ABC radio on Wednesday.
"We will be consulting with communities into the future about the basics card."
But the basics card was not a good form of income management as it restricted shopping online, outside communities and made it obvious a person was on welfare, Mr Hampton said.
The union would rather people were able to stay on the cashless debit card voluntarily instead of abolishing it altogether.
"The debate here is not the cashless debit card ... it's about mandatory income management," Mr Hampton said.
Meanwhile, the peak body for the Northern Territory community and social services sector told the committee Australia's social security system as a whole is belittling, complicated and needs reform.
The NT Council of Social Services is calling on the federal government to abolish all forms of compulsory income management for people on welfare.
There is evidence to suggest large numbers of Territorians don't have any form of social security, NTCOSS chief Deborah Di Natale said in a submission to the committee.
"Feedback suggests that many people become disengaged as they find the social security system repelling, belittling, impenetrably complicated, and damaging," she said.
"The government's desire to co-design a workable voluntary income management scheme with affected communities will not be hampered by the immediate repeal of compulsory income management."