Construction, Forestry, Maritime, Mining and Energy Union members walked off the job last week after a 54-year-old man fell up to 10 metres at the Boggo Road station site on Tuesday.
He was taken to the nearby Princess Alexandra Hospital, where he remains in a critical condition.
A CFMEU spokesman confirmed workers were back on site as a workplace health and safety investigation and full site safety audit were under way.
The Cross River Rail Delivery Authority said all major contractors would co-operate with the safety regulator.
"It is important to let the investigation of this incident take place to determine what has happened and why," the authority said in a statement.
The union continued to call for Premier Annastacia Palaszczuk to sack Transport Minister Mark Bailey over the project.
"Minister Bailey has been presiding over a dysfunctional department with a reputation for cost blowouts, project delays, poor safety and build quality, and rampant exploitation of workers," CFMEU secretary Michael Ravbar said.
"Cross River Rail is at least $2 billion over budget and 18 months behind schedule.
"If Ms Palaszczuk is serious about repairing this mess and rebuilding trust she will not only remove Mr Bailey but also abolish the Cross River Rail Authority."
The premier rejected the call for the transport minister to resign.
"There have been a number of issues - it was an absolute tragedy," the premier told reporters on Monday.
"I know that Minister Bailey and the departments are sitting down with the Cross River Rail Delivery Authority and working at addressing those issues as a matter of priority."
19°C