A Labor victory would be its first in Bega, which has been held by the Liberals since its creation in 1988 and by its retiring member Andrew Constance since 2003.
Early tallies in the south coast seat on Saturday night gave a swing of more than 14 per cent to the ALP's Michael Holland over the Liberal Party's Fiona Kotvojs, with more than 20 per cent of the vote counted, according to the ABC.
The Liberals went into the by-election with a margin of 6.9 per cent.
A loss in Bega would push the Perrottet government further into minority and would force it to rely more heavily on the votes of independent or minor party MPs.
Labor also looks set to retain Strathfield, while the Nationals are on track to hold former deputy premier John Barilaro's seat of Monaro despite a swing of more than six per cent.
Premier Dominic Perrottet applauded his party's "phenomenal" results despite the big swings against the government in early counting, including an 18 per cent swing in Willoughby against the Liberals' Tim James.
He faced independent Larissa Penn in the blue-ribbon seat on Sydney's north shore.
"One thing I do know about the people of Willoughby, they always choose well," Mr Perrottet told supporters.
In Strathfield, Labor's Jason Yat-Sen Li narrowly narrowly leads Bridget Sakr in Sydney's inner west and is projected to win on preferences.
The premier insisted Labor's vote had gone backwards in the electorate and vowed Ms Sakr would "win the election next year. Bridget Sakr will be the member for Strathfield."
Nichole Overall is on track to retain the regional seat of Monaro, holding the seat for the Nationals over Labor's Bryce Wilson.
The resignations of Mr Barilaro from Monaro and fellow senior government member Mr Constance from Bega came after Ms Berejiklian's sudden departure in October, which was prompted by the state's anti-corruption body announcing she was the subject of investigation.
Strathfield went to the polls to replace Jodi McKay, who resigned as Labor leader in May last year and later resigned from her seat.
With about 40 per cent of constituents voting early or via post, results in the seats are not expected to be finalised for some time.
The elections were viewed as a test for new Premier Dominic Perrottet and his government's handling of the state's wave of the Omicron variant.
"There's no doubt, it's very big swing," Mr Constance told ABC TV as Bega's results began to come in.
"There is no doubt that the voters in the electorate have voted on health and that's been shown this evening."
The stakes were also high for Labor leader Chris Minns, who has pitched himself as the man to make Labor a real election chance after more than a decade in opposition.
Mr Minns said voters had sent the premier a message on Saturday.
"The premier of New South Wales needs to listen to that message," he told the party faithful. "The government needs to change direction. It must listen to expert advice. It must have commonsense solutions to difficult problems."
