The king is due to pick a new prime minister after the leading contenders - opposition leader Anwar Ibrahim and former premier Muhyiddin Yassin - failed to secure enough support for a majority following a Saturday election that produced an unprecedented hung parliament.
The uncertainty prolongs political instability in the Southeast Asian country, which has had three prime ministers in as many years and risks delaying policy decisions needed to foster economic recovery.
The Council of Rulers, which groups the heads of all nine royal houses, would meet on Thursday to discuss the formation of a new government, the palace said.
"The purpose of the meeting of the Council of Malay Rulers is for the King to get the thoughts of the Malay rulers so that he can make a decision for the wellbeing for the people and the country," it said on Wednesday.
King Al-Sultan Abdullah Sultan Ahmad Shah must decide who will be prime minister after both Anwar and Muhyiddin missed his Tuesday afternoon deadline to put together an alliance.
The constitutional monarch plays a largely ceremonial role but can appoint a premier he believes will command a majority in parliament.
Malaysia has a unique constitutional monarchy in which kings are chosen in turn from the royal families of nine states to reign for a five-year term.
King Al-Sultan Abdullah made his announcement of a special council after meeting MPs from the incumbent Barisan Nasional coalition.
Anwar's coalition won the most seats in the Saturday election with 82, while Muhyiddin's Perikatan bloc won 73. They need 112 to form a government.
Barisan won only 30 seats - the worst electoral performance for a coalition that had dominated politics since independence in 1957 - but the support of its members will be crucial for Anwar and Muhyiddin to get to 112.
Barisan said it would not align with either of the rival coalitions.
Muhyiddin declined working with his rivals to form a "unity government". Muhyiddin runs an ethnic Malay Muslim conservative alliance, while Anwar heads a multi-ethnic coalition.
Muhyiddin's bloc includes an the Islamist party PAS, whose electoral gains have raised concern in a country with significant ethnic Chinese and ethnic Indian minorities, most of whom follow other faiths.
