The ceremony started at 2pm local time on Tuesday, with Abe's ashes carried into the Nippon Budokan Hall in central Tokyo by his widow, Akie, to music from a military band and the booms of the honour-guard salute.
Abe's killing at a July 8 campaign rally set off a flood of revelations about ties between lawmakers in the ruling Liberal Democratic Party (LDP) he once ran and the Unification Church, which critics call a cult, sparking a backlash against current premier Fumio Kishida.
With his support ratings dragged to their lowest ever by the controversy, Kishida has apologised and vowed to cut party ties to the church.
But opposition to honouring Abe with a state funeral, the first such event since 1967, has persisted, fed by an $US11.5-million ($A17.7 million) price tag to be borne by the state at a time of economic pain for ordinary citizens.
In one part of downtown Tokyo, protesters waved signs and chanted "No state funeral" to the tune of a guitar.
But thousands of mourners flooded to the funeral venue from early morning, forcing organisers to open the hall half an hour early. Within hours, about 10,000 people had laid flowers and bowed in silent prayer before Abe's picture, television showed, with far more waiting in three-hour long queues.
Inside the Budokan, better known as a concert venue, a large portrait of Abe draped with black ribbon hung over a bank of green, white and yellow flowers. Nearby, a wall of photos showed him strolling with G7 leaders, holding hands with children and visiting disaster areas.
A moment of silence was followed by a retrospective of Abe's political life and speeches by leading ruling party figures, including Kishida and Yoshihide Suga, Kishida's predecessor as prime minister.
Japan's longest-serving prime minister was a divisive figure who was dogged by scandals.
An unapologetic nationalist, Abe pushed the country toward a muscular defence posture that many now see as prescient amid growing concern about China, but others criticised as too hawkish.
About 4300 people were expected at the funeral ceremony itself along with at least 48 current or former government figures, including Australian Prime Minister Anthony Albanese, US Vice President Kamala Harris and Indian Prime Minister Narendra Modi.
Some 20,000 police were deployed, nearby roads were closed and even some schools shut as Japan sought to avoid the security blunders that led to Abe's shooting with a home-made gun by a suspect who, police say, accused the Unification Church of impoverishing his family.
Kishida has explained the decision to hold a state funeral as a way of honouring Abe's achievements, as well as standing up for democracy, but ordinary Japanese remain divided. Only 30 per cent of respondents in a recent poll by TV Asahi agreed with hosting the funeral, against 54 per cent opposed.
Kishida also cited the chance to conduct diplomacy as a reason for the funeral, and spent Monday night and Tuesday morning in meetings with leaders.
Even without the recent revelations about the Unification Church, it would be hard to imagine any circumstances where a majority of Japanese would favour honouring Abe with a state funeral, said Tobias Harris, a senior fellow at the Centre for American Progress and the author of a biography of the former premier.
"He was someone who almost welcomed and invited controversy and saw his mission as overturning a longstanding consensus or set of consensuses" about how Japan was run, Harris said.
Many Japanese were "attached to the postwar regime that he wanted to overturn", Harris said.
