According to BBC, the main contenders are the country's deputy president, Cyril Ramaphosa, and former cabinet minister Nkosazana Dlamini-Zuma, President Zuma's ex-wife.
The leadership battle has caused fierce political infighting, raising fears the party may split before 2019 elections.
Mr Zuma has warned the party is under threat and is at a "crossroads". BBC reports.
The party has been in power since the country transitioned to democracy in 1994 under Nelson Mandela.
More than 5,000 delegates are taking part in the four-day ANC elective conference at the Expo Centre in Johannesburg.
The leadership contest is expected to be a close one, with legal challenges a possibility. The ballot is done in secret and a result is expected later on Sunday.