According to BBC, on Wednesday, the president sacked Nhlanhla Nene in a move that sent the rand to record lows and sparked a sell-off in bank shares.
His replacement for less than a week, Mr van Rooyen, is a little-known MP. The latest move sent the rand up almost 5% on Sunday night. BBC reports.
Mr Gordhan was widely respected when he served as South Africa's finance minister from 2009 until 2014.
However, Mohammed Nalla, head of research at Nedbank Capital, said having a finance minister serve just two days did not bode well for South Africa's reputation.
"International investors are probably thinking: why didn't the president make a much more considered decision in the first place?" he said.
According to BBC reports, the leader of the opposition Democratic Alliance party, Mmusi Maimane, said: "This is reckless by President Zuma - he is playing Russian roulette with the South African economy."
A statement from Mr Zuma's office said he had "received many representations" to reconsider his decision to appoint Mr van Rooyen.
"As a democratic government, we emphasize the importance of listening to the people and to respond to their views," it added.
Credit agency Fitch downgraded South Africa earlier this month, leaving South Africa just one notch above "junk" status. It said on Thursday that Mr Nene's sacking "raised more negative than positive questions".
Mr Nene's reluctance to approve a plan to build several nuclear power stations at a cost of up to $100bn is thought to have contributed to his removal as finance minister. BBC reports.
Mr van Rooyen will take over from Mr Gordhan as minister of co-operative governance and traditional affairs.
Marches to call for Mr Zuma's removal as president are being planned for five cities in South Africa on Wednesday - a public holiday.
The US Federal Reserve is expected to raise interest rates on the same day in a move that could put economies in countries like South Africa under further pressure.