He was recognised for starting peace talks with Eritrea and establishing a peace agreement to end the long stalemate between the two countries.
"When Abiy Ahmed reached out his hand, President Isaias Afwerki grasped it and helped to formalise the peace process between the two countries," said Berit Reiss-Andersen, chair of the Nobel Peace Prize Committee.
"In Ethiopia, even if much work remains, Abiy Ahmed has initiated important reforms that give many citizens hope for a better life and a brighter future.
"He spent his first 100 days as prime minister lifting the country's state of emergency, granting amnesty to thousands of political prisoners, discontinuing media censorship, legalising outlawed opposition groups, dismissing military and civil leaders who were suspected of corruption and significantly increasing the influence of women in Ethiopian political and community life."
The office of the Ethiopian prime minister said: "We are proud as a nation" for winning the prestigious award.
'More to be done'
Abiy has helped mediate between Kenya and Somalia in a maritime territory dispute and has also been key to bringing leaders of Sudan and South Sudan to the negotiating table.
The immediate reaction in Ethiopia was of "surprise, but not shock", said Al Jazeera's Robin Kriel, reporting from Addis Ababa.
"He really has been trying to open up the landscape in a very inclusive and nationalistic way for all Ethiopians, and not just one ethnicity," she added.
"It will be a mixed bag of emotions - his critics say it will now put a lot of pressure on him to hold free and fair elections in May next year."
That was a theme echoed by Amnesty International.
"Prime Minister Abiy Ahmed's work is far from done," said Kumi Naidoo, Amnesty's secretary-general.
"This award should push and motivate him to tackle the outstanding human rights challenges that threaten to reverse the gains made so far. He must urgently ensure that his government addresses the ongoing ethnic tensions that threaten instability and further human rights abuses. He should also ensure that his government revises the Anti-Terrorism Proclamation which continues to be used as a tool of repression, and holds suspected perpetrators of past human rights violations to account."
There had been speculations about the winner in the run-up to the ceremony, with suggested winners including 16-year-old Swedish climate activist Greta Thunberg; German Chancellor Angela Merkel; and pro-democracy activists in Hong Kong.
Since 1901, 99 Nobel Peace Prizes have been handed out to individuals and 24 organisations. While the other prizes are announced in Stockholm, the peace prize is awarded in the Norwegian capital, Oslo.
This week, 12 Nobel laureates have been named, of whom 11 are men.
Two literature prizes were awarded on Thursday - one for 2018 that went to Polish novelist Olga Tokarczuk and one for 2019 that was given to Austrian author Peter Handke.
The chemistry prize went to three scientists for their work leading to the development of lithium-ion batteries; the physics award was given to a Canadian-American and two Swiss for exploring the evolution of the universe and discovering a new kind of planet; and the physiology or medicine award went to two Americans and one British scientist for discovering details of how the body's cells sense and react to low oxygen levels.
In his will, Alfred Nobel, the Swedish industrialist and inventor of dynamite, decided the peace prize should be awarded in Oslo. His exact reasons for having an institution in Norway handing out that prize is unclear, but during his lifetime Sweden and Norway were joined in a union, which was dissolved in 1905.
The economics prize was not created by Nobel, but by Sweden's central bank in 1968. It is awarded on Monday.
With the glory comes a 9-million kronor ($918,000) cash award, a gold medal and a diploma. Even though the peace prize is awarded in Norway, the amount is denominated in Swedish kronor.
The laureates receive them at elegant ceremonies on December 10 - the anniversary of Nobel's death in 1896 - in Stockholm and Oslo.