“Nobody should be president for life,” Mr Obama said.
He was speaking at the African Union’s headquarters in Ethiopia’s capital, Addis Ababa, the first time a sitting US president has addressed the body.
Earlier in the trip, Mr Obama visited Kenya, the homeland of his late father.
“I don’t understand why people want to stay so long, especially when they have got a lot of money,” he told the 54-member AU, an apparent criticism of African leaders who have done just that.
Calling on the AU to ensure leaders respect their constitutions and step down when their term ends, Mr Obama specifically mentioned Burundi, whose president Pierre Nkurunziza has controversially been re-elected for a third term.
“Sometimes you will hear leaders say ‘I’m the only person who can hold this nation together.’ If that’s true, then that leader has failed to truly build their nation.”
He said democracy was about more than just holding elections: “When journalists are put behind bars for doing their jobs or activists are threatened as governments crackdown on civil society then you may have democracy in name, but not in substance.”
And he joked about his own chances of another term in office, which he is constitutionally barred from seeking.
“I actually think I’m a pretty good president,” he said. “I think if I ran, I could win. But I can’t!”
He also called for an end to the “cancer of corruption”, saying it was the key to unlocking Africa’s economic potential.
The money could be used to create jobs and build schools and hospitals, Mr Obama said.
Treatment of women
The rapid economic growth in Africa was changing “old stereotypes” of a continent hit by war and poverty, he said.
But unemployment needed to be urgently tackled on a continent whose one billion people will double in a few decades, Mr Obama said.
“We need only look to the Middle East and North Africa to see that large numbers of young people with no jobs and stifled voices can fuel instability and disorder,” he added.
In echoes of his speech in the Kenyan capital Nairobi, Mr Obama condemned the repression of women, saying the “single best indicator of whether a nation will succeed is how it treats its women”.
His address to the AU marks the end of his five-day visit to Africa.
The trip has focussed heavily on trade and security, but he also found time in Kenya to meet relatives of his father, including his half-sister Auma.