No reason was given for the removal of Ibrahim Lamorde, the chairman of the Economic and Financial Crimes Commission (EFCC).
In August, Mr Lamorde denied allegations that $5bn (£3.3bn) had gone missing at the commission.
Mr Buhari won Nigeria’s presidential elections in March, promising to fight corruption in the country.
In August, a Senate committee said it was investigating charges that assets and cash recovered by the EFCC had been diverted.
At the time, Mr Lamorde told the BBC the charges were a smear campaign.
He led the EFCC for four years until his dismissal on Monday.
President Buhari’s spokesman Garba Shehu said later on Monday that assistant Police Commissioner Ibrahim Mustafa Magu had been appointed as the EFCC acting commissioner.
President Buhari – the former army general known for his authoritarian style – was elected in March on a platform of cleaning up Nigerian politics, the BBC’s Martin Patience in Lagos reports.
More than five months after he came to power his new cabinet is finally expected to be sworn in later this week. The reason for the extraordinary delay is that President Buhari wants to try to ensure that new ministers will not use their offices for personal gain, our correspondent says.
He adds that this has won Mr Buhari widespread support in Nigeria where many are tired of the corruption, which they believe is preventing the country from realising its full economic potential.