Twenty-four pupils were injured in the incident at the Abu Naima Islamic school on Sunday, Mohammed Abdulsalam, from the National Emergency Management Agency said.
It is unclear whether more students may be trapped under the rubble, he added.
Building collapses often occur during the rainy season in Nigeria.
The cause of the collapse is not clear, but investigators are trying to determine whether it may have been due to extra floors being added to the one-storey building.
A multi-storey church hostel run by popular Nigerian evangelist TB Joshua collapsed last September in Lagos killing 116 people.
The authorities said it had more floors than its foundation could hold.