The city of Joso, north of the capital, Tokyo, was hit by a wall of water after the Kinugawa River burst its banks. Helicopter rescue teams have been plucking people from rooftops.
One person has been reported missing in the region and at least 12 are injured.
The rains come a day after Typhoon Etau brought winds of up to 125km/h (78mph) to central Aichi prefecture.
“This is a scale of downpour that we have not experienced before. Grave danger could be imminent,” the chief forecaster at the Japan Meteorological Agency (JMA), Takuya Deshimaru, told an emergency press conference earlier on Thursday.
The hardest-hit areas have been Ibaraki and Tochigi prefectures. Japan’s Meteorological Agency had put both regions on its highest level of alert.
Television footage from Joso in Ibaraki showed people clinging to the rooftops before helicopter rescue teams winched them to safety.
Entire homes and cars were carried away on the torrent as the Kinugawa River burst its banks after two days of heavy rainfall.
In Tochigi, more than 500mm (19 inches) of rain fell in 24 hours in places, according to local public broadcaster NHK, which said that was about double what normally falls there throughout the whole of September.
Parts of central Tochigi have seen almost 60cm of rain since Monday evening, breaking records.
Many other areas of eastern and north-eastern Japan have also been issued weather warnings, including Fukushima prefecture, home to the still-damaged nuclear plant hit in 2011’s earthquake and tsunami.
The downpour overwhelmed the site’s drainage pumps, a spokesman for operator Tokyo Electric Power (Tepco) said. Huge volumes of water, used to cool the plant’s crippled reactors, are being stored at the site.