Typhoon Jebi made landfall in the west of the country, bringing heavy rain and winds of up to 216km/h (135 mph).
In Osaka Bay it swept a tanker into a bridge and in Kyoto parts of the train station roof came down.
There are no reports of casualties and the storm is expected to weaken as it continues to move across the country. The storm made landfall on Shikoku island around noon on Tuesday local time and then moved across Japan's largest main island of Honshu.
There are warnings of high waves, flooding and mudslides. It has already left tens of thousands without power and authorities have urged people to move to safety.
Hundreds of flights, trains and ferries had to be cancelled. Flooding covered the runways at Kansai International Airport in Osaka.
Universal Studios Japan, a popular amusement park near Osaka, was closed.
Typhoon Jebi swept a tanker into a bridge in Kyoto
Prime Minister Shinzo Abe convened an emergency meeting and called on people "to take action to protect your lives, including preparing and evacuating early.”
Footage from the storm making landfall showed giant waves crashing against the coastline and flying debris.
Japan's weather agency has warned of possible landslides, flooding and violent winds, as well as high tides, lightning and tornadoes in the areas affected.The country is regularly struck by major storms and this summer has been one of extreme weather.
In July landslides and massive floods - the worst in decades - killed more than 200 people. That was followed by a record deadly heatwave.