This article is 16 years old. Images might not display.
Operator Nippon Yusen said in a statement on its website that the 150,000-ton tanker Takayama was attacked while it was heading for Saudi Arabia.
None of the vessel's 23 crew members - seven Japanese and 16 Filipinos - were injured and the extent of damage to the tanker is being investigated.
The Kyodo news agency reported the tanker was leaking fuel as a result of the attack.
Yemen has the misfortune to lie across the Gulf of Aden from Somalia, the world's most dangerous and unstable country.
Somalia has not had a central government for more than 17 years and the country is believed to harbour several active Al Quada cells. Piracy in the waters between Yemen and Somalia is rife, but vessels passing between the Indian Ocean and the Suez Canal must pass through the Gulf of Aden.
Last year more than 25 ships were seized by pirates in Somali coastal waters despite US navy patrols.
The International Maritime Bureau advises merchant ships to stay at least 200 nautical miles (370km) from the Somali coast because of the risk of piracy.