Krishan Kumar, a spokesman for the National Disaster Response Force, said relief officials rushed to the remote area in Gajapati district in Orissa state.
The landslide hit on Friday, a day after a severe cyclone that caused heavy rains in parts of eastern India.
Villagers had moved to the shelter to escape the fury of Cyclone Titli, which whipped up wind speeds of up to 95 miles per hour.
State Relief Commissioner BP Sethi said there were reports that 12 people at the shelter were killed and another four trapped in the debris, the Press Trust of India news agency said.
Mr Sethi said rescue workers were clearing roads blocked by uprooted trees to reach the remote site.
At least eight people were killed in Andhra Pradesh state and one in Orissa state on Thursday when the cyclone damaged homes and blew down trees and power poles.
Orissa state is prone to cyclones that develop in the Bay of Bengal. In 1999, a devastating cyclone killed more than 15,000 people.