Prehistoric sacred structures were often build specifically for special ceremonies, then dismantled after the rituals were complete. 
The ritual dismantling of the shrines every 20 years (known as sengu-shiki) goes back to prehistoric times, when sacred structures tended to be erected for special ceremonies rather than as permanent places of worship.