The Imperial Family doesn't like when members of the public are on the Imperial Grounds.
The imperial family still resides in the palace, so the general public is admitted to the grounds on only two days each on 2 January and 23 December.