1. An apparatus for generating a skeleton model, the apparatus comprising: a data synchronization unit to perform a synchronization upon image data for a user collected by an image sensor and motion data for the user collected by a motion sensor; a silhouette extraction unit to extract a silhouette of the user from the synchronized image data; a joint information calculation unit to calculate a position or orientation of a joint of the user from the synchronized motion data; and a skeleton model generation unit to generate a three-dimensional (3D) skeleton model of the user using the silhouette and the position or orientation of the joint of the user, wherein the skeleton model generation unit sets a reference point extracted from the silhouette and a position or orientation of the joint of the user included in the motion data, initializes positions or orientations of remaining joints of the user using the a position or orientation of the joint of the user included in the motion data, and determines positions or orientations of remaining joints of the user to be in a range of the silhouette.