1. A system building a map while an image sensor is moving, the system comprising: the image sensor configured to capture images while the image sensor moves relative to one or more different locations; a sub-map building unit configured to recognize a relative location for at least the image sensor of the system using the captured images, build up a sub-map, and if a condition for stopping a building of the sub-map is met, store the sub-map which has been so far built up; an operation determining unit configured to determine whether the condition for stopping building the sub-map; an image group storing unit configured to store an image group including images that are newly captured from the image sensor after the storing of the sub-map when the condition for the stopping of the building of the sub-map is satisfied; and an overall map building unit configured to build an overall map based on the built sub-map and the stored image group when a current relative location for at least the image sensor of the system is determined to be same as a previous relative location for at least the image sensor of the system.