1. A device for removing distortion, the device comprising: a photographing unit configured to photograph a plurality of frames in succession by using a rolling shutter; a storage unit configured to store photographed images of the plurality of frames; a local motion vector generator configured to generate a local motion vector between a reference image and an input image among the photographed images; a parameter estimator configured to estimate a parameter set indicative of an amount of distortion in the input image in view of the reference image, based on the local motion vector; and an image reconstruction unit configured to generate a reconstructed image by using the estimated parameter set and information about the input image.