1. A device for removing distortion, the device comprising: a photographing unit configured to photograph a plurality of frames in succession by using a rolling shutter; a storage unit configured to store photographed images of the plurality of frames; a local motion vector generator configured to generate a local motion vector between a reference image and an input image among the photographed images; a parameter estimator configured to estimate a parameter set indicative of an amount of distortion in the input image in view of the reference image, based on the local motion vector; and an image reconstruction unit configured to generate a reconstructed image by using the estimated parameter set and information about the input image, wherein the parameter estimator is further configured to estimate the parameter set by selecting at least one scan line in the reference image and at least one corresponding line in the input image, and wherein the parameter set is a set of variables representing coordinates of a start point and an end point of the corresponding line which are located at a first curved boundary line and a second curve boundary line of the input image, respectively.