1. A method of generating a de-blurred version of an input image, the method comprising: obtaining, by a computing device, the input image having multiple patches, each patch comprising multiple pixels; on a first of multiple iterations, the computing device: extracting edges from the input image, identifying, for each of the multiple patches in the input image, a sharp patch corresponding to the patch, the sharp patch including a sharp edge and being one of multiple sharp patches in a sharp patch library, generating, based on the identified sharp patches, a predicted sharp version of the input image, generating an estimated blur kernel based on the predicted sharp version of the input image, and generating an intermediate sharpened image based on the input image and the estimated blur kernel; and on subsequent iterations of the multiple iterations, the computing device: extracting edges from an intermediate sharpened image generated in a previous iteration of the multiple iterations the intermediate sharpened image having multiple patches, identifying, for each of the multiple patches in the intermediate sharpened image, a sharp patch from the sharp patch library corresponding to the patch of the intermediate sharpened image, generating, based on the sharp patches identified in the current iteration of the multiple iterations, a predicted sharp version of the input image for the current iteration, the de-blurred version of the input image being generated from the predicted sharp version of the input image for a last of the multiple iterations, generating an estimated blur kernel for the current iteration based on the predicted sharp version of the input image for the current iteration, and generating, based on the intermediate sharpened image for the previous iteration of the multiple iterations and the estimated blur kernel for the current iteration, an intermediate sharpened image for the current iteration.