1. An image processing device comprising: an image acquisition unit configured to acquire a first image and a second image with a parallax, which are taken as a stereoscopic image; a parallax calculation unit configured to calculate a parallax indicating a shift amount of a corresponding pixel of the second image with respect to each pixel of the first image acquired by the image acquisition unit; an image processing coefficient decision unit configured to decide an image processing coefficient to perform edge emphasis or edge correction with respect to every image processing target pixel of the first image and the second image acquired by the image acquisition unit, and decide the image processing coefficient for the target pixel based on the parallax calculated by the parallax calculation unit; and an image processing unit configured to perform image processing using the image processing coefficient decided by the image processing coefficient decision unit for the first image and the second image acquired by the image acquisition unit, wherein the image processing coefficient decision unit decides a filter coefficient as the image processing coefficient, the image processing unit performs filter processing by a filter using the filter coefficient, and the image processing coefficient decision unit decides a filter coefficient in which at least one of a degree of the edge emphasis or the edge correction and a frequency band varies according to the parallax, wherein the image processing coefficient decision unit decides at least one of a filter coefficient that reduces the degree of the edge emphasis or the edge correction as the parallax increases and a filter coefficient that widens the frequency band as the parallax increases.