1. An image processing device comprising: a recognition unit configured to recognize an environment recognition matrix representing a position and an attitude of an environment appearing in an image with respect to a position and an attitude of a terminal that has captured the image; a calculation unit configured to calculate an inverse matrix of the environment recognition matrix; and a manipulation control unit configured to control manipulation of a virtual object arranged in the environment in a three-dimensional manipulation amount according to a difference between a first position or a first attitude based on the inverse matrix of the environment recognition matrix recognized at a first time point and a second position or a second attitude based on the inverse matrix of the environment recognition matrix recognized at a succeeding second time point.