1. A point cloud data processing device comprising: a first circuit having a structure that selects a square area as a local area, obtains variation of normal vectors at each point in the square area, obtains fitting accuracy of the square area with respect to a local plane, obtains coplanarity of a target point and an adjacent point in the square area, removes points in a non-plane area when the square area is judged to be the non-plane area, and constructs a non-plane, which is not a flat plane and not a curved plane, based on the variation of the normal vectors, the fitting accuracy, and the coplanarity wherein such constructing of the non-plane involves removing all points of an area in which three dimensional edges of an object in the local areas exist based on each of the following processes performed by the first circuit: 1) variation of normal vectors at each point in each of the selected local areas so as to identify and remove points of those selected areas having a local curvature greater than a first predetermined threshold, 2) fitting accuracy of a local plane in each of the selected local areas, and 3) checking coplanarity in a target point in one of the selected local areas and a surrounding point in one of the remaining selected local areas by removing the target point when the largest value of a) an inner product of a first normal vector of the selected local area containing the target point and a vector connecting the target point and a point where a second normal vector intersects the selected local area of the surrounding point and b) an inner product of the second normal vector and the vector connecting the target point and the point where the second normal vector intersects the selected local area of the surrounding point, is greater than a second predetermined threshold; and a second circuit having a structure that labels identical labels on points, other than the removed points, in a plane.