1. A method of improving image quality of a charged particle beam device, the method comprising: a charged particle beam irradiating step of irradiating and scanning charged particle beam converged over a sample that is an object to be image-captured; a design data reading step of reading a design data corresponding to the sample at a position irradiated with the charged particle beam; a detected image generating step of detecting secondary charged particles or reflection charged particles generated from the sample by the charged particle beam by using two or more detectors that are arranged at different locations and of generating detected images corresponding to respective outputs of the detectors; an output image generation controlling step of controlling a method of generating one or more output images from the detected images by using the read design data; and an output image generating step of generating one or more output images from the detected images in accordance with the method of generating the output images which is determined in the output image generation controlling step.