1. A method of improving image quality of a charged particle beam device, the method comprising: a charged particle beam irradiating step of irradiating and scanning charged particle beam converged over a sample that is an object to be image-captured; a detected image generating step of detecting secondary charged particles or reflection charged particles generated from the sample by the charged particle beam by using two or more detectors that are arranged at different locations and of generating detected images corresponding to respective outputs of the detectors; and a detected image synthesizing step by a method of synthesizing the number of output images less than the number of the detectors based on pattern information by using the detected image or a method of synthesizing the number of output images less than the number of the detectors by calculating a synthesis control parameter, wherein the detected image synthesizing step calculates a direction-dependent blocking state of scattered electrons based on pattern information, and determines a synthesis ratio based on the blocking state.