1. A charged particle beam apparatus, which detects a defect of a pattern of a sample by irradiating a charged particle beam thereto, the charged particle beam apparatus comprising: a charged particle beam optical system for detecting secondary charged particles by irradiating the charged particle beam to the sample; an image processing unit for, based upon a plurality of images to be inspected that are obtained by the secondary charged particles, obtaining a two-dimensional distribution in a sample surface of an occurrence frequency of defect candidates for each of predetermined regions inside the detected image; and a display unit for displaying the distribution of the occurrence frequency of the defect candidates in association with the pattern.