1. A charged particle beam apparatus comprising: a specimen chamber containing a specimen stage for holding a specimen; a charged particle beam column for scanning on the specimen with a primary charged particle beam, and detecting a secondary particle generated from the specimen to thereby output results of detection in the form of an output signal; and the charged particle beam column comprising: a magnetic lens unit for causing a magnetic field excited by current to act on the primary charged particle beam; a control power supply source for executing a process for reducing magnetic hysteresis present in the magnetic lens unit by supplying the magnetic lens unit with excitation current of a predetermined magnitude for a predetermined time only; and an operation unit for stopping the process for reducing the magnetic hysteresis at a point in time when a diameter of a spot of the primary charged particle beam, on the specimen, becomes smaller than a set value.