1. A composite charged particle beam apparatus, comprising: an FIB column having an ion beam irradiation axis; an SEM column having an electron beam irradiation axis, the SEM column being arranged relative to the FIB column so that the beam irradiation axes thereof intersect with each other substantially at a right angle; a sample stage for mounting a sample; a detector for detecting secondary particles generated from the sample; an observation image formation portion for forming an FIB image and an SEM image based on a detection signal of the detector; and a display portion for displaying the FIB image and the SEM image such that the horizontal display direction of the sample is the same in both the FIB image and the SEM image.