1. A medical image observation assisting system characterized by comprising: volume-region setting means for sequentially setting volume regions each enveloping a part of a luminal organ extending within a subject body, on the basis of three-dimensional image data of the subject body, such that the volume regions are adjacent to each other; luminal-organ-region-information calculating means for repeatedly calculating luminal region data in the form of region information of said part of the luminal organ within each of said volume regions set by said volume-region setting means, on the basis of the three-dimensional image data of the part of the luminal organ within said volume region; luminal-organ-structure-information calculating means for calculating luminal structure data in the form of structure information of the part of the luminal organ within the volume region for which said luminal region data has been calculated by said luminal-organ-region-information calculating means; virtual-centerline generating means for generating a virtual centerline extending in a longitudinal direction of said luminal organ, on the basis of said luminal structure data; virtual-image generating means for generating a virtual image of said luminal organ along said virtual centerline; display means for displaying said virtual image of said luminal organ; and observing-position specifying means for determining an observing position for generating said virtual image, on the basis of at least one of said virtual centerline, said luminal region data and said luminal structure data, such that a region of said luminal organ displayed on said display means has a desired size, and for moving said observing position in the longitudinal direction of said luminal organ, on the basis of said virtual centerline or said luminal structure data.