1. A microscope system, comprising: a first microscope that generates an observation image of a specimen; an image obtaining unit that obtains an RGB image of the specimen; a spectroscopic information obtaining unit that obtains spectroscopic information of the specimen; a first analyzer that analyzes the RGB image to calculate feature data of the specimen; a determining unit that: compares the feature data of the specimen calculated by the first analyzer with a threshold value to detect one or more abnormal sites in the observation image of the specimen; and determines that it is necessary to obtain the spectroscopic information in response to detecting one or more abnormal sites in the observation image of the specimen; and a control unit that controls the spectroscopic information obtaining unit to obtain the spectroscopic information in response to the determining unit determining that it is necessary to obtain the spectroscopic information.