1. An audio analysis apparatus comprising: a component acquisition part that acquires a component matrix composed of an array of component values from an audio signal which is divided into a sequence of unit periods in a time-axis direction, columns of the component matrix corresponding to the sequence of unit periods of the audio signal and rows of the component matrix corresponding to a series of unit bands of the audio signal arranged in a frequency-axis direction, the component value representing a spectrum component of the audio signal belonging to the corresponding unit period and belonging to the corresponding unit band; a difference generation part that generates a plurality of shift matrices each obtained by shifting the columns of the component matrix in the time-axis direction with a different shift amount, and that generates a plurality of difference matrices each composed of an array of element values in correspondence to the plurality of the shift matrices, the element value representing a difference between the corresponding component value of the shift matrix and the corresponding component value of the component matrix; and a feature amount extraction part that generates a tonal feature amount including a plurality of series of feature values corresponding to the plurality of difference matrices, one series of feature values corresponding to the series of unit bands of the difference matrix, one feature value representing a sequence of element values arranged in the time-axis direction at the corresponding unit band of the difference matrix.