Detecting Movie Segments Using Gaussian Mixture Models for VOD Lectures with Japanese Subtitles

A search system for VOD (video on demand) lectures is useful if it can be applied beyond searches of only text. To facilitate better searching for movie segments to be used in VOD lectures with Japanese subtitles, we propose a method using subtitles and a solving maximum likelihood estimation from a mixture of Gaussian distributions. The detection was performed by a statistical method by using the expectation-maximization algorithm. This allows for the detecting of movie segments and determining their number. In addition to improving evaluation of movie segments, we provide movie segment rankings. Movie segment rankings are computed for each movie segment using a method that removes one Gaussian distribution from a mixture of Gaussian distributions.


Introduction
In recent years, e-learning systems have proliferated on both the Internet and organizational intranets. Elearning systems such as video on demands (VOD) and web-based training systems are used in many universities. In Okayama University of Science, e-learning systems that employ VOD have been used since 2004 (1) . Many students are attending lectures that are delivered using the VOD service.
Many VOD lectures use learning management systems for delivering and managing student's education and the systems are currently under intensive research. The sharable content object reference model (SCORM) (2) is a common standard format used for e-learning contents. In addition to delivering and managing education, consideration should be given to supports for students in understanding VOD lectures. Further, it is necessary to develop a self-learning system which can be used after attending lectures. In particular, the following features are needed: (a) summary outlining lectures; (b) topic searching including location by relevance to homework and the learner's interest; (c) providing other information on the relationship to previous lectures.
As fundamental research for topic searching, we focus on the detection of movie segments from VOD lectures using search words. To achieve this aim, we proposed a method for detecting movie segments by adopting quadratic functions to generate a histogram containing the frequency distribution of a search word in Japanese subtitles (3) . However, this method cannot determine the correct number of movie segments. Therefore, we propose a method to determine the correct number of movie segments by adapting a Gaussian mixture model for VOD lectures with Japanese subtitles. A statistical expectation-maximization (EM) algorithm (4)(5)(6) can be used to estimate the parameters of the model. This algorithm fits the frequency distribution of a search word with a Gaussian mixture model and detects a movie segment from each Gaussian distribution (normal distribution). In particular, the characteristic of this research is to use the subtitles to estimate the number of movie segments. In addition, we provide movie segment rankings by an evaluation method that removes one normal distribution from a mixture of Gaussian distributions.

E-Learning and Search Systems
In Figure 1, we show a screenshot of a VOD system. It displays a movie of a lecturer on the upper left of the screen, the list of contents on the lower left side, and a slide on the right side. Each lecture consists of three sections. The period of one section varies from about 20 to 30 min. Furthermore, homework regarding the contents of the lecture is given at the end of each lecture. Homework provides a way to test the student's understanding of the lecture. In this research, we have added a new search system based on the histogram of the search word frequency in each interval, as shown on the right side of Figure 1. This search system also displays the movie segments and their rankings based on the search words.

Estimation of Word Frequency Distribution
In this research, we suppose that one simple histogram peak corresponds to one topic of a movie segment and several simple curves synthesize an approximate curve for a histogram. For example, we use the search word " (advertisement)" in the 14th lecture of the database in the cyber campus at Okayama University of Science. The keyword is used in the 14th homework assignment: "In the business model on the Internet, please discuss the advantages of the keyword advertisement. In addition, please think about how to get revenue". The graph of the word frequency histogram and the approximate curve are shown in Figure  2. The horizontal shows the movie length. The histogram has a bin width of 1 min. Thus, the vertical axis shows the frequency of words once every minute. As shown in the figure, we can consider that the histogram is synthesized by four distinct peaks.

Detection of Movie Segments by a Mixture of Gaussian Distributions
Accordingly, we consider that the approximate curve is synthesized by a mixture of Gaussian distributions from the histogram of a search word. The parameter estimation of a Gaussian mixture model is determined using an EM algorithm. Because it is a linear combination of Gaussian distributions (normal distributions), it is possible to estimate a movie segment from the estimated mixture of Gaussian distributions. The definition of the appearance-time of a search word, the Gaussian distribution and the mixture of Gaussian distributions are as follows.
(1) Let n be the number of search words and Xϭ{x 1 , … , x n } be a set of appearance times. (2) The Gaussian distribution (normal distribution) is given as: (3) The mixture of Gaussian distributions is then given as: The number of component Gaussian distributions of the Gaussian mixture model is shown by lϭ1, … , m and the parameters of the component Gaussian distributions are q m ϭ{m 1 , … , m m , s 2 1 , … , s 2 m }, where the weight of the l th Gaussian distribution is w l , the average of the l th Gaussian distribution is m l , and the variance of the l th Gaussian distribution is s 2 l .

Algorithm for the detection of movie segments 4.1.1 EM algorithm
Parameters of the Gaussian mixtures are estimated by the EM algorithm.
(1-1) Initial step: Let m l be the midpoint which divides movie times in l, s 2 l ,ϭ1 and w l ϭ1.  (1-3) M-Step

Detection of movie segments
The Gaussian mixture model of the word frequency distribution which comprises the appearance time for the word is approximated by the EM algorithm. In our VOD lecture, because each section of the lecture is anywhere between 20 and 30 min, we assume that there is a maximum of five movie segments.
These are approximated with the superposition of m, mϭ1, … 5, kinds of Gaussian distributions. From each component's Gaussian distribution of the Gaussian mixture model, we estimate the movie segment using a standard deviation of Ϯs l width from the average m l , which covers 68.26%.

Example of detected movie segments
As an example of detected movie segments, we show an example using the search word " (advertisement)". Table 1 shows movie segments detected a mixture of four Gaussian distributions. The best fitting one is the Akaike's information criterion (AIC) (6) . Table  2 shows the topics and the movie segments of the actual VOD movie. We detect four movie segments (Table 1), but the actual movie consisted of five topics. From Table  2, we conclude that Segments 1 and 2 correspond to Topic numbers 1 and 2, respectively. In addition, Segments 3 and 4 correspond to Topic Numbers 4 and 5, respectively. However, we cannot detect the movie segment corresponding to Topic number 3.

Computation of Movie Segment Rankings
In this research, we consider some evaluation methods to rank movie segments. The main point of the distribution, which removes the th Gaussian distribution from m kinds of Gaussian distributions, is defined by the following formula.

An approximation of the word frequency by kernel density estimation
Initially, we use the kernel density estimation with the Gaussian kernel function. The definitions of kernel   density are as follows.
(1) b is the bandwidth. In this research, we use the bandwidth bϭ0.75, which is the width distribution in 1.5 min.

Ranking of a movie segment
The evaluation of each movie segment computes the influence of the removed distribution on the entire distribution during the movie time tϭmϪs, … , mϩs. In this research, we propose three evaluations.

Gaussian distributions
The characteristic of EV1 is the average of the Kullback-Leibler divergence (6) , which is the difference of the two density functions from the mutual function. .     EV2 evaluates the average of two totals squared. The first term is the difference between the kernel density function and the mixture of the Gaussian distributions. The second term is the difference of the distribution which removes the kth Gaussian distribution and the mixture of the Gaussian distributions.

DETECTING MOVIE SEGMENTS USING GAUSSIAN MIXTURE MODELS FOR VOD LECTURES WITH JAPANESE SUBTITLES 42
EV3: Difference of Kullback-Leibler divergence EV3 computes the Kullback-Leibler divergence of the movie segments every second.

An example of ranking by evaluations
We show an example of experimental evaluation as an estimation of movie segments of the search word " (advertisement)".

The approximation using the Gaussian mixture model
The appearance time of the search word " (ad-

Detecting movie segments by each Gaussian distribution
The movie segments are estimated by the average and variance of each Gaussian distribution of the Gaussian distributions. The detected movie segments of each use of the word " (advertisement)" are shown as a function and the mixture of the Gaussian distributions and the total of the square of the difference of the distribution which removes the k th Gaussian distribution and the mixture of the Gaussian distributions (Table  3). Table 4 shows the movie segments obtained using the quadratic functions (3) . The results of the ranks with-   Table 5. The proposed method is effective for detecting movie segments. The Gaussian distributions and the ranking by each evaluation of movie segments are shown in  Each rank by the evaluation of movie segments is shown by the circled numbers.

Experimental results
We computed the difference between the evaluations of five search words and that by human judgment of the 14th lecture from the database. The averages and variances of the results are shown in Table 6 and Figures 3-5. EV2 and EV3 have a similar rank, whereas EV1 has a different trend. EV2 and EV3 use the difference from the sample kernel density. From the experimental results, we consider that EV1 offers the best evaluation and EV2 gives the worst one. In addition, EV1 tends to choose large segments; EV2 and EV3 tend to choose small segments.

Evaluation of Movie Segment Detection by Users
We evaluated the ranking by EV1 from the viewpoint of users. As a preliminary evaluation, we examined the top five movie segments obtained with the search words "access" , "site" , "search engine" , "keyword" and "advertisement" . The subjects were five students in the graduate school of informatics.
The results of the questionnaires are summarized in Table 7. We observe that the best fit for the detected movie segments tends to have many word frequencies.
The frequencies have a significant influence on the evaluation. In the detection of similar movie segments, the frequencies tend to be more divided. These results are in line with our expectations.

Conclusions and Future Work
In this research, we proposed a method to detect movie segments from VOD lectures with Japanese sub-titles. This method applies an EM algorithm for the search word frequency and makes an approximation using a Gaussian mixture model. Therefore, we provide information for self-learning and homework tips to students after they attend a lecture. Although this research is just a first step in the development of topic searches, we believe that the proposed method provides the learner with effective information about search words. The distribution of a search word and movie segments of the topic may be effective in understanding the topics. As future tasks, we will consider using information from electronic presentations or slides to obtain a more precise detection of movie segments, and we will develop a search system by related words for movie segments. These will support in achieving our goal of providing better support to students for understanding lectures.