1. A video summarization method based on mining story structure and semantic relations thereof, comprising: reading a video file composed of a plurality of shots and having a caption file; adding annotations to each shot; using the annotations of each shot as roots to perform concept expansion so as to create a concept expanding tree structure of each shot; calculating dependency degree between each child node and the parent node thereof in each concept expanding tree structure; dividing the annotations in each shot into a plurality of classification types and providing a structured relational graph, the plurality of shots aligned in corresponding groups in the relational graph based on the pertaining classification types, and each shot in the relational graph serving as a vertex; classifying all child nodes in the concept expanding tree based on the plurality of classification types; rearranging the original concept expanding tree into a new bi-level tree using the original annotations as root nodes and the child nodes of each classification type as child nodes of the root nodes; calculating relations of all bi-level trees; and calculating significance of all vertices and edges connected with the vertices in the relational graph in accordance with a graph entropy algorithm, and eliminating less significant vertices and edges based on the calculation result.