A unordered text document is one where sentences in the document are disordered or jumbled. It doesn't appear that unordered text documents appear in corpora, but rather are introduced as part of processing pipeline.
ABSTRACTIVE
To structure an unordered document is an essential task in many applications. It is a post-requisite for applications like multiple document extractive text summarization where we have to present a summary of multiple documents. It is a prerequisite for applications like question answering from multiple documents where we have to present an answer by processing multiple documents. In this paper, we address the task of segmenting an unordered text document into different sections. The input document/summary that may have unordered sentences is processed so that it will have sentences clustered together. Clustering is based on the similarity with the respective keyword as well as with the sentences belonging to the same cluster. Keywords are identified and clusters are formed for each keyword. To test our approach, we jumble the ordering of sentences in a document, process the unordered document and compare the similarity of the output document with the original document.
1901.10133
Introduction </s> To Ġstructure Ġan Ġun ordered Ġdocument Ġis Ġan Ġessential Ġtask Ġin Ġmany Ġapplications . ĠIt Ġis Ġa Ġpost - requisite Ġfor Ġapplications Ġlike Ġmultiple Ġdocument Ġextract ive Ġtext Ġsummar ization Ġwhere Ġwe Ġhave Ġto Ġpresent Ġa Ġsummary Ġof Ġmultiple Ġdocuments . ĠIt Ġis Ġa Ġprerequisite Ġfor Ġapplications Ġlike Ġquestion Ġanswering Ġfrom Ġmultiple Ġdocuments Ġwhere Ġwe Ġhave Ġto Ġpresent Ġan Ġanswer Ġby Ġprocessing Ġmultiple Ġdocuments . ĠIn Ġthis Ġpaper , Ġwe Ġaddress Ġthe Ġtask Ġof Ġsegment ing Ġan Ġun ordered Ġtext Ġdocument Ġinto Ġdifferent Ġsections . ĠThe Ġinput Ġdocument / summary Ġthat Ġmay Ġhave Ġun ordered Ġsentences Ġis Ġprocessed Ġso Ġthat Ġit Ġwill Ġhave Ġsentences Ġclustered Ġtogether . ĠCl ust ering Ġis Ġbased Ġon Ġthe Ġsimilarity Ġwith Ġthe Ġrespective Ġkeyword Ġas Ġwell Ġas Ġwith Ġthe Ġsentences Ġbelonging Ġto Ġthe Ġsame Ġcluster . ĠKey words Ġare Ġidentified Ġand Ġclusters Ġare Ġformed Ġfor Ġeach Ġkeyword . </s> We Ġuse ĠText Rank Ġalgorithm ĠB IB REF 0 Ġto Ġextract Ġthe Ġkeywords Ġfrom Ġa Ġtext Ġdocument . ĠText Rank Ġis Ġa Ġgraph - based Ġranking Ġalgorithm Ġwhich Ġdecides Ġthe Ġimportance Ġof Ġa Ġvertex Ġwithin Ġa Ġgraph Ġby Ġconsidering Ġthe Ġglobal Ġinformation Ġrec urs ively Ġcomputed Ġfrom Ġthe Ġentire Ġgraph Ġrather Ġthan Ġfocusing Ġon Ġlocal Ġvertex Ġspecific Ġinformation . ĠThe Ġmodel Ġuses Ġknowledge Ġacquired Ġfrom Ġthe Ġentire Ġtext Ġto Ġextract Ġthe Ġkeywords . ĠA Ġcluster Ġis Ġgenerated Ġfor Ġevery Ġkeyword . </s> While Ġgenerating Ġclusters , Ġthe Ġsimilarity Ġbetween Ġa Ġsentence Ġand Ġa Ġtopic / key word Ġis Ġcalculated Ġusing Ġthe Ġcos ine Ġsimilarity Ġof Ġembed d ings Ġgenerated Ġusing ĠGoogle 's ĠUSE Ġ( Universal ĠSent ence ĠEnc oder ) ĠB IB REF 1 Ġ. ĠUSE Ġis Ġclaimed Ġto Ġhave Ġbetter Ġperformance Ġof Ġtransfer Ġlearning Ġwhen Ġused Ġwith Ġsentence - level Ġembed d ings Ġas Ġcompared Ġto Ġword - level Ġembed d ings Ġalone . ĠThis Ġmodel Ġis Ġclaimed Ġto Ġhave Ġbetter Ġperformance Ġeven Ġif Ġthere Ġis Ġless Ġtask - specific Ġtraining Ġdata Ġavailable . ĠWe Ġobserved Ġthat Ġthe Ġquality Ġof Ġclusters / section Ġis Ġbetter Ġif Ġthe Ġsimilarity Ġof Ġa Ġsentence Ġwith Ġthe Ġkeyword Ġis Ġconsidered Ġalong Ġwith Ġthe Ġsimilarity Ġof Ġthe Ġsentence Ġwith Ġthe Ġsentences Ġalready Ġavailable Ġin Ġthe Ġrespective Ġsection . </s> To Ġtest Ġour Ġapproach , Ġwe Ġj umble Ġthe Ġordering Ġof Ġsentences Ġin Ġa Ġdocument , Ġprocess Ġthe Ġun ordered Ġdocument Ġand Ġcompare Ġthe Ġsimilarity Ġof Ġthe Ġoutput Ġdocument Ġwith Ġthe Ġoriginal Ġdocument . </s> Related ĠWork </s> Several Ġmodels Ġhave Ġbeen Ġperformed Ġin Ġthe Ġpast Ġto Ġretrieve Ġsentences Ġof Ġa Ġdocument Ġbelonging Ġto Ġa Ġparticular Ġtopic ĠB IB REF 2 Ġ. ĠGiven Ġa Ġtopic , Ġretrieving Ġsentences Ġthat Ġmay Ġbelong Ġto Ġthat Ġtopic Ġshould Ġbe Ġconsidered Ġas Ġa Ġdifferent Ġtask Ġthan Ġwhat Ġwe Ġaim Ġin Ġthis Ġpaper . ĠA Ġgraph Ġbased Ġapproach Ġfor Ġextracting Ġinformation Ġrelevant Ġto Ġa Ġquery Ġis Ġpresented Ġin ĠB IB REF 3 Ġ, Ġwhere Ġsub graph s Ġare Ġbuilt Ġusing Ġthe Ġrelated ness Ġof Ġthe Ġsentences Ġto Ġthe Ġquery . ĠAn Ġincremental Ġintegrated Ġgraph Ġto Ġrepresent Ġthe Ġsentences Ġin Ġa Ġcollection Ġof Ġdocuments Ġis Ġpresented Ġin ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġ. ĠSent ences Ġfrom Ġthe Ġdocuments Ġare Ġmerged Ġinto Ġa Ġmaster Ġsequence Ġto Ġimprove Ġco herence Ġand Ġflow . ĠThe Ġsame Ġordering Ġis Ġused Ġfor Ġsequencing Ġthe Ġsentences Ġin Ġthe Ġextracted Ġsummary . ĠOrder ing Ġof Ġsentences Ġin Ġa Ġdocument Ġis Ġdiscussed Ġin ĠB IB REF 6 Ġ. </s> In Ġthis Ġpaper , Ġwe Ġaim Ġto Ġgenerate Ġthe Ġsections / cl usters Ġfrom Ġan Ġun ordered Ġdocument . ĠTo Ġthe Ġbest Ġof Ġour Ġknowledge , Ġthis Ġis Ġa Ġfirst Ġattempt Ġto Ġaddress Ġthis Ġproblem Ġformally . </s> Prop osed Ġmodel </s> Our Ġmethodology Ġis Ġdescribed Ġin Ġthe ĠFigure Ġ1 Ġ. ĠThe Ġprocess Ġstarts Ġby Ġtaking Ġan Ġun ordered Ġdocument Ġas Ġan Ġinput . ĠThe Ġnext Ġstep Ġis Ġto Ġextract Ġthe Ġkeywords Ġfrom Ġthe Ġinput Ġdocument Ġusing ĠText Rank Ġalgorithm ĠB IB REF 0 Ġand Ġstore Ġthem Ġin Ġa Ġlist Ġ$ K $ Ġ. ĠThe Ġkeywords Ġstored Ġin Ġ$ K $ Ġact Ġas Ġcent ro ids Ġfor Ġthe Ġclusters . ĠNote Ġthat , Ġthe Ġquality Ġof Ġkeywords Ġextracted Ġwill Ġhave Ġa Ġbearing Ġon Ġthe Ġfinal Ġresults . ĠIn Ġthis Ġpaper , Ġwe Ġpresent Ġa Ġmodel Ġthat Ġcan Ġbe Ġused Ġfor Ġstruct uring Ġan Ġun struct ured Ġdocument . ĠIn Ġthe Ġprocess , Ġwe Ġuse Ġa Ġpopular Ġkeyword Ġextraction Ġalgorithm . ĠOur Ġmodel Ġis Ġnot Ġbound Ġto ĠText Rank , Ġand Ġif Ġa Ġbetter Ġkeyword Ġextraction Ġalgorithm Ġis Ġavailable , Ġit Ġcan Ġreplace ĠText Rank . </s> The Ġnext Ġstep Ġis Ġto Ġfind Ġthe Ġmost Ġrelevant Ġsentence Ġfor Ġeach Ġkeyword Ġin Ġ$ K $ Ġ. ĠThe Ġmost Ġrelevant Ġsentence Ġis Ġmapped Ġto Ġthe Ġkeyword Ġand Ġassigned Ġin Ġthe Ġrespective Ġcluster . ĠThis Ġsimilarity Ġbetween Ġthe Ġkeyword Ġand Ġthe Ġsentence Ġis Ġcalculated Ġby Ġthe Ġcos ine Ġsimilarity Ġof Ġembed d ings Ġgenerated Ġfrom ĠGoogle 's ĠUSE . ĠNow , Ġwe Ġhave Ġa Ġlist Ġof Ġkeywords Ġ$ K $ Ġ, Ġand Ġa Ġsentence Ġmapped Ġto Ġeach Ġkeyword . ĠThe Ġnext Ġstep Ġis Ġto Ġmap Ġthe Ġremaining Ġsentences . ĠIn Ġthe Ġnext Ġstep , Ġwe Ġgo Ġthrough Ġall Ġthe Ġsentences Ġin Ġthe Ġdocument Ġthat Ġhave Ġnot Ġbeen Ġmapped Ġyet Ġand Ġfind Ġthe Ġrelevant Ġkeywords Ġthat Ġcan Ġbe Ġmapped Ġwith Ġthem . ĠWe Ġdo Ġthis Ġby Ġthe Ġfollowing Ġprocedure : </s> $ S _ 1 ( x , y ) $ Ġis Ġthe Ġsimilarity Ġbetween Ġthe Ġcurrent Ġsentence Ġ$ x $ Ġand Ġthe Ġkeyword Ġ$ y $ Ġ. Ġ$ S _ 2 ( x , y ) $ Ġis Ġthe Ġmaximum Ġsimilarity Ġbetween Ġthe Ġcurrent Ġsentence Ġ$ x $ Ġand Ġthe Ġsentences Ġthat Ġare Ġalready Ġmapped Ġwith Ġthe Ġkeyword Ġ$ y $ Ġ. ĠIf Ġ$ y $ Ġhas Ġthree Ġsentences Ġmapped Ġto Ġit , Ġthen Ġthe Ġsimilarity Ġbetween Ġ$ x $ Ġand Ġthe Ġsentences Ġmapped Ġto Ġ$ y $ Ġare Ġcomputed , Ġand Ġthe Ġmaximum Ġsimilarity Ġamong Ġthe Ġthree Ġis Ġassigned Ġto Ġ$ S _ 2 ( x , y ) $ Ġ. ĠThe Ġoverall Ġsimilarity Ġ$ x $ 0 Ġis Ġcalculated Ġas : </s> $$ S ( x , y )= Ġt * S _ 1 ( x , y ) Ġ+ Ġ( 1 - t )* S _ 2 ( x , y ) $$ Ġ Ġ Ġ( E q . Ġ1 ) </s> We Ġmap Ġevery Ġother Ġsentence Ġ$ x $ Ġto Ġthe Ġkeyword Ġwith Ġwhich Ġthey Ġhave Ġmaximum Ġsimilarity Ġ$ Max _ y Ġ( S ( x , y )) $ Ġ. ĠLater , Ġwe Ġcluster Ġthe Ġkeywords Ġalong Ġwith Ġthe Ġsentences Ġmapped Ġto Ġthem . ĠThe Ġvalue Ġof Ġ$ t $ Ġsignifies Ġthe Ġimportance Ġto Ġbe Ġgiven Ġto Ġthe Ġkeyword Ġand Ġits Ġassociated Ġsentences Ġrespectively . ĠIn Ġour Ġexperiments , Ġwe Ġempir ically Ġfix Ġthe Ġvalue Ġof Ġ$ t $ Ġto Ġ$ 0 . 25 $ Ġ. </s> Met rics Ġto Ġevaluate Ġour Ġalgorithm </s> To Ġevaluate Ġour Ġalgorithm , Ġwe Ġpropose Ġtwo Ġsimilarity Ġmetrics , Ġ$ Sim 1 $ Ġand Ġ$ Sim 2 $ Ġ. ĠThese Ġmetrics Ġcompute Ġthe Ġsimilarity Ġof Ġeach Ġsection Ġof Ġthe Ġoriginal Ġdocument Ġwith Ġall Ġthe Ġsections / cl usters Ġ( key word Ġand Ġthe Ġsentences Ġmapped Ġto Ġit ) Ġof Ġthe Ġoutput Ġdocument Ġand Ġassign Ġthe Ġmaximum Ġsimilarity . Ġ$ Sim 1 $ Ġbetween Ġan Ġinput Ġsection Ġand Ġan Ġoutput Ġsection Ġis Ġcalculated Ġas Ġthe Ġnumber Ġof Ġsentences Ġof Ġthe Ġinput Ġsection Ġthat Ġare Ġpresent Ġin Ġthe Ġoutput Ġsection Ġdivided Ġby Ġthe Ġtotal Ġnumber Ġof Ġsentences Ġin Ġthe Ġinput Ġsection . ĠTo Ġcalculate Ġthe Ġfinal Ġsimilarity Ġ( similar ity Ġof Ġthe Ġentire Ġoutput Ġdocument ) Ġwe Ġtake Ġthe Ġweighted Ġmean Ġof Ġsimilarity Ġcalculated Ġcorresponding Ġto Ġeach Ġinput Ġsection . Ġ$ Sim 2 $ Ġbetween Ġan Ġinput Ġsection Ġand Ġan Ġoutput Ġsection Ġis Ġcomputed Ġas Ġthe Ġnumber Ġof Ġsentences Ġof Ġan Ġinput Ġsection Ġthat Ġare Ġpresent Ġin Ġan Ġoutput Ġsection Ġdivided Ġby Ġthe Ġsum Ġof Ġsentences Ġin Ġthe Ġinput Ġand Ġoutput Ġsections . ĠThe Ġfinal Ġsimilarity Ġis Ġcomputed Ġin Ġa Ġsimilar Ġmanner . </s> Exper imental Ġsetup Ġand ĠResults </s> For Ġour Ġexperiments , Ġwe Ġprepared Ġfive Ġsets Ġof Ġdocuments . ĠEach Ġset Ġhas Ġ100 Ġwiki Ġdocuments Ġ( random ly Ġchosen ). ĠEach Ġdocument Ġis Ġrestruct ured Ġrandomly Ġ( sent ences Ġare Ġrearr anged Ġrandomly ). ĠThis Ġrestruct ured Ġdocument Ġis Ġthe Ġinput Ġto Ġour Ġmodel Ġand Ġthe Ġoutput Ġdocument Ġis Ġcompared Ġagainst Ġthe Ġoriginal Ġinput Ġdocument . </s> Also , Ġwe Ġcompare Ġour Ġresults Ġwith Ġthe Ġbaseline Ġbeing Ġthe Ġresults Ġwhen Ġwe Ġconsider Ġonly Ġthe Ġsimilarity Ġbetween Ġsentences Ġand Ġkeywords . ĠThe Ġresults Ġare Ġshown Ġin ĠTable Ġ1 Ġ. ĠHere , Ġboth Ġ$ Sim 1 $ Ġand Ġ$ Sim 2 $ Ġare Ġthe Ġmean Ġof Ġsimilarities Ġof Ġthe Ġentire ĠSet . Ġ$ Base Sim 1 $ Ġand Ġ$ Base Sim 1 $ Ġare Ġcomputed Ġsimilar Ġto Ġ$ Sim 1 $ Ġand Ġ$ Sim 2 $ Ġrespectively Ġbut Ġwith Ġthe Ġ$ t $ Ġvalue Ġas Ġ1 Ġin ĠEqu ation Ġ1 Ġ. ĠIt Ġis Ġevident Ġthat Ġthe Ġresults Ġare Ġbetter Ġif Ġboth Ġsimilarities Ġ( Ġ$ S _ 1 ( x , y ) $ Ġand Ġ$ S _ 2 ( x , y ) $ Ġ) Ġare Ġconsidered . </s> Con clusions </s> We Ġproposed Ġan Ġefficient Ġmodel Ġto Ġstructure Ġan Ġun ordered Ġdocument . ĠWe Ġevaluated Ġour Ġmodel Ġagainst Ġthe Ġbaseline Ġand Ġfound Ġthat Ġour Ġproposed Ġmodel Ġhas Ġa Ġsignificant Ġimprovement . ĠWe Ġobserved Ġthat Ġwhile Ġordering Ġan Ġun ordered Ġdocument , Ġthe Ġinitial Ġsentences Ġassociated Ġwith Ġa Ġkeyword / topic Ġplay Ġa Ġsignificant Ġrole .
Introduction To structure an unordered document is an essential task in many applications. It is a post-requisite for applications like multiple document extractive text summarization where we have to present a summary of multiple documents. It is a prerequisite for applications like question answering from multiple documents where we have to present an answer by processing multiple documents. In this paper, we address the task of segmenting an unordered text document into different sections. The input document/summary that may have unordered sentences is processed so that it will have sentences clustered together. Clustering is based on the similarity with the respective keyword as well as with the sentences belonging to the same cluster. Keywords are identified and clusters are formed for each keyword. We use TextRank algorithm BIBREF0 to extract the keywords from a text document. TextRank is a graph-based ranking algorithm which decides the importance of a vertex within a graph by considering the global information recursively computed from the entire graph rather than focusing on local vertex specific information. The model uses knowledge acquired from the entire text to extract the keywords. A cluster is generated for every keyword. While generating clusters, the similarity between a sentence and a topic/keyword is calculated using the cosine similarity of embeddings generated using Google's USE (Universal Sentence Encoder) BIBREF1 . USE is claimed to have better performance of transfer learning when used with sentence-level embeddings as compared to word-level embeddings alone. This model is claimed to have better performance even if there is less task-specific training data available. We observed that the quality of clusters/section is better if the similarity of a sentence with the keyword is considered along with the similarity of the sentence with the sentences already available in the respective section. To test our approach, we jumble the ordering of sentences in a document, process the unordered document and compare the similarity of the output document with the original document. Related Work Several models have been performed in the past to retrieve sentences of a document belonging to a particular topic BIBREF2 . Given a topic, retrieving sentences that may belong to that topic should be considered as a different task than what we aim in this paper. A graph based approach for extracting information relevant to a query is presented in BIBREF3 , where subgraphs are built using the relatedness of the sentences to the query. An incremental integrated graph to represent the sentences in a collection of documents is presented in BIBREF4 , BIBREF5 . Sentences from the documents are merged into a master sequence to improve coherence and flow. The same ordering is used for sequencing the sentences in the extracted summary. Ordering of sentences in a document is discussed in BIBREF6 . In this paper, we aim to generate the sections/clusters from an unordered document. To the best of our knowledge, this is a first attempt to address this problem formally. Proposed model Our methodology is described in the Figure 1 . The process starts by taking an unordered document as an input. The next step is to extract the keywords from the input document using TextRank algorithm BIBREF0 and store them in a list $K$ . The keywords stored in $K$ act as centroids for the clusters. Note that, the quality of keywords extracted will have a bearing on the final results. In this paper, we present a model that can be used for structuring an unstructured document. In the process, we use a popular keyword extraction algorithm. Our model is not bound to TextRank, and if a better keyword extraction algorithm is available, it can replace TextRank. The next step is to find the most relevant sentence for each keyword in $K$ . The most relevant sentence is mapped to the keyword and assigned in the respective cluster. This similarity between the keyword and the sentence is calculated by the cosine similarity of embeddings generated from Google's USE. Now, we have a list of keywords $K$ , and a sentence mapped to each keyword. The next step is to map the remaining sentences. In the next step, we go through all the sentences in the document that have not been mapped yet and find the relevant keywords that can be mapped with them. We do this by the following procedure: $S_1(x,y)$ is the similarity between the current sentence $x$ and the keyword $y$ . $S_2(x,y)$ is the maximum similarity between the current sentence $x$ and the sentences that are already mapped with the keyword $y$ . If $y$ has three sentences mapped to it, then the similarity between $x$ and the sentences mapped to $y$ are computed, and the maximum similarity among the three is assigned to $S_2(x,y)$ . The overall similarity $x$0 is calculated as: $$S(x,y)= t*S_1(x,y) + (1-t)*S_2(x,y)$$   (Eq. 1) We map every other sentence $x$ to the keyword with which they have maximum similarity $Max_y (S(x,y))$ . Later, we cluster the keywords along with the sentences mapped to them. The value of $t$ signifies the importance to be given to the keyword and its associated sentences respectively. In our experiments, we empirically fix the value of $t$ to $0.25$ . Metrics to evaluate our algorithm To evaluate our algorithm, we propose two similarity metrics, $Sim1$ and $Sim2$ . These metrics compute the similarity of each section of the original document with all the sections/clusters (keyword and the sentences mapped to it) of the output document and assign the maximum similarity. $Sim1$ between an input section and an output section is calculated as the number of sentences of the input section that are present in the output section divided by the total number of sentences in the input section. To calculate the final similarity (similarity of the entire output document) we take the weighted mean of similarity calculated corresponding to each input section. $Sim2$ between an input section and an output section is computed as the number of sentences of an input section that are present in an output section divided by the sum of sentences in the input and output sections. The final similarity is computed in a similar manner. Experimental setup and Results For our experiments, we prepared five sets of documents. Each set has 100 wiki documents (randomly chosen). Each document is restructured randomly (sentences are rearranged randomly). This restructured document is the input to our model and the output document is compared against the original input document. Also, we compare our results with the baseline being the results when we consider only the similarity between sentences and keywords. The results are shown in Table 1 . Here, both $Sim1$ and $Sim2$ are the mean of similarities of the entire Set. $BaseSim1$ and $BaseSim1$ are computed similar to $Sim1$ and $Sim2$ respectively but with the $t$ value as 1 in Equation 1 . It is evident that the results are better if both similarities ( $S_1(x,y)$ and $S_2(x,y)$ ) are considered. Conclusions We proposed an efficient model to structure an unordered document. We evaluated our model against the baseline and found that our proposed model has a significant improvement. We observed that while ordering an unordered document, the initial sentences associated with a keyword/topic play a significant role.
What is an unordered text document, do these arise in real-world corpora?
559920ebe19e99e43418c2f0455a0ffdc8edaaa2
What Ġis Ġan Ġun ordered Ġtext Ġdocument , Ġdo Ġthese Ġarise Ġin Ġreal - world Ġcorpor a ?
A Ġun ordered Ġtext Ġdocument Ġis Ġone Ġwhere Ġsentences Ġin Ġthe Ġdocument Ġare Ġdis ordered Ġor Ġj umbled . ĠIt Ġdoesn 't Ġappear Ġthat Ġun ordered Ġtext Ġdocuments Ġappear Ġin Ġcorpor a , Ġbut Ġrather Ġare Ġintroduced Ġas Ġpart Ġof Ġprocessing Ġpipeline .
<s> What Ġis Ġan Ġun ordered Ġtext Ġdocument , Ġdo Ġthese Ġarise Ġin Ġreal - world Ġcorpor a ? </s> Introduction </s> To Ġstructure Ġan Ġun ordered Ġdocument Ġis Ġan Ġessential Ġtask Ġin Ġmany Ġapplications . ĠIt Ġis Ġa Ġpost - requisite Ġfor Ġapplications Ġlike Ġmultiple Ġdocument Ġextract ive Ġtext Ġsummar ization Ġwhere Ġwe Ġhave Ġto Ġpresent Ġa Ġsummary Ġof Ġmultiple Ġdocuments . ĠIt Ġis Ġa Ġprerequisite Ġfor Ġapplications Ġlike Ġquestion Ġanswering Ġfrom Ġmultiple Ġdocuments Ġwhere Ġwe Ġhave Ġto Ġpresent Ġan Ġanswer Ġby Ġprocessing Ġmultiple Ġdocuments . ĠIn Ġthis Ġpaper , Ġwe Ġaddress Ġthe Ġtask Ġof Ġsegment ing Ġan Ġun ordered Ġtext Ġdocument Ġinto Ġdifferent Ġsections . ĠThe Ġinput Ġdocument / summary Ġthat Ġmay Ġhave Ġun ordered Ġsentences Ġis Ġprocessed Ġso Ġthat Ġit Ġwill Ġhave Ġsentences Ġclustered Ġtogether . ĠCl ust ering Ġis Ġbased Ġon Ġthe Ġsimilarity Ġwith Ġthe Ġrespective Ġkeyword Ġas Ġwell Ġas Ġwith Ġthe Ġsentences Ġbelonging Ġto Ġthe Ġsame Ġcluster . ĠKey words Ġare Ġidentified Ġand Ġclusters Ġare Ġformed Ġfor Ġeach Ġkeyword . </s> We Ġuse ĠText Rank Ġalgorithm ĠB IB REF 0 Ġto Ġextract Ġthe Ġkeywords Ġfrom Ġa Ġtext Ġdocument . ĠText Rank Ġis Ġa Ġgraph - based Ġranking Ġalgorithm Ġwhich Ġdecides Ġthe Ġimportance Ġof Ġa Ġvertex Ġwithin Ġa Ġgraph Ġby Ġconsidering Ġthe Ġglobal Ġinformation Ġrec urs ively Ġcomputed Ġfrom Ġthe Ġentire Ġgraph Ġrather Ġthan Ġfocusing Ġon Ġlocal Ġvertex Ġspecific Ġinformation . ĠThe Ġmodel Ġuses Ġknowledge Ġacquired Ġfrom Ġthe Ġentire Ġtext Ġto Ġextract Ġthe Ġkeywords . ĠA Ġcluster Ġis Ġgenerated Ġfor Ġevery Ġkeyword . </s> While Ġgenerating Ġclusters , Ġthe Ġsimilarity Ġbetween Ġa Ġsentence Ġand Ġa Ġtopic / key word Ġis Ġcalculated Ġusing Ġthe Ġcos ine Ġsimilarity Ġof Ġembed d ings Ġgenerated Ġusing ĠGoogle 's ĠUSE Ġ( Universal ĠSent ence ĠEnc oder ) ĠB IB REF 1 Ġ. ĠUSE Ġis Ġclaimed Ġto Ġhave Ġbetter Ġperformance Ġof Ġtransfer Ġlearning Ġwhen Ġused Ġwith Ġsentence - level Ġembed d ings Ġas Ġcompared Ġto Ġword - level Ġembed d ings Ġalone . ĠThis Ġmodel Ġis Ġclaimed Ġto Ġhave Ġbetter Ġperformance Ġeven Ġif Ġthere Ġis Ġless Ġtask - specific Ġtraining Ġdata Ġavailable . ĠWe Ġobserved Ġthat Ġthe Ġquality Ġof Ġclusters / section Ġis Ġbetter Ġif Ġthe Ġsimilarity Ġof Ġa Ġsentence Ġwith Ġthe Ġkeyword Ġis Ġconsidered Ġalong Ġwith Ġthe Ġsimilarity Ġof Ġthe Ġsentence Ġwith Ġthe Ġsentences Ġalready Ġavailable Ġin Ġthe Ġrespective Ġsection . </s> To Ġtest Ġour Ġapproach , Ġwe Ġj umble Ġthe Ġordering Ġof Ġsentences Ġin Ġa Ġdocument , Ġprocess Ġthe Ġun ordered Ġdocument Ġand Ġcompare Ġthe Ġsimilarity Ġof Ġthe Ġoutput Ġdocument Ġwith Ġthe Ġoriginal Ġdocument . </s> Related ĠWork </s> Several Ġmodels Ġhave Ġbeen Ġperformed Ġin Ġthe Ġpast Ġto Ġretrieve Ġsentences Ġof Ġa Ġdocument Ġbelonging Ġto Ġa Ġparticular Ġtopic ĠB IB REF 2 Ġ. ĠGiven Ġa Ġtopic , Ġretrieving Ġsentences Ġthat Ġmay Ġbelong Ġto Ġthat Ġtopic Ġshould Ġbe Ġconsidered Ġas Ġa Ġdifferent Ġtask Ġthan Ġwhat Ġwe Ġaim Ġin Ġthis Ġpaper . ĠA Ġgraph Ġbased Ġapproach Ġfor Ġextracting Ġinformation Ġrelevant Ġto Ġa Ġquery Ġis Ġpresented Ġin ĠB IB REF 3 Ġ, Ġwhere Ġsub graph s Ġare Ġbuilt Ġusing Ġthe Ġrelated ness Ġof Ġthe Ġsentences Ġto Ġthe Ġquery . ĠAn Ġincremental Ġintegrated Ġgraph Ġto Ġrepresent Ġthe Ġsentences Ġin Ġa Ġcollection Ġof Ġdocuments Ġis Ġpresented Ġin ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġ. ĠSent ences Ġfrom Ġthe Ġdocuments Ġare Ġmerged Ġinto Ġa Ġmaster Ġsequence Ġto Ġimprove Ġco herence Ġand Ġflow . ĠThe Ġsame Ġordering Ġis Ġused Ġfor Ġsequencing Ġthe Ġsentences Ġin Ġthe Ġextracted Ġsummary . ĠOrder ing Ġof Ġsentences Ġin Ġa Ġdocument Ġis Ġdiscussed Ġin ĠB IB REF 6 Ġ. </s> In Ġthis Ġpaper , Ġwe Ġaim Ġto Ġgenerate Ġthe Ġsections / cl usters Ġfrom Ġan Ġun ordered Ġdocument . ĠTo Ġthe Ġbest Ġof Ġour Ġknowledge , Ġthis Ġis Ġa Ġfirst Ġattempt Ġto Ġaddress Ġthis Ġproblem Ġformally . </s> Prop osed Ġmodel </s> Our Ġmethodology Ġis Ġdescribed Ġin Ġthe ĠFigure Ġ1 Ġ. ĠThe Ġprocess Ġstarts Ġby Ġtaking Ġan Ġun ordered Ġdocument Ġas Ġan Ġinput . ĠThe Ġnext Ġstep Ġis Ġto Ġextract Ġthe Ġkeywords Ġfrom Ġthe Ġinput Ġdocument Ġusing ĠText Rank Ġalgorithm ĠB IB REF 0 Ġand Ġstore Ġthem Ġin Ġa Ġlist Ġ$ K $ Ġ. ĠThe Ġkeywords Ġstored Ġin Ġ$ K $ Ġact Ġas Ġcent ro ids Ġfor Ġthe Ġclusters . ĠNote Ġthat , Ġthe Ġquality Ġof Ġkeywords Ġextracted Ġwill Ġhave Ġa Ġbearing Ġon Ġthe Ġfinal Ġresults . ĠIn Ġthis Ġpaper , Ġwe Ġpresent Ġa Ġmodel Ġthat Ġcan Ġbe Ġused Ġfor Ġstruct uring Ġan Ġun struct ured Ġdocument . ĠIn Ġthe Ġprocess , Ġwe Ġuse Ġa Ġpopular Ġkeyword Ġextraction Ġalgorithm . ĠOur Ġmodel Ġis Ġnot Ġbound Ġto ĠText Rank , Ġand Ġif Ġa Ġbetter Ġkeyword Ġextraction Ġalgorithm Ġis Ġavailable , Ġit Ġcan Ġreplace ĠText Rank . </s> The Ġnext Ġstep Ġis Ġto Ġfind Ġthe Ġmost Ġrelevant Ġsentence Ġfor Ġeach Ġkeyword Ġin Ġ$ K $ Ġ. ĠThe Ġmost Ġrelevant Ġsentence Ġis Ġmapped Ġto Ġthe Ġkeyword Ġand Ġassigned Ġin Ġthe Ġrespective Ġcluster . ĠThis Ġsimilarity Ġbetween Ġthe Ġkeyword Ġand Ġthe Ġsentence Ġis Ġcalculated Ġby Ġthe Ġcos ine Ġsimilarity Ġof Ġembed d ings Ġgenerated Ġfrom ĠGoogle 's ĠUSE . ĠNow , Ġwe Ġhave Ġa Ġlist Ġof Ġkeywords Ġ$ K $ Ġ, Ġand Ġa Ġsentence Ġmapped Ġto Ġeach Ġkeyword . ĠThe Ġnext Ġstep Ġis Ġto Ġmap Ġthe Ġremaining Ġsentences . ĠIn Ġthe Ġnext Ġstep , Ġwe Ġgo Ġthrough Ġall Ġthe Ġsentences Ġin Ġthe Ġdocument Ġthat Ġhave Ġnot Ġbeen Ġmapped Ġyet Ġand Ġfind Ġthe Ġrelevant Ġkeywords Ġthat Ġcan Ġbe Ġmapped Ġwith Ġthem . ĠWe Ġdo Ġthis Ġby Ġthe Ġfollowing Ġprocedure : </s> $ S _ 1 ( x , y ) $ Ġis Ġthe Ġsimilarity Ġbetween Ġthe Ġcurrent Ġsentence Ġ$ x $ Ġand Ġthe Ġkeyword Ġ$ y $ Ġ. Ġ$ S _ 2 ( x , y ) $ Ġis Ġthe Ġmaximum Ġsimilarity Ġbetween Ġthe Ġcurrent Ġsentence Ġ$ x $ Ġand Ġthe Ġsentences Ġthat Ġare Ġalready Ġmapped Ġwith Ġthe Ġkeyword Ġ$ y $ Ġ. ĠIf Ġ$ y $ Ġhas Ġthree Ġsentences Ġmapped Ġto Ġit , Ġthen Ġthe Ġsimilarity Ġbetween Ġ$ x $ Ġand Ġthe Ġsentences Ġmapped Ġto Ġ$ y $ Ġare Ġcomputed , Ġand Ġthe Ġmaximum Ġsimilarity Ġamong Ġthe Ġthree Ġis Ġassigned Ġto Ġ$ S _ 2 ( x , y ) $ Ġ. ĠThe Ġoverall Ġsimilarity Ġ$ x $ 0 Ġis Ġcalculated Ġas : </s> $$ S ( x , y )= Ġt * S _ 1 ( x , y ) Ġ+ Ġ( 1 - t )* S _ 2 ( x , y ) $$ Ġ Ġ Ġ( E q . Ġ1 ) </s> We Ġmap Ġevery Ġother Ġsentence Ġ$ x $ Ġto Ġthe Ġkeyword Ġwith Ġwhich Ġthey Ġhave Ġmaximum Ġsimilarity Ġ$ Max _ y Ġ( S ( x , y )) $ Ġ. ĠLater , Ġwe Ġcluster Ġthe Ġkeywords Ġalong Ġwith Ġthe Ġsentences Ġmapped Ġto Ġthem . ĠThe Ġvalue Ġof Ġ$ t $ Ġsignifies Ġthe Ġimportance Ġto Ġbe Ġgiven Ġto Ġthe Ġkeyword Ġand Ġits Ġassociated Ġsentences Ġrespectively . ĠIn Ġour Ġexperiments , Ġwe Ġempir ically Ġfix Ġthe Ġvalue Ġof Ġ$ t $ Ġto Ġ$ 0 . 25 $ Ġ. </s> Met rics Ġto Ġevaluate Ġour Ġalgorithm </s> To Ġevaluate Ġour Ġalgorithm , Ġwe Ġpropose Ġtwo Ġsimilarity Ġmetrics , Ġ$ Sim 1 $ Ġand Ġ$ Sim 2 $ Ġ. ĠThese Ġmetrics Ġcompute Ġthe Ġsimilarity Ġof Ġeach Ġsection Ġof Ġthe Ġoriginal Ġdocument Ġwith Ġall Ġthe Ġsections / cl usters Ġ( key word Ġand Ġthe Ġsentences Ġmapped Ġto Ġit ) Ġof Ġthe Ġoutput Ġdocument Ġand Ġassign Ġthe Ġmaximum Ġsimilarity . Ġ$ Sim 1 $ Ġbetween Ġan Ġinput Ġsection Ġand Ġan Ġoutput Ġsection Ġis Ġcalculated Ġas Ġthe Ġnumber Ġof Ġsentences Ġof Ġthe Ġinput Ġsection Ġthat Ġare Ġpresent Ġin Ġthe Ġoutput Ġsection Ġdivided Ġby Ġthe Ġtotal Ġnumber Ġof Ġsentences Ġin Ġthe Ġinput Ġsection . ĠTo Ġcalculate Ġthe Ġfinal Ġsimilarity Ġ( similar ity Ġof Ġthe Ġentire Ġoutput Ġdocument ) Ġwe Ġtake Ġthe Ġweighted Ġmean Ġof Ġsimilarity Ġcalculated Ġcorresponding Ġto Ġeach Ġinput Ġsection . Ġ$ Sim 2 $ Ġbetween Ġan Ġinput Ġsection Ġand Ġan Ġoutput Ġsection Ġis Ġcomputed Ġas Ġthe Ġnumber Ġof Ġsentences Ġof Ġan Ġinput Ġsection Ġthat Ġare Ġpresent Ġin Ġan Ġoutput Ġsection Ġdivided Ġby Ġthe Ġsum Ġof Ġsentences Ġin Ġthe Ġinput Ġand Ġoutput Ġsections . ĠThe Ġfinal Ġsimilarity Ġis Ġcomputed Ġin Ġa Ġsimilar Ġmanner . </s> Exper imental Ġsetup Ġand ĠResults </s> For Ġour Ġexperiments , Ġwe Ġprepared Ġfive Ġsets Ġof Ġdocuments . ĠEach Ġset Ġhas Ġ100 Ġwiki Ġdocuments Ġ( random ly Ġchosen ). ĠEach Ġdocument Ġis Ġrestruct ured Ġrandomly Ġ( sent ences Ġare Ġrearr anged Ġrandomly ). ĠThis Ġrestruct ured Ġdocument Ġis Ġthe Ġinput Ġto Ġour Ġmodel Ġand Ġthe Ġoutput Ġdocument Ġis Ġcompared Ġagainst Ġthe Ġoriginal Ġinput Ġdocument . </s> Also , Ġwe Ġcompare Ġour Ġresults Ġwith Ġthe Ġbaseline Ġbeing Ġthe Ġresults Ġwhen Ġwe Ġconsider Ġonly Ġthe Ġsimilarity Ġbetween Ġsentences Ġand Ġkeywords . ĠThe Ġresults Ġare Ġshown Ġin ĠTable Ġ1 Ġ. ĠHere , Ġboth Ġ$ Sim 1 $ Ġand Ġ$ Sim 2 $ Ġare Ġthe Ġmean Ġof Ġsimilarities Ġof Ġthe Ġentire ĠSet . Ġ$ Base Sim 1 $ Ġand Ġ$ Base Sim 1 $ Ġare Ġcomputed Ġsimilar Ġto Ġ$ Sim 1 $ Ġand Ġ$ Sim 2 $ Ġrespectively Ġbut Ġwith Ġthe Ġ$ t $ Ġvalue Ġas Ġ1 Ġin ĠEqu ation Ġ1 Ġ. ĠIt Ġis Ġevident Ġthat Ġthe Ġresults Ġare Ġbetter Ġif Ġboth Ġsimilarities Ġ( Ġ$ S _ 1 ( x , y ) $ Ġand Ġ$ S _ 2 ( x , y ) $ Ġ) Ġare Ġconsidered . </s> Con clusions </s> We Ġproposed Ġan Ġefficient Ġmodel Ġto Ġstructure Ġan Ġun ordered Ġdocument . ĠWe Ġevaluated Ġour Ġmodel Ġagainst Ġthe Ġbaseline Ġand Ġfound Ġthat Ġour Ġproposed Ġmodel Ġhas Ġa Ġsignificant Ġimprovement . ĠWe Ġobserved Ġthat Ġwhile Ġordering Ġan Ġun ordered Ġdocument , Ġthe Ġinitial Ġsentences Ġassociated Ġwith Ġa Ġkeyword / topic Ġplay Ġa Ġsignificant Ġrole .