<jupyter_start><jupyter_text>**LexRank Summarizer**<jupyter_code>import sumy
from sumy.parsers.plaintext import PlaintextParser
from sumy.nlp.tokenizers import Tokenizer
from sumy.summarizers.lex_rank import LexRankSummarizer

def lex_rank_summarizer(string, n=1):
    """
    For text summarization; this returns the top n most relevant sentences from a string of sentences. It uses LexRank, which is
    an algorithm that uses word2vec to compute the pairwise similarities between each of the sentences and then ranks them.
    """
    parser = PlaintextParser.from_string(string,Tokenizer("english"))
    summarizer = LexRankSummarizer()
    summary = summarizer(parser.document, n)   
    sentence_list = [str(summary[i]) for i in range(n)]
    
    return sentence_list


#Example of lex_rank_summarizer() usage: the 3 most relevant sentences.
article_title = df.loc[7317, 'article_title']
print('Wikipedia article title: ' + article_title)

doc = wiki[7317] #The Wikipedia article with row index of 7317.
n = 3
top_n_sentences = lex_rank_summarizer(doc, n)
print('Top ' + str(n) + ' sentences: ' + '\n')
for i in range(n):
    print(str(i+1) + ') ' + top_n_sentences[i] + '\n')<jupyter_output>Wikipedia article title: Random close pack
Top 3 sentences: 

1) Random close pack	https://en.wikipedia.org/wiki/Random_close_pack	"Random close packing(RCP) is an empirical parameter used to characterize the maximum volume fraction of solid objects obtained when they are packed randomly.

2) But shaking cannot increase the density indefinitely a limit is reached and if this is reached without obvious packing into a regular crystal lattice this is the empirical random close-packed density.

3) The definition of packing fraction can be given as: ""the volume taken by number of particles in a given space of volume"".<jupyter_text>**Luhn Summarizer**<jupyter_code>import sumy
from sumy.parsers.plaintext import PlaintextParser
from sumy.nlp.tokenizers import Tokenizer
from sumy.summarizers.luhn import LuhnSummarizer

def luhn_summarizer(string, n=1):
    """
    For text summarization; this returns the top n most relevant sentences from a string of sentences. 
    It uses Luhn, a heuristic method.
    """    
    parser = PlaintextParser.from_string(string,Tokenizer("english"))
    summarizer_luhn = LuhnSummarizer()
    summary = summarizer_luhn(parser.document,n)
    sentence_list = [str(summary[i]) for i in range(n)]
    
    return sentence_list


#Example of luhn_summarizer() usage: the 3 most relevant sentences.
article_title = df.loc[7317, 'article_title']
print('Wikipedia article title: ' + article_title)

doc = wiki[7317] #The Wikipedia article with row index of 7317.
n = 3
top_n_sentences = luhn_summarizer(doc, n)
print('Top ' + str(n) + ' sentences: ' + '\n')
for i in range(n):
    print(str(i+1) + ') ' + top_n_sentences[i] + '\n')<jupyter_output>Wikipedia article title: Random close pack
Top 3 sentences: 

1) For example when a solid container is filled with grain shaking the container will reduce the volume taken up by the objects thus allowing more grain to be added to the container.

2) But shaking cannot increase the density indefinitely a limit is reached and if this is reached without obvious packing into a regular crystal lattice this is the empirical random close-packed density.

3) Thus RCP is the packing fraction given by the limit as the tapping amplitude goes to zero and the limit as the number of taps goes to infinity.<jupyter_text>**Latent Semantic Analysis Summarizer**<jupyter_code>import sumy
from sumy.parsers.plaintext import PlaintextParser
from sumy.nlp.tokenizers import Tokenizer
from sumy.summarizers.lsa import LsaSummarizer

def lsa_summarizer(string, n=1):
    """
    For text summarization; this returns the top n most relevant sentences from a string of sentences. 
    It uses Latent Semantic Analysis.
    """    
    parser = PlaintextParser.from_string(string,Tokenizer("english"))
    summarizer_lsa = LsaSummarizer()
    summary = summarizer_lsa(parser.document,n)
    sentence_list = [str(summary[i]) for i in range(n)]
    
    return sentence_list

#Example of lsa_summarizer() usage: the 3 most relevant sentences.
article_title = df.loc[7317, 'article_title']
print('Wikipedia article title: ' + article_title)

doc = wiki[2019] #The Wikipedia article with row index of 7317.
n = 3
top_n_sentences = lsa_summarizer(doc, n)
print('Top ' + str(n) + ' sentences: ' + '\n')
for i in range(n):
    print(str(i+1) + ') ' + top_n_sentences[i] + '\n')<jupyter_output>Wikipedia article title: Random close pack
Top 3 sentences: 

1) While a conventional CCA generalizes principal component analysis(PCA) to two sets of random variables a gCCA generalizes PCA to more than two sets of random variables.

2) They can always be made to vanish by introducing a new regression parameter for each common factor.

3) <NAME>.; <NAME>.(2012) ""Enhancing reproducibility of fMRI statistical maps using generalized canonical correlation analysis in NPAIRS framework"" NeuroImage 60(4): 1970–1981.<jupyter_text>**TextRank**<jupyter_code>#gensim is for topic modeling and similarity retrieval.
from gensim.summarization.summarizer import summarize
from gensim.summarization import keywords 

#Used for retrieving text from a webpage.
from bs4 import BeautifulSoup
from urllib.request import urlopen
 
def text_rank_summarizer(url, n=10):
    """ 
    This returns the n most relevant terms from a webpage, unless there are less than n, in which case all are returned.
    The terms are in descending order of relevance. Note that a group of words with a common root can be returned but only 
    count as 1, so the returned list can actually be longer than n.
    """
    page = urlopen(url)
    soup = BeautifulSoup(page, "lxml")
    text = ' '.join(map(lambda p: p.text, soup.find_all('p')))
    
    #We need to count the number of terms so that we can use it in the min() function. Otherwise, if we input a value of
    #n which is too large for the given URL then an error will be returned.
    number_terms = len(keywords(str(text)).split('\n'))
    
    #Retrieving the relevant terms, saved as a list.
    relevant_terms = keywords(str(text), words=min(n, number_terms)).split('\n')
    
    return relevant_terms


#Example of relevant_terms() usage. Note that the terms are lists in descending order of relevance.
text_rank_summarizer("https://en.wikipedia.org/wiki/Calculus", n=10)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08279370952821462, 'fraction_numerical': 0.012642614862781376, 'mean_word_length': 4.573024054982818, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 16, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '230933', 'license': 'MIT', 'n_tokens_mistral': 2014, 'n_tokens_neox': 1899, 'n_words': 748, 'path': 'Machine Learning Project Final Version.ipynb', 'repo_name': 'nateofspades/Wikipedia-Analysis', 'size': '6503'}