<jupyter_start><jupyter_text>Text summarization  Importing dependency<jupyter_code>from gensim.summarization import summarize<jupyter_output><empty_output><jupyter_text>importing data<jupyter_code>textData = """Just A Rather Very Intelligent System a.k.a JARVIS is created by Tony Stark natural-language and a sophisticated artificial intelligence user interface computer system, named after <NAME>, the butler who worked for Howard Stark. Though its primary duty is to automate Stark’s Malibu estate, the lifelike program fulfills many other needs for Stark, like being an information source for him, a diagnostic tool, a consultant and a voice of reason in Stark’s life. It was also responsible to provide security for Tony Stark's Mansion and Stark Tower. After creating the Mark II armor, Stark uploaded JARVIS into all of the Iron Man Armors, as well as allowing him to interact with the other Avengers, giving them valuable information during combat. JARVIS may be the one intellect Stark feels most comfortable opening up to.  JARVIS can object to Stark’s commands if necessary. JARVIS speaks with a refined British accent, and is capable of back talk, sarcasm and condescension. During the Ultron Offensive, JARVIS was destroyed by Ultron, although his remaining programming codes unknowingly continued to thwart Ultron's plans of gaining access to nuclear missiles. His remains were found by Stark, who uploaded them into a synthetic body made of vibranium and, in conjunction with Ultron's personality and an Infinity Stone. JARVIS' duties were then taken over by FRIDAY."""<jupyter_output><empty_output><jupyter_text>length of the data<jupyter_code>len(textData)<jupyter_output><empty_output><jupyter_text>length of the summarized data<jupyter_code>len(summarize(textData))<jupyter_output><empty_output><jupyter_text>function to implement the percentage of summarization to be created from the actual data<jupyter_code>def summarization(x):
        alpha = summarize(textData,ratio=x).split('\n')
        alpha = ''.join(alpha)
        return alpha


summarization(0.2)
summarization(0.5)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.055894786284640674, 'fraction_numerical': 0.0018788163457022077, 'mean_word_length': 6.02970297029703, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 22, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '208086', 'license': 'MIT', 'n_tokens_mistral': 592, 'n_tokens_neox': 556, 'n_words': 253, 'path': 'Text summarization.ipynb', 'repo_name': 'Paulsanjo/NLP-projects', 'size': '2135'}