<jupyter_start><jupyter_text>NLP Task  0. Install Requirements (uncomment as required)<jupyter_code>import sys
#!{sys.executable} -m pip install numpy
#!{sys.executable} -m pip install youtube_transcript_api
#!{sys.executable} -m pip install deepsegment
#!{sys.executable} -m pip install fastpunct<jupyter_output><empty_output><jupyter_text>1. Get YouTube video Transcript<jupyter_code>a = input("Enter YouTube Video ID: ")
if a == "":
    # default value if no input is given by the user
    video_id = "eIho2S0ZahI"
else:
    video_id = a
import youtube_transcript_api
from youtube_transcript_api import YouTubeTranscriptApi<jupyter_output><empty_output><jupyter_text>2. Aggregate YouTube Transcript<jupyter_code>try:
    unrefined_transcript = YouTubeTranscriptApi.get_transcript(video_id)
    refined_transcript = ""

    for line in unrefined_transcript:
        refined_transcript += line['text'] + " "
        
except:
    print("YouTube Transctipt API Exception. (The video most likeyly doesn't have subtitles enabled).")<jupyter_output><empty_output><jupyter_text>3. a) Segment Transcript to fit within the maximum sentence length (512 characters)<jupyter_code>from deepsegment import DeepSegment

segmenter = DeepSegment('en')
list_of_sentences = segmenter.segment_long(refined_transcript)<jupyter_output>Using TensorFlow backend.
/home/akash/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/akash/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/akash/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/akash/anaconda3/e[...]<jupyter_text>3. b) Add Punctuations using FastPunct<jupyter_code>import numpy
from fastpunct import FastPunct

fastpunct = FastPunct()
list_of_senntences = fastpunct.punct(list_of_sentences)<jupyter_output>/home/akash/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 9010). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at  /pytorch/c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
I0514 01:36:06.251214 139858425243456 fastpunct.py:55] Warming up<jupyter_text>Final Transcript:<jupyter_code>final_transcript = " ".join(list_of_sentences)
print(final_transcript)<jupyter_output>The human voice: It's the instrument we all play. It's the most powerful sound in the world, probably. It's the only one that can start a war or say "I love you." And yet many people have the experience that when they speak, people don't listen to them. And why is that? How can we speak powerfully to make change in the world? What I'd like to suggest, there are a number of habits that we need to move away from. I've assembled for your pleasure here seven deadly sins of speaking. I'm not pretending this is an exhaustive list, but these seven, I think, are pretty large habits that we can all fall into. First, gossip. Speaking ill of somebody who's not present. Not a nice habit, and we know perfectly well the person gossiping, five minutes later, will be gossiping about us. Second, judging. We know people who are like this in conversation, and it's very hard to listen to somebody if you know that you're being judged and found wanting at the same time. Third, negativity. You can fall into [...]
{'fraction_non_alphanumeric': 0.09669151910531221, 'fraction_numerical': 0.024231127679403542, 'mean_word_length': 5.2398255813953485, 'pattern_counts': {'":': 1, '<': 22, '<?xml version=': 0, '>': 23, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '255104', 'license': 'MIT', 'n_tokens_mistral': 1459, 'n_tokens_neox': 1361, 'n_words': 466, 'path': 'YouTube_Transcript_Punctuation.ipynb', 'repo_name': 'grassknoted/NLP-Tasks', 'size': '4292'}