<jupyter_start><jupyter_text>**TOPIC MODELLING ON NEWS WEBLINKS USING LDA**https://colab.research.google.com/drive/1EJfAZ9zoia4U0wv8C5eOcCXAkx0w_5Lc?usp=sharing*by GROUP 2 (**<NAME>**, <NAME>)*---  **I. INTRODUCTION**Given a list of news weblinks, our group shall be using the unsupervised modelling for Topic Modelling by using the Latent Dirichlet Allocation (LDA) model to determine what are the topics generated by simply having the web address. We will attempt to make sense of such grouping.  **II. DATASET**The dataset is taken from the UCLA website under the News Aggregator machine learning problem. You may click [here](https://archive.ics.uci.edu/ml/datasets/News+Aggregator) to be redirected to the page.Dataset is comprised of two files. The below name and schema are lifted from the website and will shall be elaborated after.**FIRST DATASET: newsCorpora.csv****ID** Numeric ID **TITLE** News title **URL** Url **PUBLISHER** Publisher name **CATEGORY** News category (b = business, t = science and technology, e = entertainment, m = health) **STORY** Alphanumeric ID of the cluster that includes news about the same story **HOSTNAME** Url hostname **TIMESTAMP** Approximate time the news was published, as the number of milliseconds since the epoch 00:00:00 GMT, January 1, 1970 **SECOND DATASET: 2pageSessions.csv****HOSTNAME** Url hostname **STORY** Alphanumeric ID of the cluster that includes news about the same story **HOSTNAME** Url hostname **CATEGORY** News category (b = business, t = science and technology, e = entertainment, m = health) **URL** Two space-delimited urls representing a browsing session For the first dataset, we will look into the TITLE. But as we shall see below, some of the titles will not be useful so we will do data cleaning to sort these out and replace them with their corresponding URL.For the second dataset, we will solely be relying on the URL. As we shall see below, the main issue for the second dataset are the duplicates.After which, we shall consolidate both and perform LDA.  **III. PROBLEM STATEMENT****Primary**: Beyond the current categories provided, can we create a new and descriptive groupings of the articles to determine what types of news are being released on the internet?**Secondary**: Is there apossibility that certainhostname/urls areleaning towards acertain topic?The secondary problem statement was not attempted in this run. Suggest to have look into this after.  **IV. METHODS**This project would be using the LDA (Latent Dirichlet Allocation) model to group these websites into the topic resulted from the mining.  **V. LOADING THE DATA**We have two sets of data:1. First dataset of URLs over 400k+ with the news article title carved out properly. Only issue is that some of the entries in the TITLE column is "Posted by". This means we only have the author but not what the article is about. This data needs to be trasformed. 2. Second dataset of URLs over 10k+ without the news article title. Since we need to do NLP, transformation is needed.Also both datasets have duplicates so we need to remove them.  **A. PACKAGES**<jupyter_code># Data manipulation package used
import pandas as pd

# Data visualization packages used
import matplotlib.pyplot as plt 
%matplotlib inline

## Word Cloud
!pip install wordcloud
from wordcloud import WordCloud
from wordcloud import ImageColorGenerator
from wordcloud import STOPWORDS
import matplotlib.pyplot as plt

## Word Histogram
import seaborn as sns
from nltk import FreqDist

# LDA Packages used

## Processing the data for LDA
import nltk 
#nltk.download('stopwords')
from nltk.corpus import stopwords

!pip install spacy
import spacy

import gensim
from gensim import corpora

## Dashboard for LDA
!pip install pyLDAvis
import pyLDAvis
import pyLDAvis.gensim_models

# Ignore unnecessariy warnings
import warnings
warnings.filterwarnings("ignore")<jupyter_output>Requirement already satisfied: wordcloud in c:\users\joey\anaconda3\lib\site-packages (1.8.1)
Requirement already satisfied: matplotlib in c:\users\joey\anaconda3\lib\site-packages (from wordcloud) (3.4.3)
Requirement already satisfied: numpy>=1.6.1 in c:\users\joey\anaconda3\lib\site-packages (from wordcloud) (1.20.3)
Requirement already satisfied: pillow in c:\users\joey\anaconda3\lib\site-packages (from wordcloud) (8.4.0)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\joey\anaconda3\lib\site-packages (from matplotlib->wordcloud) (1.3.1)
Requirement already satisfied: cycler>=0.10 in c:\users\joey\anaconda3\lib\site-packages (from matplotlib->wordcloud) (0.10.0)
Requirement already satisfied: python-dateutil>=2.7 in c:\users\joey\anaconda3\lib\site-packages (from matplotlib->wordcloud) (2.8.2)
Requirement already satisfied: pyparsing>=2.2.1 in c:\users\joey\anaconda3\lib\site-packages (from matplotlib->wordcloud) (3.0.4)
Requirement already satisfied: six in c:\users\jo[...]<jupyter_text>**B. LOAD DATASET** Load dataset from google drive to colab. Load dataset in colab and define the columns.<jupyter_code>df1_colnames = ['ID','TITLE','URL','PUBLISHER','CATEGORY','STORY','HOSTNAME','TIMESTAMP']
df2_colnames = ['STORY','HOSTNAME','CATEGORY','URL']

df1 = pd.read_csv('newsCorpora.csv', delimiter='\t', header=None, names=df1_colnames)
df2 = pd.read_csv('2pageSessions.csv', delimiter='\t', header=None, names=df2_colnames)<jupyter_output><empty_output><jupyter_text>**VI. DATA CLEANING/PRE-PROCESSING**  **A. DATA CHECKING**Look into the data to determine whether cleaning and transformations are needed to be done. After which, we will consolidate them into one dataset.<jupyter_code>df1.columns
df2.columns<jupyter_output><empty_output><jupyter_text>Looking at the first dataset, we note that the needed columns to create the word cloud is complete. For the sake of data cleaning, we shall correct the TIMESTAMP format.<jupyter_code>df1.head()<jupyter_output><empty_output><jupyter_text>For the second dataset, we would be needing the data under the URL column. The following cleaning and transformation are needed to be done:1. Separate the two URLS. We would need the second URL.2. There are duplicate URLS. These have to be removed and reported as only one entity.<jupyter_code>df2.head()
print(df1.shape)
print(df2.shape)
df1.describe()
df1.value_counts()
df2.describe()
df2.value_counts()
df1.nunique() == 0
df2.nunique() == 0
df1.tail()
df2.tail()<jupyter_output><empty_output><jupyter_text>We note that the PUBLISHER column in the first dataset has 2 null values. It would not matter in our NLP run so we leave it as is.<jupyter_code>df1.info()
df2.info()
df1.notnull().sum()
df2.notnull().sum()
df1.dtypes
df2.dtypes
print(df1.nunique())
print(df2.nunique())
df1
df2<jupyter_output><empty_output><jupyter_text>Let us look at the last character of the URL column. This will help us later on how to extract the last portion of the URL.Given the results, we see that the "/" is not necessarily the last character. Note this for later when we are splitting this column.<jupyter_code>df2['URL'].str.strip().str[-1].unique()<jupyter_output><empty_output><jupyter_text>**B. DATA CLEANING**We shall look clean the first dataset then  move to the second.  **--FIRST DATASET**Remove duplicate title lines and hostname.<jupyter_code>df1_step1 = df1.drop_duplicates(subset = ['TITLE','HOSTNAME'], keep = 'first', ignore_index=True)
df1_step1.info()
df1_step1.head()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 417037 entries, 0 to 417036
Data columns (total 8 columns):
 #   Column     Non-Null Count   Dtype 
---  ------     --------------   ----- 
 0   ID         417037 non-null  int64 
 1   TITLE      417037 non-null  object
 2   URL        417037 non-null  object
 3   PUBLISHER  417035 non-null  object
 4   CATEGORY   417037 non-null  object
 5   STORY      417037 non-null  object
 6   HOSTNAME   417037 non-null  object
 7   TIMESTAMP  417037 non-null  int64 
dtypes: int64(2), object(6)
memory usage: 25.5+ MB<jupyter_text>Now we look at the titles which contains "Posted by". We need to replace these with the URL equivalent.<jupyter_code>df1_step1[df1_step1['TITLE'].str.contains("Posted by")].head()
df1_step2 = df1_step1

df1_step2['URL'] = df1_step2.apply(lambda x : x['URL'].replace(str(x['HOSTNAME']), ''), axis=1) #row-wise replacement based on another column
df1_step2['URL'] = df1_step2.URL.str.replace('http://','')
df1_step2['URL'] = df1_step2.URL.str.replace('https://','')
df1_step2['URL'] = df1_step2.URL.str.replace('.com',' ')
df1_step2['URL'] = df1_step2.URL.str.replace('.html',' ')
df1_step2['URL'] = df1_step2.URL.str.replace('.htm',' ')
df1_step2['URL'] = df1_step2.URL.str.replace('.aspx',' ')
df1_step2['URL'] = df1_step2.URL.str.replace('/',' ')
df1_step2['URL'] = df1_step2.URL.str.replace('-',' ')
df1_step2['URL'] = df1_step2.URL.str.replace('.',' ')

df1_step2.head(5)<jupyter_output><empty_output><jupyter_text>Replace the TITLE with URL for those which contains 'Posted by'.<jupyter_code>df1_step3 = df1_step2

df1_step3['TITLE'][df1_step2['TITLE'].str.contains('Posted by')] = df1_step3['URL'][df1_step2['TITLE'].str.contains('Posted by')]

# check with the sample ID 308 that the title is the same as URL
df1_step3[df1_step3.ID == 308]<jupyter_output><empty_output><jupyter_text>**--SECOND DATASET**Similar to the first dataset, implement the same code but only for the entire table.Create ID column for easy reference to original dataset for later.<jupyter_code>df2_step1 = df2['URL'].str.split(' ', expand=True)
df2_step1['ID'] = df2.index
df2_step1.head()<jupyter_output><empty_output><jupyter_text>Code below ensures that column 2 only has the empty set so we can drop it.<jupyter_code>df2_step1.nunique() == 0<jupyter_output><empty_output><jupyter_text>Code below checks if there is null. The first column is the host name so a null signifies we failed to split the URL column correctly.Also we need to check the second column since we might have URLs there because of the splitting (we would need to correct them if not). Easy way to implement this is to simply get the unique values. We are good if the value is null.<jupyter_code>df2_step1[0].isna().sum()
df2_step1[2].unique()
df2_step2 = df2_step1[1]
df2_step2
df2_step1<jupyter_output><empty_output><jupyter_text>We will use this column for our dataset than the original URL column. Let us drop that column in the original and join this data with that.Also, let us define and ID tag as well.<jupyter_code>df2_step3 = df2.iloc[:,:-1].join(df2_step2)
df2_step3.columns = ['STORY', 'HOSTNAME', 'CATEGORY', 'URL']
df2_step3['ID'] = df2.index
df2_step3.head()<jupyter_output><empty_output><jupyter_text>Before extracting the title from the URL column, notice that we have duplicate lines. Now we remove them. Notice that the count significantly drops from 15,516 rows to 1,074.<jupyter_code>df2_step4 = df2_step3.drop_duplicates(subset = 'URL', keep = 'first', ignore_index=True)
df2_step4.info()
df2_step4.head()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 2804 entries, 0 to 2803
Data columns (total 5 columns):
 #   Column    Non-Null Count  Dtype 
---  ------    --------------  ----- 
 0   STORY     2804 non-null   object
 1   HOSTNAME  2804 non-null   object
 2   CATEGORY  2804 non-null   object
 3   URL       2804 non-null   object
 4   ID        2804 non-null   int64 
dtypes: int64(1), object(4)
memory usage: 109.7+ KB<jupyter_text>Now we can slowly extract the important portions of the URL.<jupyter_code>df2_step5 = df2_step4
df2_step5['URL'] = df2_step4.apply(lambda x : x['URL'].replace(str(x['HOSTNAME']), ''), axis=1)
df2_step5['URL'] = df2_step5['URL'].str.replace('http://','')
df2_step5['URL'] = df2_step5['URL'].str.replace('https://','')
df2_step5['URL'] = df2_step5['URL'].str.replace('.com',' ')
df2_step5['URL'] = df2_step5['URL'].str.replace('.html',' ')
df2_step5['URL'] = df2_step5['URL'].str.replace('.htm',' ')
df2_step5['URL'] = df2_step5['URL'].str.replace('.aspx',' ')
df2_step5['URL'] = df2_step5['URL'].str.replace('/',' ')
df2_step5['URL'] = df2_step5['URL'].str.replace('-',' ')
df2_step5['URL'] = df2_step5['URL'].str.replace('.',' ')

df2_step5.head()<jupyter_output><empty_output><jupyter_text>**C. DATA TRANSFORMATION** We would need to ensure that we get the columns needed for the clustering. In addition, we need to update the ID tag to reflect which dataset they came from. For example, in the second dataset, we need to convert ID number 5 to 5B (and so on). Transform the first dataset.<jupyter_code># Note that df1_step3 has the ID code. We need this later to update our original dataset with the topic clusters.
df1_transform = df1_step3.loc[:,['ID','HOSTNAME','TITLE']]
df1_transform.columns = ['ID','HOSTNAME','NEWSARTICLE']<jupyter_output><empty_output><jupyter_text>Let us look at how many news articles that each hostname has done.<jupyter_code>df1_transform['HOSTNAME'].value_counts()#.plot(kind='pie')
# update the ID
df1_transform['ID'] = df1_transform['ID'].astype(str)
df1_transform['ID'] = df1_transform['ID'] + 'A'
df1_transform<jupyter_output><empty_output><jupyter_text>Transform the second dataset.<jupyter_code>df2_transform = df2_step5.loc[:,['ID','HOSTNAME','URL']]
df2_transform.columns = ['ID','HOSTNAME','NEWSARTICLE']
df2_transform['HOSTNAME'].value_counts()
# update the ID
df2_transform['ID'] = df2_transform['ID'].astype(str)
df2_transform['ID'] = df2_transform['ID'] + 'A'
df2_transform<jupyter_output><empty_output><jupyter_text>**D. CONSOLIDATE DATASET**We only consolidated now because it helps to create word clouds individually per dataset. It allowed us to check and see extraneous words which should have been removed in the data transformation portion.Now, combine the two datasets and create the consolidated word cloud.<jupyter_code>df = df1_transform.append(df2_transform)
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 419841 entries, 0 to 2803
Data columns (total 3 columns):
 #   Column       Non-Null Count   Dtype 
---  ------       --------------   ----- 
 0   ID           419841 non-null  object
 1   HOSTNAME     419841 non-null  object
 2   NEWSARTICLE  419841 non-null  object
dtypes: object(3)
memory usage: 12.8+ MB<jupyter_text>The cleaning below employs the code learned from class. This might have been done above (esp the stopwords removal for later). The code will be enhanced soon.<jupyter_code># removing everything except alphabets`
df.NEWSARTICLE = df.NEWSARTICLE.str.replace("[^a-zA-Z#]", " ")
df.NEWSARTICLE.shape
#removing short words/concatenation
df.NEWSARTICLE=df.NEWSARTICLE.apply(lambda x: ' '.join([w for w in x.split() if len(w)>2]))
df.NEWSARTICLE.shape
df.NEWSARTICLE = df.NEWSARTICLE.apply(lambda x: x.lower())
df.NEWSARTICLE.shape

#drop duplicates
#df.NEWSARTICLE = df.NEWSARTICLE.drop_duplicates()
#print(df.NEWSARTICLE.shape)

#remove null fields
df.NEWSARTICLE = df.NEWSARTICLE[df.NEWSARTICLE.notnull()]
df.NEWSARTICLE.shape
#tokenization
token = df['NEWSARTICLE'].apply(lambda x: x.split())<jupyter_output><empty_output><jupyter_text>**E. PREPROCESSING FOR EDA**<jupyter_code>stop_words = stopwords.words('english')

#remove stopwords
token = token.apply(lambda x: [item for item in x if item not in stop_words])
token.head(3)
len(df)
len(token)
token = token.reset_index()
token
token.ind = token.index
token = token.drop(columns='index')
token = token.sort_index()
token.head(3)
type(token)
token['NEWSARTICLE'] = [' '.join(map(str,l)) for l in token['NEWSARTICLE']]
cleaned = token['NEWSARTICLE']
#df.NEWSARTICLE
cleaned<jupyter_output><empty_output><jupyter_text>**VI. EXPLORATORY DATA ANALYSIS**<jupyter_code>#TITLE = " ".join(i for i in df.NEWSARTICLE)
TITLE = " ".join(i for i in cleaned)
stopwords = set(STOPWORDS)
wordcloud = WordCloud(stopwords=stopwords, background_color="white", max_words=5000, contour_width=3, contour_color='steelblue').generate(TITLE)
plt.figure(figsize=(15,10))
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis("off")
plt.show()
def freq_words(x, terms = 10):
  all_words = ' '.join([text for text in x])
  all_words = all_words.split()
  
  fdist = FreqDist(all_words)
  words_df = pd.DataFrame({'word':list(fdist.keys()), 'count':list(fdist.values())})
  
  # selecting top 20 most frequent words
  d = words_df.nlargest(columns='count', n = terms) 
  plt.figure(figsize=(20,10))
  ax = sns.barplot(data=d, x= 'count', y = 'word', palette = 'GnBu_d')
  ax.set(xlabel = 'COUNT',ylabel = 'WORD')
  plt.rcParams.update({'font.size': 15})
  plt.show()
freq_words(cleaned)<jupyter_output><empty_output><jupyter_text>**VII. PROCESSING FOR LDA**  **A. LEMMATIZATION**<jupyter_code>nlp = spacy.load('en_core_web_md', disable=['parser', 'ner'])

def lemmatization(texts,allowed_postags=['NOUN', 'ADJ']): 
       output = []
       for sent in texts:
             doc = nlp(sent) 
             output.append([token.lemma_ for token in doc if token.pos_ in allowed_postags ])
       return output
text_list=cleaned.tolist()
print(text_list[1])
tokenized_newspages = lemmatization(text_list)
print(tokenized_newspages[1])<jupyter_output>fed charles plosser sees high bar change pace tapering
['high', 'bar', 'change', 'pace']<jupyter_text>**B. MODELLING**<jupyter_code>dictionary = corpora.Dictionary(tokenized_newspages)
doc_term_matrix = [dictionary.doc2bow(rev) for rev in tokenized_newspages]
# Creating the object for LDA model using gensim library
LDA = gensim.models.ldamodel.LdaModel

# Build LDA model
lda_model = LDA(corpus=doc_term_matrix, id2word=dictionary, num_topics=6, random_state=100,
                chunksize=1000, passes=50,iterations=100)<jupyter_output><empty_output><jupyter_text>**C. RESULTING TOPICS**<jupyter_code>lda_model.print_topics()<jupyter_output><empty_output><jupyter_text>**D. LDA DASHBOARD**<jupyter_code># Visualize the topics
#https://github.com/bmabey/pyLDAvis
#https://speakerdeck.com/bmabey/visualizing-topic-models
pyLDAvis.enable_notebook()
vis = pyLDAvis.gensim_models.prepare(lda_model, doc_term_matrix, dictionary)
vis<jupyter_output><empty_output><jupyter_text>**E. COHERENCE SCORE**<jupyter_code>print('\nPerplexity: ', lda_model.log_perplexity(doc_term_matrix,total_docs=10000))  # a measure of how good the model is. lower the better.

# Compute Coherence Score
from gensim.models.coherencemodel import CoherenceModel
coherence_model_lda = CoherenceModel(model=lda_model, texts=tokenized_newspages, dictionary=dictionary , coherence='c_v')
coherence_lda = coherence_model_lda.get_coherence()
print('\nCoherence Score: ', coherence_lda)<jupyter_output>Perplexity:  -11.831561347426375

Coherence Score:  0.32993635516787445<jupyter_text>Coherence score for multiple topics<jupyter_code>def compute_coherence_values(dictionary, corpus, texts, limit, start=2, step=3):
    """
    Compute c_v coherence for various number of topics

    Parameters:
    ----------
    dictionary : Gensim dictionary
    corpus : Gensim corpus
    texts : List of input texts
    limit : Max num of topics

    Returns:
    -------
    model_list : List of LDA topic models
    coherence_values : Coherence values corresponding to the LDA model with respective number of topics
    """
    coherence_values = []
    model_list = []
    for num_topics in range(start, limit, step):
        model = gensim.models.ldamodel.LdaModel(corpus=corpus, num_topics=num_topics, id2word=dictionary)
        model_list.append(model)
        coherencemodel = CoherenceModel(model=model, texts=texts, dictionary=dictionary, coherence='c_v')
        coherence_values.append(coherencemodel.get_coherence())

    return model_list, coherence_values
model_list, coherence_values = compute_coherence_values(dictionary=dictionary, corpus=doc_term_matrix, texts=tokenized_newspages, start=2, limit=10, step=1)
# Show graph
limit=50; start=2; step=1;
x = range(start, limit, step)
plt.plot(x, coherence_values)
plt.xlabel("Number of Topics")
plt.ylabel("Coherence score")
plt.legend(("coherence_values"), loc='best')
plt.show()
for m, cv in zip(x, coherence_values):
    print("Num Topics =", m, " has Coherence Value of", round(cv, 4))<jupyter_output>Num Topics = 2  has Coherence Value of 0.3546
Num Topics = 3  has Coherence Value of 0.3874
Num Topics = 4  has Coherence Value of 0.382
Num Topics = 5  has Coherence Value of 0.3551
Num Topics = 6  has Coherence Value of 0.3653
Num Topics = 7  has Coherence Value of 0.3469
Num Topics = 8  has Coherence Value of 0.3697
Num Topics = 9  has Coherence Value of 0.3516
Num Topics = 10  has Coherence Value of 0.3903
Num Topics = 11  has Coherence Value of 0.3976
Num Topics = 12  has Coherence Value of 0.3962
Num Topics = 13  has Coherence Value of 0.4017
Num Topics = 14  has Coherence Value of 0.4075
Num Topics = 15  has Coherence Value of 0.3969
Num Topics = 16  has Coherence Value of 0.3841
Num Topics = 17  has Coherence Value of 0.4053
Num Topics = 18  has Coherence Value of 0.4223
Num Topics = 19  has Coherence Value of 0.4213
Num Topics = 20  has Coherence Value of 0.4374
Num Topics = 21  has Coherence Value of 0.454
Num Topics = 22  has Coherence Value of 0.4599
Num Topics = 23  has Co[...]<jupyter_text>**VIII. INSIGHTS**1. Topics formed:**Cluster 1** Finance and Marketing **Cluster 2** Global Trends **Cluster 3** Crime, Disease & Natural Disaster **Cluster 4** Music, TV and Entertainment **Cluster 5** Movies, TV, and Entertainment **Cluster 6** Recent Developments 2. Cluster 4 and 6 overlaps which means their topic seem to have similarities. We can consider updating our stopwords to remove those words causing this overlap OR identify how they can be differentiated. Other clusters are easily distinguishable and identifiable. Upon putting New and News as stopwords and looking at five topics, we have:![image-7.png](attachment:image-7.png)3. Old categories: business, science and technology, entertainment, health. New categories seem to be more descriptive. Also entertainment seem to have a large scope worth to break down into more than one category. 4. LDA model does not account the ‘sense’ or the correlation of each w for each d. In our case, the model seem fit since scraping internet URLs would not necessarily form sensible phrases.  **VI. REFERENCES**Data source: https://archive.ics.uci.edu/ml/datasets/News+Aggregator<NAME>. and <NAME>. (2019). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science. <NAME>., <NAME>. and <NAME>. (2009). Natural Language Processing with Python. O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.https://www.machinelearningplus.com/nlp/topic-modeling-gensim-python/14computemodelperplexityandcoherencescore<jupyter_code># for saving to HTML (this was previously done in google colab)
#%%shell
#jupyter nbconvert --to html ///content/GROUP_2_Project_NLP_on_News_Sites.ipynb<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11648111765989959, 'fraction_numerical': 0.03291857672997162, 'mean_word_length': 5.246523043359694, 'pattern_counts': {'":': 0, '<': 124, '<?xml version=': 0, '>': 134, 'https://': 8, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '30', 'dir': 'jupyter-structured-clean-dedup', 'id': '618425', 'license': 'Unlicense', 'n_tokens_mistral': 8065, 'n_tokens_neox': 7531, 'n_words': 2313, 'path': 'Machine Learning/LDA Topic Modelling on News Pages_FINAL.ipynb', 'repo_name': 'joeytuason83/joeytuason.github.io', 'size': '22922'}