<jupyter_start><jupyter_text>Topic modelling on patent main claims> Given a large volume of texts, topic modelling is an efficient technique to detect underlying themes within text documents. Throughout this notebook, we are going to perform topic modelling by means of the algorithm called Latent Dirichlet Allocation (LDA). The level of our analysis is the 1' main claim of a patent publication.  - toc: true - badges: true- hide_binder_badge: true- comments: true- categories: [NLP, Text Cleaning, Text Tokenization, Corpus Creation, Topic modelling, LDA optmiation, Topic Visualization, Topic Inference, Intellectual property, Patent data]  Import of packages<jupyter_code>import pandas as pd 
from io import StringIO 
import matplotlib.pyplot as plt 
import numpy as np
import seaborn as sns
%matplotlib inline 
sns.set(color_codes=True)<jupyter_output><empty_output><jupyter_text>Load the dataThe data are retrieved from PatBase. I select publications with a specific legal status, e.g., granted publications, and from a unique jurisdiction, such as the USA's one.<jupyter_code>df = pd.read_csv('/content/drive/MyDrive/Colab Notebooks/DSBA Patent Project /Paper /EDA & NLP /patbase_export_274777127.csv')
col = ['Cooperative Patent Class', '1st Main Claim'] 
df = df[col]
df = df.dropna()
df['cpc'] = df['Cooperative Patent Class'].str.extract(r'(^.{0,1})')<jupyter_output><empty_output><jupyter_text>Text cleaning<jupyter_code>df['main_claim'] = df['1st Main Claim'].str.replace('\[EN\]\s1.\s', '', regex = True) # string stripping EN
df['main_claim'] = df['main_claim'].str.lower()<jupyter_output><empty_output><jupyter_text>TokenizationTokenization is a way of separating a piece of text into smaller units called tokens. Here, tokens are words.  Modules to tokenize<jupyter_code># Import WordNetLemmatizer
import nltk
nltk.download('stopwords')
nltk.download('wordnet')
from nltk.stem import WordNetLemmatizer
from nltk.corpus import stopwords
english_stops = set(stopwords.words('english'))
# make sure the index is as long as the dataframe
df.index = range(len(df))<jupyter_output><empty_output><jupyter_text>Using spaCy to tokenizespaCy is a free open-source library for Natural Language Processing in Python.<jupyter_code>import spacy
nlp = spacy.load("en")
# create a empty list
tokens = []

# iterate SpaCy preprocessing throughout all the main claims
for claim in nlp.pipe(df['main_claim']):
  proj_tok = [token.lemma_.lower() for token in claim if token.pos_ in ['NOUN', 'PROPN', 'ADJ', 'ADV'] and not token.is_stop]
  tokens.append(proj_tok)

# it takes 11 minutes   
df['tokens'] = tokens<jupyter_output><empty_output><jupyter_text>Create a corpus with Gensim Gensim is a Python library for topic modelling, document indexing and similarity retrieval with large corpora. Target audience is the natural language processing (NLP) and information retrieval (IR) community.<jupyter_code># Import the dictionary builder
from gensim.corpora.dictionary import Dictionary
# Create a Dictionary from the articles: dictionary
dictionary = Dictionary(df['tokens'])
# filter out low-frequency / high-frequency stuff, NOT also limit the vocabulary to max 1000 words
dictionary.filter_extremes(no_below=5, no_above=0.5)
# construct corpus using this dictionary
corpus = [dictionary.doc2bow(doc) for doc in df['tokens']]
# Human readable format of corpus (term-frequency)
# Create Dictionary
import gensim.corpora as corpora

id2word = corpora.Dictionary(df['tokens'])

[[(id2word[id], freq) for id, freq in cp] for cp in corpus[:1]]<jupyter_output><empty_output><jupyter_text>LDA LDA is a probabilistic topic modelling method that identifies abstract themes from a set of text documents. LDA makes two assumptions: 1. it assumes that documents with similar topics will use similar groups of words. 2. it assumes that each document is a mix of multiple topics. Moreover, each topic is described by a set of words.<jupyter_code># we'll use the faster multicore version of LDA

from gensim.models import LdaMulticore<jupyter_output><empty_output><jupyter_text>Finding the optimal number of topicThe approach to finding the optimal number of topics is to build many LDA models with different values of number of topics (k) and pick the one that gives the highest coherence value. The optimal number of 'k' is the one that appears first after a fast growth in the coherence value. A causality of 'k' being too large is that you see the same words repeated across multiple topics.<jupyter_code>import gensim
from gensim.models import CoherenceModel
def compute_coherence_values(dictionary, corpus, texts, limit, start=2, step=3):
    """
    Compute c_v coherence for various number of topics

    Parameters:
    ----------
    dictionary : Gensim dictionary
    corpus : Gensim corpus
    texts : List of input texts
    limit : Max num of topics

    Returns:
    -------
    model_list : List of LDA topic models
    coherence_values : Coherence values corresponding to the LDA model with respective number of topics
    """
    coherence_values = []
    model_list = []
    for num_topics in range(start, limit, step):
        model = LdaMulticore(corpus=corpus, num_topics=num_topics, id2word=id2word)
        model_list.append(model)
        coherencemodel = CoherenceModel(model=model, texts=texts, dictionary=dictionary, coherence='c_v')
        coherence_values.append(coherencemodel.get_coherence())

    return model_list, coherence_values
model_list, coherence_values = compute_coherence_values(dictionary=id2word, corpus=corpus, texts= df['tokens'], start=2, limit=40, step=6)
# Show graph
limit=40; start=2; step=6;
x = range(start, limit, step)
plt.plot(x, coherence_values)
plt.xlabel("Num Topics")
plt.ylabel("Coherence score")
plt.legend(("coherence_values"), loc='best')
plt.show()<jupyter_output><empty_output><jupyter_text>Coherence Scores<jupyter_code># Print the coherence scores
for m, cv in zip(x, coherence_values):
    print("Num Topics =", m, " has Coherence Value of", round(cv, 4))
print(model_list[2])
from pprint import pprint<jupyter_output><empty_output><jupyter_text>Visualization<jupyter_code># let's fist install this nice visualizer
!pip install -qq pyLDAvis
!pip install -qq pyLDAvis
import pyLDAvis
# and import it
import pyLDAvis.gensim_models
%matplotlib inline
pyLDAvis.enable_notebook()
# Training the model
lda_model = LdaMulticore(corpus, id2word=dictionary, num_topics=14)
# Let's try to visualize
lda_display = pyLDAvis.gensim_models.prepare(lda_model, corpus, dictionary)
# Let's Visualize
pyLDAvis.display(lda_display)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08971056220639491, 'fraction_numerical': 0.007273829368086074, 'mean_word_length': 5.633165829145729, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 42, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '615865', 'license': 'Apache-2.0', 'n_tokens_mistral': 2109, 'n_tokens_neox': 1967, 'n_words': 698, 'path': '_notebooks/2022_02_18_TopicModel.ipynb', 'repo_name': 'TheMuahDibPortfolio/test', 'size': '6599'}