<jupyter_start><jupyter_text>Turbo-charge your spaCy NLP pipeline> Tips and tricks to significantly speed up text processing using multi-core spaCy custom pipelines.Consider you have a large tabular dataset on which you want to apply some non-trivial NLP transformations, such as stopword removal followed by lemmatizing (i.e. reducing to root form) the words in a text. [spaCy](https://spacy.io/usage) is an industrial strength NLP library designed for just such a task.In the example shown below, the [New York Times Kaggle dataset](https://www.kaggle.com/nzalake52/new-york-times-articles) is used to showcase how to significantly speed up a spaCy NLP pipeline. the goal is to take in an article's text, and speedily return a list of lemmas with unnecessary words, called "stopwords", removed.Pandas DataFrames provide a convenient interface to work with tabular data of this nature. First, import the necessary modules shown.<jupyter_code>import re
import pandas as pd
import spacy
from tqdm.notebook import tqdm
tqdm.pandas()<jupyter_output><empty_output><jupyter_text>Input variablesThe tabular data is stored in a tab-separated file obtained by running the preprocessing notebook `preprocessing.ipynb` on the raw text data from Kaggle and stored in the `data/` directory. A curated stopword file is also provided in this location.Additionally, during initial testing, we can limit the size of the DataFrame being worked on (to around $2000$ samples) for faster execution. For the final run, disable the limit by setting it to zero.<jupyter_code>inputfile = "data/nytimes.tsv"
stopwordfile = "data/stopwords/stopwords.txt"
limit = 0<jupyter_output><empty_output><jupyter_text>Load spaCy modelFor lemmatization, a simple spaCy model can be initialized. Since we will not be doing any specialized tasks such as dependency parsing and named entity recognition in this exercise, these components are disabled.spaCy has a `sentencizer` component that can instead be enabled - this simply performs tokenization and sentence boundary detection, following which lemmas can be extracted as token properties.<jupyter_code>nlp = spacy.load('en_core_web_sm', disable=['tagger', 'parser', 'ner'])
nlp.add_pipe(nlp.create_pipe('sentencizer'))<jupyter_output><empty_output><jupyter_text>A method is defined to read in stopwords from a text file and convert it to a set in Python (for efficient lookup).<jupyter_code>def get_stopwords():
    "Return a set of stopwords read in from a file."
    with open(stopwordfile) as f:
        stopwords = []
        for line in f:
            stopwords.append(line.strip("\n"))
    # Convert to set for performance
    stopwords_set = set(stopwords)
    return stopwords_set

stopwords = get_stopwords()<jupyter_output><empty_output><jupyter_text>Read in New York Times DatasetThe pre-processed version of the NYT news dataset is read in as a Pandas DataFrame. The columns are named `date`, `headline` and `content` - the text present in the content column is what will be preprocessed to remove stopwords and generate token lemmas.<jupyter_code>def read_data(inputfile):
    "Read in a tab-separated file with date, headline and news content"
    df = pd.read_csv(inputfile, sep='\t', header=None,
                     names=['date', 'headline', 'content'])
    df['date'] = pd.to_datetime(df['date'], format="%Y-%m-%d")
    return df
df = read_data(inputfile)
df.head()<jupyter_output><empty_output><jupyter_text>Define text cleanerSince the news article data comes from a raw HTML dump, it is very messy and contains a host of unnecessary symbols, social media handles, URLs and other artifacts. An easy way to clean it up is to use a regex that parses only alphanumeric strings and hyphens (so as to include hyphenated words) that are between a given length (3 and 50). This filters each document down to only meaningful text for the lemmatizer.<jupyter_code>def cleaner(df):
    "Extract relevant text from DataFrame using a regex"
    # Regex pattern for only alphanumeric, hyphenated text with 3 or more chars
    pattern = re.compile(r"[A-Za-z0-9\-]{3,50}")
    df['clean'] = df['content'].str.findall(pattern).str.join(' ')
    if limit > 0:
        return df.iloc[:limit, :].copy()
    else:
        return df
df_preproc = cleaner(df)
df_preproc.head(3)<jupyter_output><empty_output><jupyter_text>Now that we have just the clean, alphanumeric tokens left over, these can be further cleaned up by removing stopwords before proceeding to lemmatization.  Option 1. Work directly on the data using `pandas.Series.apply`The straightforward way to process this text is to use an existing method, in this case the `lemmatize` method shown below, and apply it to the `clean` column of the DataFrame. Lemmatization is done using the spaCy's underlying [`Doc` representation](https://spacy.io/usage/spacy-101annotations) of each token, which contains a `lemma_` property. Stopwords are removed simultaneously with the lemmatization process, as each of these steps involves iterating through the same list of tokens.<jupyter_code>def lemmatize(text):
    """Perform lemmatization and stopword removal in the clean text
       Returns a list of lemmas
    """
    doc = nlp(text)
    lemma_list = [str(tok.lemma_).lower() for tok in doc
                  if tok.is_alpha and tok.text.lower() not in stopwords]
    return lemma_list<jupyter_output><empty_output><jupyter_text>The resulting lemmas as stored as a list in a separate column `preproc` as shown below.<jupyter_code>%%time
df_preproc['preproc'] = df_preproc['clean'].progress_apply(lemmatize)
df_preproc[['date', 'content', 'preproc']].head(3)<jupyter_output><empty_output><jupyter_text>Applying this method to the `clean` column of the DataFrame and timing it shows that it takes almost a minute to run on $8800$ news articles.  Option 2. Use `nlp.pipe`Can we do better? in the [spaCy documentation](https://spacy.io/api/languagepipe), it is stated that "processing texts as a stream is usually more efficient than processing them one-by-one". This is done by calling a language pipe, which internally divides the data into batches to reduce the number of pure-Python function calls. This means that the larger the data, the better the performance gain that can be obtained by `nlp.pipe`.To use the language pipe to stream texts, a separate lemmatizer method is defined that directly works on a spaCy `Doc` object. This method is then called in batches to work on a *sequence* of `Doc` objects that are streamed through the pipe as shown below.<jupyter_code>def lemmatize_pipe(doc):
    lemma_list = [str(tok.lemma_).lower() for tok in doc
                  if tok.is_alpha and tok.text.lower() not in stopwords] 
    return lemma_list

def preprocess_pipe(texts):
    preproc_pipe = []
    for doc in tqdm(nlp.pipe(texts, batch_size=20), total=len(df_preproc)):
        preproc_pipe.append(lemmatize_pipe(doc))
    return preproc_pipe<jupyter_output><empty_output><jupyter_text>Just as before, a new column is created by passing data from the `clean` column of the existing DataFrame. Note that unlike in workflow $1$, we do not use the `apply` method - instead, the column of data (an iterable) is directly passed as an argument to the preprocessor pipe method.<jupyter_code>%%time
df_preproc['preproc_pipe'] = preprocess_pipe(df_preproc['clean'])
df_preproc[['date', 'content', 'preproc_pipe']].head(3)<jupyter_output><empty_output><jupyter_text>Timing this workflow shows barely any improvement, but it still takes almost a minute on the entire set of $8800$ news articles. One would expect that as we work on bigger and bigger datasets, the timing gain using `nlp.pipe` would become more noticeable (on average).  Option 3. Parallelize the work using joblibWe can do still better! The previous workflows sequentially worked through each news document to produce the lemma lists, which were then appended to the DataFrame as a new column. Because each row's output is completely independent of the other, this is an *embarassingly parallel* problem, making it ideal for using multiple cores.The `joblib` library is recommended by spaCy for processing blocks of an NLP pipeline in parallel. Make sure that you `pip install joblib` before running the below section.To parallelize the workflow, a few more helper methods must be defined. * **Chunking:** The news article content is a list of (long) strings where each document represents a single article's text. This data must be fed in "chunks" to each worker process started by `joblib`. Each call of the `chunker` method returns a generator that only contains that particular chunk's text as a list of strings. During lemmatization, each new chunk is retrieved based on the iterator index (with the previous chunks being "forgotten").* **Flattening:** Once joblib creates a set of worker processes that work on each chunk, each worker returns a "list of list" containing lemmas for each document. These lists are then combined by the executor to provide a deeply nested final "list of list of lists". To ensure that the length of the output from the executor is the same as the actual number of articles, a "flatten" method is defined to combine the result into a list of lists containing lemmas. For example, if the executor returns a final result `[[[a, b, c], [d, e, f]], [[g, h, i], [j, k, l]]]`, a flattened version of this result would be `[[a, b, c], [d, e, f], [g, h, i], [j, k, l]]`.In addition to the above methods, a similar `nlp.pipe` method is used as in workflow $2$, on each chunk of texts. Each of these methods is wrapped into a `preprocess_parallel` method that defines the number of worker processes to be used ($7$ in this case), breaks the input data into chunks and returns a flattened result that can then be appended to the DataFrame.<jupyter_code>from joblib import Parallel, delayed
from functools import partial

def chunker(iterable, total_length, chunksize):
    return (iterable[pos: pos + chunksize] for pos in range(0, total_length, chunksize))

def flatten(list_of_lists):
    "Flatten a list of lists to a combined list"
    return [item for sublist in list_of_lists for item in sublist]

def process_chunk(texts):
    preproc_pipe = []
    for doc in nlp.pipe(texts, batch_size=20):
        preproc_pipe.append(lemmatize_pipe(doc))
    return preproc_pipe

def preprocess_parallel(texts, chunksize=100):
    executor = Parallel(n_jobs=7, backend='multiprocessing', prefer="processes")
    do = delayed(process_chunk)
    tasks = (do(chunk) for chunk in chunker(texts, len(df_preproc), chunksize=chunksize))
    result = executor(tasks)
    return flatten(result)
%%time
df_preproc['preproc_parallel'] = preprocess_parallel(df_preproc['clean'], chunksize=1000)
df_preproc[['date', 'content', 'preproc_parallel']].head(3)<jupyter_output><empty_output><jupyter_text>Timing this parallelized workflow shows significant performance gains (almost **3x** reduction in run time)! As the number of documents becomes larger, the additional overhead of starting multiple worker threads with `joblib` is quickly paid for, and this method can significantly outperform the sequential methods.  Effect of chunk size and batch sizeNote that in the parallelized workflow, two parameters need to be specified - the optimum number can vary depending on the dataset. The `chunksize` controls the number of chunks being worked on by each process. In this example, for $8800$ documents, a chunksize of $1000$ is used. Too small a chunksize would mean that a large number of worker threads would spawn (each one waiting for other threads to complete), which can slow down execution. Generally, a chunksize of around $1/10^{th}$ of the total number of documents can be used as a starting point (assuming that all chunks fit into memory at any given time).The batch size is parameter specific to `nlp.pipe`, and again, a good value depends on the data being worked on. For reasonably long-sized text such as news articles, it makes sense to keep the batch size reasonably small (so that each batch doesn't contain *really* long texts), so in this case $20$ was chosen for the batch size. For other cases (e.g. Tweets) where each document is much shorter in length, a larger batch size can be used.**It is recommended to experiment with either parameter to see which combination produces the best performance**.  Bonus: Use sets over lists for lookups wherever possibleNote that in the `get_stopwords()` method defined earlier on, the list of stopwords read in from the stopword file was converted to a set before using it in the lemmatizer method for stopword removal via lookups. This is a very useful trick in general, but specifically for stopword removal, the use of sets becomes **all the more important**. Why? In any realistic stopword list, such as this one for a news dataset, it's reasonable to expect *several hundred* stopwords. This is because for downstream tasks such as topic modelling or sentiment analysis, there are a number of domain-specific words that need to be removed (very common verbs, useless abbreviations such as timezones, days of the week, etc.). Each word in each and every document needs to be compared against every word in the stopword list, which is an expensive operation over tens of thousands of documents.It's well known that sets have $O(1)$ (i.e. consant) lookup time as opposed to lists, which have $O(n)$ lookup time. In the `lemmatize()` method, since we're checking each word for membership in the set of stopwords, we would expect sets to be much better than lists. To test this, we can rerun workflow $1$ but this time, use a stopword *list* instead.<jupyter_code>stopwords = list(stopwords)
%%time
df_preproc['preproc_stopword_list'] = df_preproc['clean'].progress_apply(lemmatize)
df_preproc[['date', 'content', 'preproc_stopword_list']].head(3)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07276657060518732, 'fraction_numerical': 0.004899135446685879, 'mean_word_length': 4.921928327645051, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 51, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '614623', 'license': 'MIT', 'n_tokens_mistral': 3855, 'n_tokens_neox': 3711, 'n_words': 1843, 'path': 'spacy_multiprocess.ipynb', 'repo_name': 'prrao87/spacy-nlp', 'size': '13880'}