<jupyter_start><jupyter_text>Runing Pretrained modelsIn the following example, we walk-through different use cases of some of our Pretrained models and pipelines which could be used off the shelf.There is BasicPipeline which will return tokens, normalized tokens, lemmas and part of speech tags. The AdvancedPipeline will return same as the BasicPipeline plus Stems, Spell Checked tokens and NER entities using the CRF model. All the pipelines and pre trained models are downloaded from internet at run time hence would require internet access.   Spark `2.4` and Spark NLP `2.0.1`  1. Call necessary imports and create the spark session<jupyter_code>import os
import sys
sys.path.append('../../')

print(sys.version)

from sparknlp.pretrained import ResourceDownloader
from sparknlp.base import DocumentAssembler
from sparknlp.annotator import *

from pyspark.sql import SparkSession
from pyspark.ml import Pipeline

spark = SparkSession.builder \
    .appName("Model Downloader")\
    .master("local[*]")\
    .config("spark.driver.memory","4G")\
    .config("spark.driver.maxResultSize", "2G")\
    .config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.1")\
    .config("spark.kryoserializer.buffer.max", "500m")\
    .getOrCreate()<jupyter_output><empty_output><jupyter_text>2. Create a dummy spark dataframe<jupyter_code>l = [
  (1,'To be or not to be'),
  (2,'This is it!')
]

data = spark.createDataFrame(l, ['docID','text'])<jupyter_output><empty_output><jupyter_text>3. We use predefined BasicPipeline in order to annotate a dataframe with it<jupyter_code># download predefined - pipelines
from sparknlp.pretrained import PretrainedPipeline

basic_pipeline = PretrainedPipeline("pipeline_basic")
basic_data = basic_pipeline.annotate(data, 'text') 
basic_data.show()<jupyter_output><empty_output><jupyter_text>We can also annotate a single string<jupyter_code># annotat quickly from string
basic_pipeline.annotate("This world is made up of good and bad things")<jupyter_output><empty_output><jupyter_text>4. Now we intend to use one of the fast pretrained models such as Preceptron model which is a POS model trained with ANC American Corpus<jupyter_code>document_assembler = DocumentAssembler() \
    .setInputCol("text")

sentence_detector = SentenceDetector() \
    .setInputCols(["document"]) \
    .setOutputCol("sentence")

tokenizer = Tokenizer() \
    .setInputCols(["sentence"]) \
    .setOutputCol("token")
    
# download directly - models
pos = PerceptronModel.pretrained() \
    .setInputCols(["sentence", "token"]) \
    .setOutputCol("pos")
    
pipeline = Pipeline(stages=[document_assembler, sentence_detector, tokenizer, pos])

output = pipeline.fit(data).transform(data)
output.show()<jupyter_output><empty_output><jupyter_text>5. Now we proceed to download a Fast CRF Named Entity Recognitionl which is trained with Glove embeddings. Then, we retrieve the Basic Pipeline and combine these models to use them appropriately meeting their requirements.<jupyter_code># download predefined - models
pos = PerceptronModel.pretrained() \
     .setInputCols(["document", "normal"])\
     .setOutputCol("pos")

ner = NerCrfModel.pretrained()
ner.setInputCols(["pos", "normal", "document"]).setOutputCol("ner")

annotation_data = basic_pipeline.transform(data)

pos_tagged = pos.transform(annotation_data)
ner_tagged = ner.transform(pos_tagged)
ner_tagged.show()<jupyter_output><empty_output><jupyter_text>6. Finally, lets try a pre trained sentiment analysis pipeline<jupyter_code>PretrainedPipeline("pipeline_vivekn").annotate("This is a good movie!!!")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10645966176878292, 'fraction_numerical': 0.005822012752980316, 'mean_word_length': 5.74392523364486, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '279901', 'license': 'Apache-2.0', 'n_tokens_mistral': 1137, 'n_tokens_neox': 1103, 'n_words': 311, 'path': 'python/example/model-downloader/ModelDownloaderExample.ipynb', 'repo_name': 'picachu93/spark-nlp', 'size': '3607'}