<jupyter_start><jupyter_text>![JohnSnowLabs](https://nlp.johnsnowlabs.com/assets/images/logo.png)[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/streamlit_notebooks/ocr/PDF_TEXT_NER.ipynb)  Recognize entities in scanned PDFs To run this yourself, you will need to upload your **Spark OCR** license keys to the notebook. Otherwise, you can look at the example outputs at the bottom of the notebook. To upload license keys, open the file explorer on the left side of the screen and upload `workshop_license_keys.json` to the folder that opens. For more in-depth tutorials: https://github.com/JohnSnowLabs/spark-ocr-workshop/tree/master/jupyter  1. Colab Setup Read licence key<jupyter_code>import json
import os

from google.colab import files

license_keys = files.upload()
os.rename(list(license_keys.keys())[0], 'spark_ocr.json')

with open('spark_ocr.json') as f:
    license_keys = json.load(f)

# Defining license key-value pairs as local variables
locals().update(license_keys)<jupyter_output><empty_output><jupyter_text>Install Dependencies<jupyter_code># Installing pyspark and spark-nlp
! pip install --upgrade -q pyspark==3.0.2 spark-nlp==$PUBLIC_VERSION

# Installing Spark NLP Healthcare
! pip -q install --upgrade spark-nlp-jsl==$JSL_VERSION  --extra-index-url https://pypi.johnsnowlabs.com/$SECRET

# Installing Spark OCR
! pip install spark-ocr==$OCR_VERSION\+spark30 --extra-index-url=https://pypi.johnsnowlabs.com/$SPARK_OCR_SECRET --upgrade<jupyter_output><empty_output><jupyter_text>!!! ATTENTION !!! After running previous cell, RESTART the COLAB RUNTIME  and go ahead. Importing Libraries<jupyter_code>import json, os

with open("spark_ocr.json", 'r') as f:
  license_keys = json.load(f)

# Adding license key-value pairs to environment variables
os.environ.update(license_keys)

# Defining license key-value pairs as local variables
locals().update(license_keys)
import pandas as pd
import numpy as np
import os

#Pyspark Imports
from pyspark.sql import SparkSession
from pyspark.ml import PipelineModel
from pyspark.sql import functions as F

# Necessary imports from Spark OCR library
import sparkocr
from sparkocr import start
from sparkocr.transformers import *
from sparkocr.enums import *
from sparkocr.utils import display_image, to_pil_image
from sparkocr.metrics import score
import pkg_resources

# import sparknlp packages
from sparknlp.annotator import *
from sparknlp.base import *
import sparknlp_jsl
from sparknlp_jsl.annotator import *<jupyter_output><empty_output><jupyter_text>Start Spark Session<jupyter_code># Start spark
spark = sparkocr.start(secret=SPARK_OCR_SECRET, 
                       nlp_version=PUBLIC_VERSION,
                       nlp_secret=SECRET,
                       nlp_internal=JSL_VERSION
                       )
spark<jupyter_output>Spark version: 3.0.2
Spark NLP version: 3.3.4
Spark OCR version: 3.9.1<jupyter_text>2. Download and read scanned pdf image. **To process PDF, download it and just use pdf_to_image annotator instead of binary_to_image in the pipeline**<jupyter_code>!wget https://www.reneelab.com/wp-content/uploads/sites/2/2015/11/target-500x600.png -O 1.jpg
image_df = spark.read.format("binaryFile").load('./1.jpg').cache()
image_df.show()<jupyter_output>+-------------------+-------------------+------+--------------------+
|               path|   modificationTime|length|             content|
+-------------------+-------------------+------+--------------------+
|file:/content/1.jpg|2016-12-19 13:28:45|145284|[89 50 4E 47 0D 0...|
+-------------------+-------------------+------+--------------------+<jupyter_text>3. Construct OCR & NLP pipelines OCR Pipleline<jupyter_code># To load PDF instead of Image,
#pdf_to_image = PdfToImage() \
#            .setInputCol("content") \
#            .setOutputCol("image_raw") \
#            .setKeepInput(True)

# Read binary as image
binary_to_image = BinaryToImage()
binary_to_image.setInputCol('content')
binary_to_image.setOutputCol('image')

# Scale image
scaler = ImageScaler()
scaler.setInputCol('image')
scaler.setOutputCol('scaled_image')
scaler.setScaleFactor(2.0)

# Binarize using adaptive tresholding
binarizer = ImageAdaptiveThresholding()
binarizer.setInputCol('scaled_image')
binarizer.setOutputCol('binarized_image')
binarizer.setBlockSize(91)
binarizer.setOffset(70)

# Remove extraneous objects from image
remove_objects = ImageRemoveObjects()
remove_objects.setInputCol('binarized_image')
remove_objects.setOutputCol('cleared_image')
remove_objects.setMinSizeObject(30)
remove_objects.setMaxSizeObject(4000)

# Apply morphology opening
morpholy_operation = ImageMorphologyOperation()
morpholy_operation.setKernelShape(KernelShape.DISK)
morpholy_operation.setKernelSize(1)
morpholy_operation.setOperation('closing')
morpholy_operation.setInputCol('cleared_image')
morpholy_operation.setOutputCol('corrected_image')

# Extract text from corrected image with OCR
ocr = ImageToText()
ocr.setInputCol('binarized_image')
ocr.setOutputCol('text')
ocr.setConfidenceThreshold(50)
ocr.setIgnoreResolution(False)

# Create pipeline
pipeline = PipelineModel(stages=[
    binary_to_image,
    scaler,
    binarizer,
    remove_objects,
    morpholy_operation,
    ocr])<jupyter_output><empty_output><jupyter_text>NLP Pipeline containing **Spell Correction** and **NER**<jupyter_code>documentAssembler = DocumentAssembler().setInputCol("text").setOutputCol("document")

tokenizer = Tokenizer() \
    .setInputCols(["document"]) \
    .setOutputCol("token")

spellModel = ContextSpellCheckerModel\
    .pretrained('spellcheck_dl')\
    .setInputCols("token")\
    .setOutputCol("checked")
    
embeddings = WordEmbeddingsModel.pretrained('glove_100d').\
                    setInputCols(["document", 'checked']).\
                    setOutputCol("embeddings")

public_ner = NerDLModel.pretrained('onto_100', 'en') \
          .setInputCols(["document", "token", "embeddings"]) \
          .setOutputCol("ner")

ner_converter = NerConverter() \
                .setInputCols(["document", "token", "ner"]) \
                  .setOutputCol("ner_chunk")

nlp_pipeline =  Pipeline(stages=[documentAssembler, 
                                tokenizer,
                                spellModel,
                                embeddings,
                                public_ner,
                                ner_converter])<jupyter_output>spellcheck_dl download started this may take some time.
Approximate size to download 111.4 MB
[OK!]
glove_100d download started this may take some time.
Approximate size to download 145.3 MB
[OK!]
onto_100 download started this may take some time.
Approximate size to download 13.5 MB
[OK!]<jupyter_text>4. Run OCR pipeline<jupyter_code>result = pipeline.transform(image_df).cache()<jupyter_output><empty_output><jupyter_text>5. Visualize Results Display result dataframe<jupyter_code>result.select("text", "confidence").show()<jupyter_output>+--------------------+----------------+
|                text|      confidence|
+--------------------+----------------+
|ADVERTISEMENT.

T...|91.8821029663086|
+--------------------+----------------+<jupyter_text>Display text and images<jupyter_code>result_arr = []
for r in result.distinct().collect():
  print (r.text)
  result_arr.append(r.text)<jupyter_output>ADVERTISEMENT.

Tuts publication of the Works of <NAME>, it is
supposed, will extend to Five Volumes. It was thought
advisable to commence the series with his History of
the Reformation in Scotland, as the work of greatest
importance. The next volume will thus contain the
Third and Fourth Books, which continue the History to
the year 1564; at which period his historical labours
may be considered to terminate. But the Fifth Book,
forming a sequel to the History, and published under
his name in 1644, will also be included. His Letters
and Miscellancous Writings will be arranged in the
subsequent volumes, as nearly as possible in chronolo-
gical order; each portion being introduced by a separate
avtice, respecting the manuscript or printed copies from
which they have been taken.

It may perhaps be expected that a Life of the Author
thould have been prefixed to this volume. The Life of
Knox, by Dr. M-Crig, is however a work so universally
known, and of so much historical value, as to su[...]<jupyter_text>6. Run NLP pipeline<jupyter_code>empty_df = spark.createDataFrame([['']]).toDF("text")
pipelineModel = nlp_pipeline.fit(empty_df)
df = spark.createDataFrame(pd.DataFrame({"text":result_arr}))
nlp_result = pipelineModel.transform(df)<jupyter_output><empty_output><jupyter_text>7. Visualize NLP results Contextual Spell Correction<jupyter_code>nlp_result.select(F.explode(F.arrays_zip('token.result', 'checked.result')).alias("cols")) \
.select(F.expr("cols['0']").alias("original"),
        F.expr("cols['1']").alias("corrected")).show(truncate=False)<jupyter_output>+-------------+-------------+
|original     |corrected    |
+-------------+-------------+
|ADVERTISEMENT|ADVERTISEMENT|
|.            |.            |
|Tuts         |puts         |
|publication  |publication  |
|of           |of           |
|the          |the          |
|Works        |Works        |
|of           |of           |
|Jonn         |John         |
|Kwox         |Knox         |
|,            |,            |
|it           |it           |
|is           |is           |
|supposed     |supposed     |
|,            |,            |
|will         |will         |
|extend       |extend       |
|to           |to           |
|Five         |Five         |
|Volumes      |Volumes      |
+-------------+-------------+
only showing top 20 rows<jupyter_text>NER<jupyter_code>nlp_result.select(F.explode(F.arrays_zip('ner_chunk.result', 'ner_chunk.metadata')).alias("cols")) \
.select(F.expr("cols['0']").alias("chunk"),
        F.expr("cols['1']['entity']").alias("ner_label")).show(truncate=False)<jupyter_output>+--------------------------------------+-----------+
|chunk                                 |ner_label  |
+--------------------------------------+-----------+
|the Works of <NAME>                |WORK_OF_ART|
|Five Volumes                          |WORK_OF_ART|
|History of
the Reformation            |WORK_OF_ART|
|Scotland                              |GPE        |
|Third                                 |ORDINAL    |
|Fourth                                |ORDINAL    |
|History to
the year 1564              |DATE       |
|labours                               |ORG        |
|Fifth                                 |ORDINAL    |
|History                               |WORK_OF_ART|
|1644                                  |DATE       |
|His Letters
and Miscellancous Writings|WORK_OF_ART|
|a Life of the Author                  |WORK_OF_ART|
|The Life of
Knox                      |WORK_OF_ART|
|Dr                                    |PERSON     |
|M-Crig                                |PERSON[...]
{'fraction_non_alphanumeric': 0.1453752498637107, 'fraction_numerical': 0.012902053425404325, 'mean_word_length': 2.7248730964467005, 'pattern_counts': {'":': 1, '<': 48, '<?xml version=': 0, '>': 48, 'https://': 7, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 13, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '516674', 'license': 'Apache-2.0', 'n_tokens_mistral': 3482, 'n_tokens_neox': 3268, 'n_words': 778, 'path': 'tutorials/streamlit_notebooks/ocr/PDF_TEXT_NER.ipynb', 'repo_name': 'fcivardi/spark-nlp-workshop', 'size': '11012'}