<jupyter_start><jupyter_text>My first Notebook with expert.ai Detectors API  **expert.ai Natural Language API v2** (https://developer.expert.ai/) parses and "understands" large volumes of text.In this section we will explain how to install and use our Detectors API and then introduce some key concepts regarding Natural Language Processing.You can download this notebook and our Python SDK's source code from Github at https://github.com/therealexpertai/  Installation and SetupFirst of all, you have to install the expert.ai-nlapi library using pip. * https://pypi.org/project/expertai-nlapi/Then, you will also need to install __pandas__ library since it will be useful in order to create tables and show the results of the analysis.  Working with NL API in PythonNow you have to setup your account credentials; if you don't have them, you can get them at https://developer.expert.ai/ui/login Set your environment variables with NL API credentials ```bashSET EAI_USERNAME=YOUR_USERSET EAI_PASSWORD=<PASSWORD>```or ```bashexport EAI_USERNAME=YOUR_USERexport EAI_PASSWORD=<PASSWORD>```as an alternative you can always add to your notebook the following statements```pythonimport osos.environ["EAI_USERNAME"] = 'YOUR_USER'os.environ["EAI_PASSWORD"] = '<PASSWORD>'``` __Now let's play with Python and Natural Language Processing__ Currently our Detectors API support two languages (English and Italian).In order to run the Detector's analysis you have to define the text you want to analyze and the language model to be used.<jupyter_code>from expertai.nlapi.cloud.client import ExpertAiClient
import json, os
client = ExpertAiClient()
textsToAnalyzeIt=[]
for file in os.listdir("demo_it"):
    with open('demo_it/'+file) as f:
        textsToAnalyzeIt.append({'txt':f.read(),'fileName':file})<jupyter_output><empty_output><jupyter_text>Quick runNow you can start using our Detectors API by just sending a text. This is how it looks like:<jupyter_code>documentsIt = []
for textObj in textsToAnalyzeIt:
    documentsIt.append({
        'file': textObj['fileName'],
        'res': client.detection(body={"document": {"text": textObj['txt']}}, params={'language': 'it','detector':'pii'})
    })<jupyter_output><empty_output><jupyter_text>The `detect` analysis for PII returns all the information in a document that could be considered sensitive since they could be used to identify a specific individual.  DataTableGenerate dataTable extractions from demo texts in Italian:<jupyter_code>import pandas as pd
import json
from IPython.core.display import display, HTML

pd.set_option('display.max_rows', None)
mapColoredCell = set()
def coloredCell(s):
    key = '-'.join(s.name[0:3])
    if(key not in mapColoredCell):
        mapColoredCell.add(key)
        return ['border-top: 1px solid !important']
    
    return['']
   

dataToShowIt = []   
for doc in documentsIt:
    mapInstances = {}
    partianElements = []
    fieldName=""
    for extraction in doc['res'].extractions:
        
        if extraction.template in mapInstances:
            mapInstances[extraction.template] += 1
        else:
            mapInstances[extraction.template] = 1
        dateCount=0;
        for field in extraction.fields:
            fieldName = field.name
            if field.name == "dateTime":
                dateCount+=1
                fieldName+= " #"+str(dateCount)
            row = {
                "file" : doc['file'],
                "template": extraction.template,
                "instance" : '#' + str(mapInstances[extraction.template]),
                'field':fieldName,
                'value':field.value
            }
            partianElements.append(row)
            dataToShowIt.append(row)
    for e in partianElements:
        inst = mapInstances[e['template']]
        if inst == 1:
            e['instance'] = ''
            

dfIt = pd.DataFrame(dataToShowIt)
dfIt.set_index(['file','template', 'instance', 'field'], inplace=True)
left_aligned_dfIt = dfIt.style.set_properties(**{'text-align': 'left', 'padding-left': '30px'})  
left_aligned_dfIt.apply(coloredCell,axis=1)
display(left_aligned_dfIt)<jupyter_output><empty_output><jupyter_text>JSON LD responseMoreover, you can also retrive json-ld data by using the following code:<jupyter_code>for doc in documentsIt:
    print("************************")
    print (doc['file']+": ")
    print(json.dumps(doc['res'].extra_data, indent=2, sort_keys=True))
    print("************************")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11657397107897664, 'fraction_numerical': 0.0028921023359288096, 'mean_word_length': 3.89760348583878, 'pattern_counts': {'":': 5, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '156403', 'license': 'Apache-2.0', 'n_tokens_mistral': 1322, 'n_tokens_neox': 1284, 'n_words': 397, 'path': 'nlapi/PII/pii_detection_it_nlapi.ipynb', 'repo_name': 'camato-es/notebooks', 'size': '4504'}