<jupyter_start><jupyter_text>SKBL - JSON data occupation * GITHUB [salgo60/HISCOKoder](https://github.com/salgo60/HISCOKoder)  * this [Notebook](https://github.com/salgo60/HISCOKoder/blob/main/Jupyter/SKBL%20yrken%20JSON.ipynb)SKBL är en av de få svenska aktörer som har strukturerad data för yrken och ett API. Dock saknas externa kopplingar som Wikidata och HISCOkoder.... även tid anges som textsträngar i olika format, plats saknar även det "samma som" utan blir en textsträng med koordinat.Summa sumarum bra med API men datat är långt från optimalt för att bearbetas vidare. Jag var med på SKBL ledningsgruppsmöte och pratade om Linked data 2020 dec men dom har inte orkat ta steget ännu.... se [presentation](https://docs.google.com/presentation/d/1EvQHdzPsIA4hr2WuNXnbpsLHec2GS89b_pHvilO6xhQ/edit?usp=sharing) tror inte KARP plattformen är optimal för "samma som" plus att det behövs nya kompetenser som Linked data i projektet för att det skall ta fart.... Other sources we sync* [Arken](https://github.com/salgo60/open-data-examples/blob/master/Arken.ipynb)   * WD [Property:P8899](https://www.wikidata.org/wiki/Property:P8899) * [Kulturpersoner Uppsalakyrkogård](https://github.com/salgo60/open-data-examples/blob/master/Check%20WD%20kulturpersoner%20uppsalakyrkogardar.ipynb)* [Litteraturbanken](https://github.com/salgo60/open-data-examples/blob/master/Litteraturbanken%20Author.ipynb)   * WD property [P5101](https://www.wikidata.org/wiki/Property_talk:P5101) [P5123](https://www.wikidata.org/wiki/Property_talk:P5123)* [Nobelprize.org](https://github.com/salgo60/open-data-examples/blob/master/Nobel%20API.ipynb)  * WD [property 8024](https://www.wikidata.org/wiki/Property:P8024)* [SBL](https://github.com/salgo60/open-data-examples/blob/master/SBL.ipynb)   * WD [property 3217](https://www.wikidata.org/wiki/Property:P3217) * [SKBL](https://github.com/salgo60/open-data-examples/blob/master/Svenskt%20Kvinnobiografiskt%20lexikon%20part%203.ipynb)  * WD [property 4963](https://www.wikidata.org/wiki/Property:P4963)* [Svenska Akademien](https://github.com/salgo60/open-data-examples/blob/master/Svenska%20Akademien.ipynb)   * WD [property 5325](https://www.wikidata.org/wiki/Property:P5325)<jupyter_code>from datetime import datetime
start_time  = datetime.now()
print("Last run: ", start_time)
import urllib3, json
import pandas as pd 
http = urllib3.PoolManager() 


url = "https://ws.spraakbanken.gu.se/ws/karp/v5/query?mode=skbl&q=extended%7C%7Cand%7Cskblstatus%7Cequals%7Cok&size=10000"
r = http.request('GET', url) 
data = json.loads(r.data.decode('utf-8')) 

listSKBLYrke = []
i = 0 
for row in (data["hits"]["hits"]):
#    print(row)
#    print (row["_source"]["url"])

    new_item = dict()
    new_item['skblid'] = row["_source"]["url"]
   
    try:
        #print (row["_source"]["occupation"])
        for o in row["_source"]['occupation']:    
#            print("\t",o['description'])
#            print("\t\t",o['detail'])
#            print("\t\t",o['place'])
#            print("\t\t",o['type'])
#	 Sakkunnig
#		 Sakkunnig, bl a 1924 års privatskoleutredning
#		 {'pin': {'lat': 59.329323, 'lon': 18.068581}, 'place': 'Stockholm'}
#		 Ideellt arbete
#	 Chef
#		 VD, <NAME>
#		 {'pin': {'lat': 59.329323, 'lon': 18.068581}, 'place': 'Stockholm'}
#		 Yrke
            try:
                description = o['description']
            except:
                description = ""
            new_item['description'] = description 
            try:
                detail = o['detail']               
            except:
                detail = ""
            new_item['detail'] = detail 
            try:
                Ofrom = o['from']               
            except:
                Ofrom = ""
            new_item['from'] = Ofrom 
            try:
                Oto = o['to']               
            except:
                Oto = ""
            new_item['to'] = Oto 
            try:
                place = o['place']               
            except:
                place = ""
            new_item['place'] = place 
            try:
                occupationtype = o['type']
            except:
                occupationtype = ""
            new_item['occupationtype'] = occupationtype 

    except Exception as e: 
        print("Error occupation\t",row["_source"]["url"],e)
    listSKBLYrke.append(new_item)
    i = i + 1 
    #print (i)
print (len(listSKBLYrke) ," antal poster")

listSKBLYrke[1:10]
import pandas as pd
newListdf = pd.DataFrame(listSKBLYrke,columns=list
                         (["skblid","description","detail","occupationtype","from","to","place"]))
newListdf.info()
newListdf.head(50)
newListdf["skblid"].value_counts()
Yrkendf = newListdf["description"].value_counts()
Yrkendf
newListdf["occupationtype"].value_counts()
import matplotlib.pyplot as plt
plt.rcParams["figure.figsize"] = (10,10)
ax = Yrkendf.plot(kind='pie')

ax.set_title("Vanligaste Yrken i SKBL " + start_time.strftime('%Y %m %d'))
fig = ax.get_figure()
plt.rcParams['savefig.facecolor']='white'
fig.savefig('../img/SKBL_yrken_json.png')
newListdf.to_csv("SKBLyrken_newListdf.csv")
Yrkendf.to_csv("SKBLyrken_valuecount.csv")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1487762574677202, 'fraction_numerical': 0.033339757178647136, 'mean_word_length': 3.155324259407526, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 6, 'https://': 18, 'lorem ipsum': 0, 'www.': 7, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '488070', 'license': 'CC0-1.0', 'n_tokens_mistral': 1996, 'n_tokens_neox': 1833, 'n_words': 316, 'path': 'Jupyter/.ipynb_checkpoints/SKBL yrken JSON-checkpoint.ipynb', 'repo_name': 'salgo60/HISCOKoder', 'size': '5220'}