<jupyter_start><jupyter_text>One interesting thing to know about the work plan species is their connection to State Species of Greatest Conservation Need. The USGS builds and maintains a synthesis of state species, linking species names to taxonomic authorities (ITIS and WoRMS) to produce a synthesized National list for each decadal reporting period (with some periodic updates during the intervening years). This notebook leverages an sgcn module in the bispy package to search the National List SGCN API. It returns and caches the summarized National List records, which include the list of states that have the species in their conservation planning process.<jupyter_code>import requests
import bispy
from IPython.display import display
from joblib import Parallel, delayed
from collections import Counter
import jsonschema

sgcn = bispy.sgcn.Search()
bis_utils = bispy.bis.Utils()

import helperfunctions
name_list = helperfunctions.workplan_species()
%%time
# Use joblib to run multiple requests for SGCN records in parallel via scientific names
sgcn_results = Parallel(n_jobs=8)(delayed(sgcn.search)(name, name_source) for name, name_source in name_list)

# Cache the array of retrieved documents and return/display a random sample for verification
display(bis_utils.doc_cache("../cache/sgcn.json", sgcn_results))<jupyter_output><empty_output><jupyter_text>Schema Validation<jupyter_code>sgcn_schema = helperfunctions.load_schema('sgcn')
display(sgcn_schema)

jsonschema.validate(sgcn_results, sgcn_schema)<jupyter_output><empty_output><jupyter_text>Interesting FacetsAs a point of reference, we were able to find a number of additional matches to the SGCN species based on having established a linkage to a valid ITIS record. The following code block provides a quick reference on the number of SGCN matches found (high number linked to state species of conservation need as we would expect) and the number found by having the ITIS information included in the name lookup.Also, in looking at the overall results, we see that there are 72 names in the workplan species list that are not found on any current state list of species of greatest conservation need.<jupyter_code>Counter(spp["parameters"]["Name Source"] for spp in [r for r in sgcn_results if r["processing_metadata"]["status"] == "success"])
Counter(spp["processing_metadata"]["status"] for spp in sgcn_results)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.05682289506428868, 'fraction_numerical': 0.001244296972210701, 'mean_word_length': 6.421538461538462, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '277919', 'license': 'Unlicense', 'n_tokens_mistral': 641, 'n_tokens_neox': 617, 'n_words': 290, 'path': 'workflow/Check SGCN List.ipynb', 'repo_name': 'usgs-bcb/FWS-ESA-WorkPlanSpecies', 'size': '2411'}