<jupyter_start><jupyter_text>Open Data Standards, As Standard  Jupyter Setup Assuming we are in the "dm_public/notebooks" directory, let's change to the base directory to keep consistency between paths in Jupyter and Python at the command line. If you did not launch Jupyter from the dm_public directory, do not execute this cell. And be sure to only run it once!<jupyter_code>%cd ../..<jupyter_output><empty_output><jupyter_text>Specify some constants referring to the input XML files and schemas. We'll work with these for the rest of this notebook.  Validating PCML and PCRRUse the lxml library to validate that our recipe PCML and runtime PCRR adheres to the respective standards.<jupyter_code>#validate the PCML against its XSD schema
from dm.data.xml import validate_xml

pcml_schema_file = './02_PCML_and_PCRR/data/pcml-1.3.4.xsd'
pcml_recipe_file = './02_PCML_and_PCRR/data/3a_recipe.pcml'
validate_xml(pcml_recipe_file, pcml_schema_file)
print("XML for '{}' adheres to schema '{}'".format(pcml_schema_file, pcml_recipe_file))
    
#verify the signatures in all PCRR files.
pcrr_schema_file = './02_PCML_and_PCRR/data/pcrr-0.0.1.xsd'
pcrr_recipe_files = ['./02_PCML_and_PCRR/data/3a_run_01.pcrr',
                     './02_PCML_and_PCRR/data/3a_run_02.pcrr',
                     './02_PCML_and_PCRR/data/3a_run_03.pcrr']

for rr in pcrr_recipe_files:
    validate_xml(rr, pcrr_schema_file)
    print("XML for '{}' adheres to schema '{}'".format(rr, pcrr_schema_file))<jupyter_output><empty_output><jupyter_text>Verifying X.509 Signatures Now let's validate the X.509 certificate. As this is self signed, it is included in the same directory as the data.<jupyter_code>#verify the X509 signature of the PCML
from dm.data.xml import verify_x509_signature
from signxml import InvalidSignature


def verify_sig(xml_file, cer_file):

    try:
        verify_x509_signature(xml_file, cer_file)
        print("X.509 signature of '{}' is valid as generated by DigitalGlassware".format(xml_file))
    except InvalidSignature as e:
        raise ValueError("X.509 signature for '{}' is invalid -- it may not have been generated by DigitalGlassware, or has been modified after signing.".format(xml_file)) from e
    except Exception as e:
        raise Exception("An error occurred whilst verifying X.509 signature of '{}'".format(xml_file)) from e

    
pcml_cer_path = "./02_PCML_and_PCRR/data/x509/pcml.crt"
verify_sig(pcml_recipe_file, pcml_cer_path)
#verify the signatures in all PCRR files.

pcrr_cer_path = "./02_PCML_and_PCRR/data/x509/pcrr.crt"

for rr in pcrr_recipe_files:
    verify_sig(rr, pcrr_cer_path)<jupyter_output><empty_output><jupyter_text>Transforming PCML<jupyter_code>#apply an XSL transformation to the PCML data

from dm.data.xml import transform as pcml_transform

pcml_xslt_flow_file = "./02_PCML_and_PCRR/data/flow.xsl"
pcml_transform(pcml_recipe_file, pcml_xslt_flow_file, outfile = "./02_PCML_and_PCRR/out/3a_pcml_flow.html")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08813672922252011, 'fraction_numerical': 0.022453083109919572, 'mean_word_length': 4.910891089108911, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 13}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '255879', 'license': 'MIT', 'n_tokens_mistral': 1091, 'n_tokens_neox': 1031, 'n_words': 280, 'path': '02_PCML_and_PCRR/notebooks/standards.ipynb', 'repo_name': 'deepmatterltd/dm_datascience', 'size': '2984'}