<jupyter_start><jupyter_text>Degine Data Processing Step<jupyter_code># -z https://aiadvocate.blob.core.windows.net/public/tacodata.zip -t train.txt
food_dataset = PipelineData(
    "training_set",
    datastore=datastore)

processStep = PythonScriptStep(
    script_name="data.py",
    arguments=["-b", datareference, "-z", "https://aiadvocate.blob.core.windows.net/public/tacodata.zip", "-t", food_dataset],
    inputs=[datareference],
    outputs=[food_dataset],
    compute_target=compute,
    source_directory="."
)<jupyter_output><empty_output><jupyter_text>Test Run<jupyter_code>pipeline1 = Pipeline(workspace=ws, steps=[processStep])
# Submit the pipeline to be run
pipeline_run1 = Experiment(ws, 'burritos_tacos').submit(pipeline1)
pipeline_run1.wait_for_completion()<jupyter_output>Created step data.py [891fff58][28d12a48-7941-4d99-8f78-427615c8dc45], (This step will run and generate new outputs)
Created data reference tacosandburritos for StepId [87bea556][7a055284-e57c-4f6b-b1bd-cecddb85757f], (Consumers of this data will generate new runs.)
Submitted pipeline run: 30846f4b-ebf9-4b69-864b-3a9e1f1e460f
PipelineRunId: 30846f4b-ebf9-4b69-864b-3a9e1f1e460f
Link to Portal: https://mlworkspace.azure.ai/portal/subscriptions/91d27443-f037-45d9-bb0c-428256992df6/resourceGroups/Robots/providers/Microsoft.MachineLearningServices/workspaces/hal/experiments/burritos_tacos/runs/30846f4b-ebf9-4b69-864b-3a9e1f1e460f
PipelineRun Status: NotStarted
PipelineRun Status: Running


StepRunId: 0d6f4e74-7516-4d24-b979-ea482596a0e1
Link to Portal: https://mlworkspace.azure.ai/portal/subscriptions/91d27443-f037-45d9-bb0c-428256992df6/resourceGroups/Robots/providers/Microsoft.MachineLearningServices/workspaces/hal/experiments/burritos_tacos/runs/0d6f4e74-7516-4d24-b979-ea482596a0e1
StepR[...]
{'fraction_non_alphanumeric': 0.11797440178074568, 'fraction_numerical': 0.11686143572621036, 'mean_word_length': 6.716738197424893, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '235183', 'license': 'MIT', 'n_tokens_mistral': 812, 'n_tokens_neox': 708, 'n_words': 88, 'path': 'hour6/pipeline.ipynb', 'repo_name': 'sethjuarez/cloudscaleml', 'size': '1797'}