<jupyter_start><jupyter_text>1000 Genomes Project's data on Azure Genomics Data Lake Jupyter notebook is a great tool for data scientists who is working on Genomics data analysis. We will demonstrate Azure Jupyter notebook usage via GATK and Picard with Azure Open Dataset. **Here is the coverage of this notebook:**1. Download the specific data from Azure Genomics Data Lake2. BuildBamIndex (Picard)**Dependencies:**This notebook requires the following libraries:- Azure storage `pip install azure-storage-blob==2.1.0`. Please visit [this page](https://github.com/Azure/azure-storage-python/wiki) for frequently encountered problem for this SDK.- Genome Analysis Toolkit (GATK) (*Users need to download GATK from Broad Institute's webpage into the same compute environment with this notebook: https://github.com/broadinstitute/gatk/releases*)- Technical note: [Explore Azure Genomics Data Lake with Azure Storage Explorer](https://github.com/microsoft/genomicsnotebook/blob/main/docs/Genomics_Data_Lake_Azure_Storage_Explorer.pdf)**Important information: This notebook is using Python 3.6 kernel**  1. Getting the 1000 Genomes Project's data from Azure Open DatasetSeveral public genomics data has been uploaded as an Azure Open Dataset [here](https://azure.microsoft.com/services/open-datasets/catalog/). We create a blob service linked to this open datasets. You can find example of data calling procedure from Azure Open Dataset for `1000 Genomes Project` datasets in below: **1.a.Install Azure Blob Storage SDK**<jupyter_code>pip install azure-storage-blob==2.1.0<jupyter_output><empty_output><jupyter_text>**1.b.Download the targeted file**<jupyter_code>import os
import uuid
import sys
from azure.storage.blob import BlockBlobService, PublicAccess

blob_service_client = BlockBlobService(account_name='dataset1000genomes', sas_token='<KEY>')     
blob_service_client.get_blob_to_path('dataset/phase3/data/HG00096/alignment', 'HG00096.chrom11.ILLUMINA.bwa.GBR.low_coverage.20120522.bam', './HG00096.chrom11.ILLUMINA.bwa.GBR.low_coverage.20120522.bam')<jupyter_output><empty_output><jupyter_text>2. BuildBamIndex (Picard)Generates a BAM index ".bai" file. This tool creates an index file for the input BAM that allows fast look-up of data in a BAM file, lke an index on a database. Note that this tool cannot be run on SAM files, and that the input BAM file must be sorted in coordinate order[1].<jupyter_code>!./gatk BuildBamIndex I=HG00096.chrom11.ILLUMINA.bwa.GBR.low_coverage.20120522.bam O=HG00096.chrom11.ILLUMINA.bwa.GBR.low_coverage.20120522.bam.bai<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08719135802469136, 'fraction_numerical': 0.03742283950617284, 'mean_word_length': 6.046195652173913, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '340794', 'license': 'MIT', 'n_tokens_mistral': 862, 'n_tokens_neox': 765, 'n_words': 257, 'path': 'sample-notebooks/1000-genomes_Azure_Genomics_Data_Lake.ipynb', 'repo_name': 'microsoft/genomicsnotebook', 'size': '2664'}