<jupyter_start><jupyter_text>Inspecting the HBN/BIDS_curated datasetThis notebook demonstrates simple usage of Cloud-BIDS-Layout to inspect the HBN/BIDS_curated dataset hosted on FCP-INDI.<jupyter_code>import cloud_bids_layout as cbl
from datetime import datetime
print(datetime.now())
layout = cbl.CloudBIDSLayout(
    remote_location="s3://fcp-indi/data/Projects/HBN/BIDS_curated",
    download_dir="./hbn-curated"
)
print(datetime.now())<jupyter_output>2020-10-30 14:10:36.073978<jupyter_text>Now we have a pybids `BIDSLayout` instance and we can use the familiar `get()` method to query our dataset. For example:<jupyter_code># Get the subject IDs for all subjects with DWI data
dwi_subs = layout.get(return_type="id", target="subject", datatype="dwi")
all_subs = layout.get(return_type="id", target="subject")
print("Total number of subjects: ", len(all_subs))
print("Total number of subjects with DWI: ", len(dwi_subs))
dwi_subs_with_json = layout.get(return_type="id", target="subject", datatype="dwi", extension="json")
print("Total number of subject with json sidecars in the DWI directory: ", len(dwi_subs_with_json))<jupyter_output>Total number of subject with json sidecars in the DWI directory:  960
{'fraction_non_alphanumeric': 0.11221122112211221, 'fraction_numerical': 0.019801980198019802, 'mean_word_length': 6.980263157894737, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '150520', 'license': 'MIT', 'n_tokens_mistral': 423, 'n_tokens_neox': 405, 'n_words': 109, 'path': 'examples/hbn_curated.ipynb', 'repo_name': 'richford/cloud_bids_layout', 'size': '1212'}