<jupyter_start><jupyter_text>Loading an HCA matrix into Bioconductor This vignette illustrates requesting an expression matrix from the HCA matrix service and loading it into a Bioconductor R object.First, install and import some dependencies:<jupyter_code>install.packages("downloader")
install.packages("BiocManager")

library("downloader")
library("BiocManager")

BiocManager::install("LoomExperiment")
library(LoomExperiment)

library("httr")<jupyter_output>The downloaded binary packages are in
	/var/folders/nl/dgln31tj7l35g879d6f_tjtc0000gn/T//RtmpDOn1ah/downloaded_packages

The downloaded binary packages are in
	/var/folders/nl/dgln31tj7l35g879d6f_tjtc0000gn/T//RtmpDOn1ah/downloaded_packages<jupyter_text>Now, we're going to make some requests to describe what fields and values we can filter on when we're selecting our matrix.<jupyter_code>r <- GET("https://matrix.data.humancellatlas.org/v1/filters")
content(r)<jupyter_output><empty_output><jupyter_text>That's the list of metadata fields we can filter on when requesting the matrix. We can describe any of them with further API calls:<jupyter_code>r <- GET("https://matrix.data.humancellatlas.org/v1/filters/project.project_core.project_short_name")
print(content(r))
r <- GET("https://matrix.data.humancellatlas.org/v1/filters/genes_detected")
print(content(r))<jupyter_output>$field_description
[1] "Count of genes with a non-zero count."

$field_name
[1] "genes_detected"

$field_type
[1] "numeric"

$maximum
[1] 13108

$minimum
[1] 358<jupyter_text>For categorical data, we see the number of cells associated with each category. For numeric, we see the range of value. If we want to request a matrix based on these metadata values, we can add them to the filter in the body of a POST request to the matrix service:<jupyter_code>payload = list(
    filter =  list(
          op = "and", 
          value = list(
              list(op = "=", value = "Single cell transcriptome analysis of human pancreas",
                   field = "project.project_core.project_short_name"),
              list(op = ">=", value = 300,
                   field = "genes_detected")
    )),
    format = "loom"
)
r <- POST("https://matrix.data.humancellatlas.org/v1/matrix", body = payload, encode = "json")
response <- content(r)
print(response)<jupyter_output>$eta
[1] ""

$matrix_url
[1] ""

$message
[1] "Job started."

$request_id
[1] "ca657a20-5896-4583-80fc-16c4f02657da"

$status
[1] "In Progress"<jupyter_text>That call responds right away and tells us that the matrix is being prepared. We can use the request_id to wait until the matrix is done.<jupyter_code>request_id <- response["request_id"]
status <- response["status"]
message(status)
while (status != "Complete") 
{
    url = paste("https://matrix.data.humancellatlas.org/v1/matrix/", request_id, sep = "")
    r <- GET(url)
    response <- content(r)
    status = response["status"]
    message(status)
    Sys.sleep(15)
}
print(response)<jupyter_output>In Progress
In Progress
In Progress
In Progress
In Progress
In Progress
In Progress
In Progress
In Progress
In Progress
In Progress
Complete<jupyter_text>Now, that the matrix is ready, we can download it. The file we download is a loom-formatted matrix. Loom is the default matrix format, but others can be specified in the matrix request (csv, mtx).<jupyter_code>matrix_file_url = unlist(response["matrix_url"])

download.file(url=matrix_file_url,
              destfile='matrix.loom', method='curl')<jupyter_output><empty_output><jupyter_text>HCA Matrix Service Loom OutputThe Loom format is documented more fully, along with code samples, [here](https://linnarssonlab.org/loompy/index.html).Per Loom [conventions](https://linnarssonlab.org/loompy/conventions/index.html), columns in the loom-formatted expression matrix represent cells, and rows represent genes. The column and row attributes follow Loom conventions where applicable as well: `CellID` uniquely identifies a cell, `Gene` is a gene name, and `Accession` is an ensembl gene id.Descriptions of the remaining metadata fields are available at the [HCA Data Browser](https://prod.data.humancellatlas.org/metadata).And finally, we can `import` the loom file into a `Bioconductor::SingleCellLoomExperiment` object for further analysis in R.<jupyter_code>scle <- import("./matrix.loom", type="SingleCellLoomExperiment")
scle<jupyter_output><empty_output><jupyter_text>The `SingleCellLoomExperiment` also adheres to Loom [conventions](https://linnarssonlab.org/loompy/conventions/index.html) representing features as rows and samples as columns. Expression data is available via the `assays()` method, specifying a named assay.<jupyter_code>assays(scle)$matrix<jupyter_output><empty_output><jupyter_text>Row and column attribute data are available through `rowData()` and `colData()` methods respectively.<jupyter_code>rowData(scle)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10441356763383736, 'fraction_numerical': 0.016142214957090315, 'mean_word_length': 5.348897535667963, 'pattern_counts': {'":': 0, '<': 43, '<?xml version=': 0, '>': 34, 'https://': 9, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '154999', 'license': 'MIT', 'n_tokens_mistral': 1606, 'n_tokens_neox': 1511, 'n_words': 451, 'path': 'docs/HCA Matrix Service to Bioconductor.ipynb', 'repo_name': 'ambrosejcarr/matrix-service', 'size': '4894'}