<jupyter_start><jupyter_text>Data analytics<jupyter_code># !pip install google_auth_oauthlib 
# !pip install install google-api-python-client
# !pip install pandas 
# !pip install numpy
# !pip install matplotlib
# !pip install google-cloud-bigquery
# !pip install db-dtypes

# DEV GUIDES
#https://developers.google.com/analytics/devguides/reporting
    



import warnings
warnings.filterwarnings('ignore')
%load_ext autoreload
%autoreload 2

import analytics_charts as ac
plot_dl_sessions, plot_dl_sessions_types, plot_dl_sessions_matrices = ac.plot_downloads()<jupyter_output>D:\Hunter_ext\anaconda3\lib\site-packages\google\auth\_default.py:79: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK without a quota project. You might receive a "quota exceeded" or "API not enabled" error. We recommend you rerun `gcloud auth application-default login` and make sure a quota project is added. Or you can use service accounts instead. For more information about service accounts, see https://cloud.google.com/docs/authentication/
  warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)<jupyter_text>File downloads - From download logsFrom Nov, 2021 to Mar, 2022 we were able to query download logs from the HCA DCP Google Cloud Platform (GCP). Here a session is counted as activity from a sinle IP address during a single day. This is likely an under esitmate of "downlaod sessions".  While downloads are logged for compliance reasons we no longer have access to the downlogs logs for analytics purposes. This represents downloads from all sources e.g. Terra workspaces, Azul API, Data Browser, notebooks etc.<jupyter_code>plot_dl_sessions()<jupyter_output><empty_output><jupyter_text>The following graphs break down the above downloads by file type category (FASTQ, BAM, matrix, or other). These categories are non-exclusive -- a single session may download files of multiple types.<jupyter_code>plot_dl_sessions_types()<jupyter_output><empty_output><jupyter_text>The matrix downloads are here broken down further, to show sessions downloading contributor-generated matrices, sessions downloading DCP matrices, and the overlap.<jupyter_code>plot_dl_sessions_matrices()
plot_ethnicity_spec_unspec, plot_ethnicity_spec = ac.plot_ethnicity()<jupyter_output>D:\Hunter_ext\anaconda3\lib\site-packages\google\auth\_default.py:79: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK without a quota project. You might receive a "quota exceeded" or "API not enabled" error. We recommend you rerun `gcloud auth application-default login` and make sure a quota project is added. Or you can use service accounts instead. For more information about service accounts, see https://cloud.google.com/docs/authentication/
  warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)<jupyter_text>Donor ethnicity Below we display the donor ethnicity metadata labels entered for donors up to the DCP15 releasse. (April, 2022)<jupyter_code>plot_ethnicity_spec_unspec()<jupyter_output><empty_output><jupyter_text>Of donors with recorded ethnicity fields, the number of occurrences of each ethnicity category is shown below.<jupyter_code>plot_ethnicity_spec()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06613102595797281, 'fraction_numerical': 0.0064894932014833125, 'mean_word_length': 6.407322654462242, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 11, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '335547', 'license': 'Apache-2.0', 'n_tokens_mistral': 967, 'n_tokens_neox': 906, 'n_words': 332, 'path': 'analytics/hdgar-book/.ipynb_checkpoints/data-analytics-checkpoint.ipynb', 'repo_name': 'DataBiosphere/data-browser', 'size': '3236'}