<commit_before># Apache Spark for High Energy Physics
In this page you can find links and examples related to using Spark for reading and processing experiments data.

--- 
### LHCb Open Data Analysis Using PySpark
 * **Jupyter notebook on GitHub at: [LHCb_OpenData_Spark.ipynb](LHCb_OpenData_Spark.ipynb)**  
 * [CERN SWAN service](https://swan.web.cern.ch) users, run from [this link to CERN Box](https://cernbox.cern.ch/index.php/s/98RK9xIU1s9Lf08)
   
### CMS Big Data project
 * Example notebook using CMS opendata, spark-root and Hadoop-XRootD: [CMS_BigData_Opendata_Spark_Example1.ipynb](CMS_BigData_Opendata_Spark_Example1.ipynb)
---
Relevant technology and links:
 * [spark-root](https://github.com/diana-hep/spark-root): a library to read HEP files in ROOT format into Spark DataFrames.
 * [CERN Open Data portal](http://opendata.cern.ch/)
 * [CMS Big Data project](https://cms-big-data.github.io)
 * [Hadoop-XRootD connector](https://gitlab.cern.ch/awg/hadoop-xrootd-connector) - allows to read files from EOS using the XRootD protocol (currently available on CERN Gitlab, will be available on GitHub)
 <commit_msg>Minor updates<commit_after># Apache Spark for High Energy Physics
In this page you can find links and examples related to using Spark for reading and processing experiments data.

--- 
### LHCb Open Data Analysis Using PySpark
 * **Jupyter notebook on GitHub at: [LHCb_OpenData_Spark.ipynb](LHCb_OpenData_Spark.ipynb)**  
 * [CERN SWAN service](https://swan.web.cern.ch) users, run from [this link to CERN Box](https://cernbox.cern.ch/index.php/s/98RK9xIU1s9Lf08)
   
### CMS Big Data project
 * Example notebook using CMS opendata, spark-root and Hadoop-XRootD: [CMS_BigData_Opendata_Spark_Example1.ipynb](CMS_BigData_Opendata_Spark_Example1.ipynb)
---
Relevant technology and links:
 * [spark-root](https://github.com/diana-hep/spark-root): a library to read HEP files in ROOT format into Spark DataFrames.
 * [CERN Open Data portal](http://opendata.cern.ch/)
 * [CMS Big Data project](https://cms-big-data.github.io)
 * [LHCb Open Data project](https://github.com/lhcb/opendata-project)
 * [Hadoop-XRootD connector](https://gitlab.cern.ch/awg/hadoop-xrootd-connector) - allows to read files from EOS using the XRootD protocol (currently available on CERN Gitlab, will be available on GitHub)
 
{'fraction_non_alphanumeric': 0.1119724375538329, 'fraction_numerical': 0.007751937984496124, 'mean_word_length': 4.8075, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 11, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '659316', 'n_tokens_mistral': 826, 'n_tokens_neox': 760, 'n_words': 223}