<commit_before>Title: Activities and Research Products
date: 2016-05-10 10:53
slug: activities.html
Authors: <NAME>
Summary: DIANA Activities


### Software products

   *  [<i class="fa fa-github-square fa-lg"></i> DIANA/HEP organization on GitHub](https://github.com/diana-hep)

   * Carl: a toolbox for likelihood-free inference ([documentation](http://diana-hep.org/carl), [code](https://github.com/diana-hep/carl)) [![DOI](https://zenodo.org/badge/doi/10.5281/zenodo.47798.svg)](http://dx.doi.org/10.5281/zenodo.47798)

   * Histogrammar: a multilingual specification for histogram aggregation with implementations in Python and Scala - [docs](http://histogrammar.org), [GitHub](https://github.com/histogrammar)

   * uproot: pure Python implementation of ROOT I/O - [GitHub](https://github.com/scikit-hep/uproot)

   * OAMap: tools for handling hierarchically nested data as arrays with an Object Array Map - [GitHub](https://github.com/diana-hep/oamap)

   * Scikit-HEP: a community-driven Python project for HEP core and common tools ([web page](http://scikit-hep.org/), [GitHub](https://github.com/scikit-hep)) [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.1043950.svg)](http://doi.org/10.5281/zenodo.1043950)

   * Scikit-Optimize: a library for sequential model-based optimization ([documentation](https://scikit-optimize.github.io/), [code](https://github.com/scikit-optimize/scikit-optimize))

   * yadage: a language and engine for flexible, distributed workflows - [docs](http://yadage.readthedocs.io) [code](https://github.com/diana-hep/yadage)

   * REANA: a system that permits to instantiate research data analyses on the cloud. [docs]( https://reana.readthedocs.io/en/latest/?badge=latest) [code](https://github.com/reanahub)


### Papers / Documents

   * [HEP Community White Paper on Software trigger and event reconstruction: Executive Summary](https://arxiv.org/abs/1802.08640) - <NAME>, <NAME>, <NAME> *in* Johannes Albrecht *et al*.
   [arXiv:1802.08640](https://arxiv.org/abs/1802.08640)

   * [HEP Community White Paper on Software trigger and event reconstruction](https://arxiv.org/abs/1802.08638) - <NAME>, <NAME>, <NAME> *in* Johannes Albrecht *et al*.
   [arXiv:1802.08638](https://arxiv.org/abs/1802.08638)

   * [A Roadmap for HEP Software and Computing R&D for the  2020s](https://arxiv.org/abs/1712.06982) - <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME> *in* <NAME> Jr *et al*.
   [arXiv:1712.06982](https://arxiv.org/abs/1712.06982)

   * [Optimizing ROOT IO For Analysis](https://arxiv.org/abs/1711.02659) - <NAME>, <NAME>, <NAME>, submitted to Proceedings of [18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT 2017)](https://indico.cern.ch/event/567550/) (Seattle) [arXiv:1711.02659](https://arxiv.org/abs/1711.02659)

   * [Toward real-time data query systems in HEP](https://arxiv.org/abs/1711.01229) - <NAME>, <NAME>, <NAME>, submitted to Proceedings of [18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT 2017)](https://indico.cern.ch/event/567550/) (Seattle) [arXiv:1711.01229](https://arxiv.org/abs/1711.01229)

   * [CMS Analysis and Data Reduction with Apache Spark](https://arxiv.org/abs/1711.00375) - <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, submitted to Proceedings of [18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT 2017)](https://indico.cern.ch/event/567550/) (Seattle) [arXiv:1708.08319](https://arxiv.org/abs/1711.00375)

   * [Fast Access to Columnar, Hierarchical Data via Code Transformation](https://arxiv.org/abs/1708.08319) - <NAME>, <NAME>, <NAME>, and <NAME>. Submitted to [IEEE Big Data 2017](http://cci.drexel.edu/bigdata/bigdata2017/index.html) (Boston) [arXiv:1708.08319](https://arxiv.org/abs/1708.08319)

   * [Adversarial Variational Optimization of Non-Differentiable Simulators](https://arxiv.org/abs/1707.07113) - <NAME> and <NAME> [arXiv:1703.07113](https://arxiv.org/abs/1707.07113) | [source and code](https://github.com/glouppe/paper-avo)

   * [HEPData: a repository for high energy physics data](http://iopscience.iop.org/article/10.1088/1742-6596/898/10/102006/meta) - <NAME>, <NAME>, <NAME>. Proceedings of the [22nd International Conference on Computing in High Energy and Nuclear Physics (CHEP 2016)](http://chep2016.org/) (San Francisco) [arXiv:1704.05473](http://arxiv.org/abs/arXiv:1704.05473)

   * [QCD-Aware Recursive Neural Networks for Jet Physics](https://arxiv.org/abs/1702.00748) - G.Louppe, K.Cho, C.Becot, K.Cranmer [arXiv:1702.00748](https://arxiv.org/abs/1702.00748)

   * [Exploring compression techniques for ROOT IO](http://iopscience.iop.org/article/10.1088/1742-6596/898/7/072043/meta) - Z.Zhang, B.Bockelman; Proceedings of the [22nd International Conference on Computing in High Energy and Nuclear Physics (CHEP 2016)](http://chep2016.org/) (San Francisco) [arXiv:1704.06976](https://arxiv.org/abs/1704.06976)

   * [Big Data in HEP: A comprehensive use case study](http://iopscience.iop.org/article/10.1088/1742-6596/898/7/072012/meta) - O.Gutsche, M.Cremonesi, P.Elmer, B.Jayatilaka, J.Kowalkowski, J.Pivarski, S.Sehrish, <NAME>, A.Svyatkovskiy, N.Tran; Proceedings of the [22nd International Conference on Computing in High Energy and Nuclear Physics (CHEP 2016)](http://chep2016.org/) (San Francisco) [arXiv:1703.04171](https://arxiv.org/abs/1703.04171)

   * [Analysis Preservation and Reinterpretation](https://indico.cern.ch/event/505613/contributions/2227701/), <NAME> Proceedings of the [22nd International Conference on Computing in High Energy and Nuclear Physics (CHEP 2016)](http://chep2016.org/) (San Francisco)

   * [Learning to Pivot with Adversarial Networks](http://arxiv.org/abs/arXiv:1611.01046) - G.Louppe, M.Kagan, K.Cranmer [arXiv:1611.01046](https://arxiv.org/abs/1611.01046)

   * Carl in the Journal of Open Source Software - G.Louppe, K.Cranmer, J.Pavez; [![status](http://joss.theoj.org/papers/26a9ffd9e7b98b1911d89d2ceb268f37/status.svg)](http://joss.theoj.org/papers/26a9ffd9e7b98b1911d89d2ceb268f37)

   * [Approximating Likelihood Ratios with Calibrated Discriminative Classifiers](https://arxiv.org/abs/1506.02169) - K.Cranmer, J.Pavez, G.Louppe [arXiv:1506.02169](https://arxiv.org/abs/1506.02169)

   * [Visualization of Publication Impact](http://arxiv.org/abs/arXiv:1605.06242) - E.Maguire, J.Montull, G.Louppe; Proceedings of Eurographics Conference on Visualization (EuroVis) 2016 (Groningen) [arXiv:1605.06242](https://arxiv.org/abs/1605.06242)

   * NSF Proposal - [Data-Intensive Analysis for High Energy Physics (DIANA/HEP)](http://diana-hep.org/downloads/ssi2014-nsf-proposal.pdf)


### Google Summer of Code program

Participation in the GSoC 2017 program.

  * Project [Efficient Python routines for analysis on massively multi-threaded platforms - Python bindings for the Hydra C++ library](https://summerofcode.withgoogle.com/archive/2017/projects/6217497468469248/) - <NAME>, mentor for this [CERN-HSF organisation](http://hepsoftwarefoundation.org/) project.


### DIANA Topical Meetings

  <!-- this is the atom feed URL https://indico.cern.ch/category/7192/events.atom -->

  The DIANA team organizes periodic meetings on various topics of interest, with both invited external speakers and DIANA team contributions. These and other DIANA meetings can be found in the [DIANA-HEP Organization in the INDICO agenda system](https://indico.cern.ch/category/7192/). Videos of these meetings are published in [YouTube](https://www.youtube.com/channel/UCzFnto-EbPAfERLr0AZb58g).

### Presentations by the DIANA team

  * 28 Feb 2018 - [Uproot: rapidly moving data from ROOT to Numpy and Pandas](https://indico.cern.ch/event/686641/contributions/2894906/attachments/1606247/2548596/pivarski-uproot.pdf), <NAME>, [IML Meeting](https://indico.cern.ch/event/686641/)

  * 26 Feb 2018 - [APPAVO - Augmented tools for Particle Physics Analysis, Visualisation and Outreach](https://indico.cern.ch/event/702570/contributions/2905377/attachments/1606585/2549393/2018-02-26_CERNSoftRandD_EduardoRodrigues.pdf), <NAME>, [CERN EP Department Software R&D Second Lightning Talks Session](https://indico.cern.ch/event/702570/)

  * 21 Feb 2018 - [Enhance TTreeCache defaults](https://indico.fnal.gov/event/16264/contribution/5/material/slides/0.pdf), <NAME>, [ROOT I/O Workshop, Early Spring 2018](https://indico.fnal.gov/event/16264/)

  * 21 Feb 2018 - [LZ4, ZStd updates](https://indico.fnal.gov/event/16264/contribution/8/material/slides/0.pdf), <NAME>, [ROOT I/O Workshop, Early Spring 2018](https://indico.fnal.gov/event/16264/)

  * 21 Feb 2018 - [Parquet Data Format Performance](https://indico.fnal.gov/event/16264/contribution/1/material/slides/0.pdf), <NAME>, [ROOT I/O Workshop, Early Spring 2018](https://indico.fnal.gov/event/16264/)

  * 12 Feb 2018 - [GooFit Python bindings](https://indico.cern.ch/event/699424/contributions/2868678/attachments/1599216/2534704/Himadri.pdf), <NAME>, [DIANA Meeting](https://indico.cern.ch/event/699424/)

  * 5 Feb 2018 - [Activities in Data Analysis](https://indico.cern.ch/event/702591/contributions/2881721/attachments/1594686/2525180/pivarski-data-analysis.pdf), <NAME>, [CMS Offline/Computing R&D](https://indico.cern.ch/event/702591/)
<commit_msg>Add relevant presentations by Jim and Eduardo at WLCG/HSF meeting<commit_after>Title: Activities and Research Products
date: 2016-05-10 10:53
slug: activities.html
Authors: <NAME>
Summary: DIANA Activities


### Software products

   *  [<i class="fa fa-github-square fa-lg"></i> DIANA/HEP organization on GitHub](https://github.com/diana-hep)

   * Carl: a toolbox for likelihood-free inference ([documentation](http://diana-hep.org/carl), [code](https://github.com/diana-hep/carl)) [![DOI](https://zenodo.org/badge/doi/10.5281/zenodo.47798.svg)](http://dx.doi.org/10.5281/zenodo.47798)

   * Histogrammar: a multilingual specification for histogram aggregation with implementations in Python and Scala - [docs](http://histogrammar.org), [GitHub](https://github.com/histogrammar)

   * uproot: pure Python implementation of ROOT I/O - [GitHub](https://github.com/scikit-hep/uproot)

   * OAMap: tools for handling hierarchically nested data as arrays with an Object Array Map - [GitHub](https://github.com/diana-hep/oamap)

   * Scikit-HEP: a community-driven Python project for HEP core and common tools ([web page](http://scikit-hep.org/), [GitHub](https://github.com/scikit-hep)) [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.1043950.svg)](http://doi.org/10.5281/zenodo.1043950)

   * Scikit-Optimize: a library for sequential model-based optimization ([documentation](https://scikit-optimize.github.io/), [code](https://github.com/scikit-optimize/scikit-optimize))

   * yadage: a language and engine for flexible, distributed workflows - [docs](http://yadage.readthedocs.io) [code](https://github.com/diana-hep/yadage)

   * REANA: a system that permits to instantiate research data analyses on the cloud. [docs]( https://reana.readthedocs.io/en/latest/?badge=latest) [code](https://github.com/reanahub)


### Papers / Documents

   * [HEP Community White Paper on Software trigger and event reconstruction: Executive Summary](https://arxiv.org/abs/1802.08640) - <NAME>, <NAME>, <NAME> *in* Johannes Albrecht *et al*.
   [arXiv:1802.08640](https://arxiv.org/abs/1802.08640)

   * [HEP Community White Paper on Software trigger and event reconstruction](https://arxiv.org/abs/1802.08638) - <NAME>, <NAME>, <NAME> *in* Johannes Albrecht *et al*.
   [arXiv:1802.08638](https://arxiv.org/abs/1802.08638)

   * [A Roadmap for HEP Software and Computing R&D for the  2020s](https://arxiv.org/abs/1712.06982) - <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME> *in* Ant<NAME> Jr *et al*.
   [arXiv:1712.06982](https://arxiv.org/abs/1712.06982)

   * [Optimizing ROOT IO For Analysis](https://arxiv.org/abs/1711.02659) - <NAME>, <NAME>, <NAME>, submitted to Proceedings of [18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT 2017)](https://indico.cern.ch/event/567550/) (Seattle) [arXiv:1711.02659](https://arxiv.org/abs/1711.02659)

   * [Toward real-time data query systems in HEP](https://arxiv.org/abs/1711.01229) - <NAME>, <NAME>, <NAME>, submitted to Proceedings of [18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT 2017)](https://indico.cern.ch/event/567550/) (Seattle) [arXiv:1711.01229](https://arxiv.org/abs/1711.01229)

   * [CMS Analysis and Data Reduction with Apache Spark](https://arxiv.org/abs/1711.00375) - <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, submitted to Proceedings of [18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT 2017)](https://indico.cern.ch/event/567550/) (Seattle) [arXiv:1708.08319](https://arxiv.org/abs/1711.00375)

   * [Fast Access to Columnar, Hierarchical Data via Code Transformation](https://arxiv.org/abs/1708.08319) - <NAME>, <NAME>, <NAME>, and <NAME>. Submitted to [IEEE Big Data 2017](http://cci.drexel.edu/bigdata/bigdata2017/index.html) (Boston) [arXiv:1708.08319](https://arxiv.org/abs/1708.08319)

   * [Adversarial Variational Optimization of Non-Differentiable Simulators](https://arxiv.org/abs/1707.07113) - <NAME> and <NAME> [arXiv:1703.07113](https://arxiv.org/abs/1707.07113) | [source and code](https://github.com/glouppe/paper-avo)

   * [HEPData: a repository for high energy physics data](http://iopscience.iop.org/article/10.1088/1742-6596/898/10/102006/meta) - <NAME>, <NAME>, <NAME>. Proceedings of the [22nd International Conference on Computing in High Energy and Nuclear Physics (CHEP 2016)](http://chep2016.org/) (San Francisco) [arXiv:1704.05473](http://arxiv.org/abs/arXiv:1704.05473)

   * [QCD-Aware Recursive Neural Networks for Jet Physics](https://arxiv.org/abs/1702.00748) - G.Louppe, K.Cho, C.Becot, K.Cranmer [arXiv:1702.00748](https://arxiv.org/abs/1702.00748)

   * [Exploring compression techniques for ROOT IO](http://iopscience.iop.org/article/10.1088/1742-6596/898/7/072043/meta) - Z.Zhang, B.Bockelman; Proceedings of the [22nd International Conference on Computing in High Energy and Nuclear Physics (CHEP 2016)](http://chep2016.org/) (San Francisco) [arXiv:1704.06976](https://arxiv.org/abs/1704.06976)

   * [Big Data in HEP: A comprehensive use case study](http://iopscience.iop.org/article/10.1088/1742-6596/898/7/072012/meta) - O.Gutsche, M.Cremonesi, P.Elmer, B.Jayatilaka, J.Kowalkowski, J.Pivarski, S.Sehrish, <NAME>, A.Svyatkovskiy, N.Tran; Proceedings of the [22nd International Conference on Computing in High Energy and Nuclear Physics (CHEP 2016)](http://chep2016.org/) (San Francisco) [arXiv:1703.04171](https://arxiv.org/abs/1703.04171)

   * [Analysis Preservation and Reinterpretation](https://indico.cern.ch/event/505613/contributions/2227701/), <NAME>rich Proceedings of the [22nd International Conference on Computing in High Energy and Nuclear Physics (CHEP 2016)](http://chep2016.org/) (San Francisco)

   * [Learning to Pivot with Adversarial Networks](http://arxiv.org/abs/arXiv:1611.01046) - G.Louppe, M.Kagan, K.Cranmer [arXiv:1611.01046](https://arxiv.org/abs/1611.01046)

   * Carl in the Journal of Open Source Software - G.Louppe, K.Cranmer, J.Pavez; [![status](http://joss.theoj.org/papers/26a9ffd9e7b98b1911d89d2ceb268f37/status.svg)](http://joss.theoj.org/papers/26a9ffd9e7b98b1911d89d2ceb268f37)

   * [Approximating Likelihood Ratios with Calibrated Discriminative Classifiers](https://arxiv.org/abs/1506.02169) - K.Cranmer, J.Pavez, G.Louppe [arXiv:1506.02169](https://arxiv.org/abs/1506.02169)

   * [Visualization of Publication Impact](http://arxiv.org/abs/arXiv:1605.06242) - E.Maguire, J.Montull, G.Louppe; Proceedings of Eurographics Conference on Visualization (EuroVis) 2016 (Groningen) [arXiv:1605.06242](https://arxiv.org/abs/1605.06242)

   * NSF Proposal - [Data-Intensive Analysis for High Energy Physics (DIANA/HEP)](http://diana-hep.org/downloads/ssi2014-nsf-proposal.pdf)


### Google Summer of Code program

Participation in the GSoC 2017 program.

  * Project [Efficient Python routines for analysis on massively multi-threaded platforms - Python bindings for the Hydra C++ library](https://summerofcode.withgoogle.com/archive/2017/projects/6217497468469248/) - <NAME>ues, mentor for this [CERN-HSF organisation](http://hepsoftwarefoundation.org/) project.


### DIANA Topical Meetings

  <!-- this is the atom feed URL https://indico.cern.ch/category/7192/events.atom -->

  The DIANA team organizes periodic meetings on various topics of interest, with both invited external speakers and DIANA team contributions. These and other DIANA meetings can be found in the [DIANA-HEP Organization in the INDICO agenda system](https://indico.cern.ch/category/7192/). Videos of these meetings are published in [YouTube](https://www.youtube.com/channel/UCzFnto-EbPAfERLr0AZb58g).

### Presentations by the DIANA team

  * 29 Mar 2018 - [Summary of Programming for Concurrency and Co-Processors](https://indico.cern.ch/event/658060/contributions/2940562/attachments/1624592/2586582/pivarski-concurrency-summary.pdf), <NAME>, [Joint WLCG/HSF Workshop 2018](https://indico.cern.ch/event/658060/)

  * 29 Mar 2018 - [Summary of Analysis Facilities and Use Cases](https://indico.cern.ch/event/658060/contributions/2940565/attachments/1625105/2587664/2018-03-29_WLCG-HSFWorkshopNaples.pdf), <NAME>, [Joint WLCG/HSF Workshop 2018](https://indico.cern.ch/event/658060/)

  * 28 Mar 2018 - [Overview of Serialization Technologies](https://indico.cern.ch/event/658060/contributions/2898569/attachments/1622526/2582399/pivarski-serialization.pdf), <NAME>, [Joint WLCG/HSF Workshop 2018](https://indico.cern.ch/event/658060/)

  * 28 Mar 2018 - [Becoming Mainstream](https://indico.cern.ch/event/658060/contributions/2918129/attachments/1624548/2586477/2018-03-28_WLCG-HSFWorkshopNaples.pdf), <NAME>, [Joint WLCG/HSF Workshop 2018](https://indico.cern.ch/event/658060/)

  * 27 Mar 2018 - [Spark-like and query-like analysis systems and tools](https://indico.cern.ch/event/658060/contributions/2889117/attachments/1621961/2581100/pivarski-querylike.pdf), <NAME>, [Joint WLCG/HSF Workshop 2018](https://indico.cern.ch/event/658060/)

  * 26 Mar 2018 - [CWP Roadmap](https://indico.cern.ch/event/658060/contributions/2876976/attachments/1622751/2582923/2018-03-26_WLCG-HSFWorkshopNaples.pdf), <NAME>, [Joint WLCG/HSF Workshop 2018](https://indico.cern.ch/event/658060/)

  * 28 Feb 2018 - [Uproot: rapidly moving data from ROOT to Numpy and Pandas](https://indico.cern.ch/event/686641/contributions/2894906/attachments/1606247/2548596/pivarski-uproot.pdf), <NAME>, [IML Meeting](https://indico.cern.ch/event/686641/)

  * 26 Feb 2018 - [APPAVO - Augmented tools for Particle Physics Analysis, Visualisation and Outreach](https://indico.cern.ch/event/702570/contributions/2905377/attachments/1606585/2549393/2018-02-26_CERNSoftRandD_EduardoRodrigues.pdf), <NAME>, [CERN EP Department Software R&D Second Lightning Talks Session](https://indico.cern.ch/event/702570/)

  * 21 Feb 2018 - [Enhance TTreeCache defaults](https://indico.fnal.gov/event/16264/contribution/5/material/slides/0.pdf), <NAME>, [ROOT I/O Workshop, Early Spring 2018](https://indico.fnal.gov/event/16264/)

  * 21 Feb 2018 - [LZ4, ZStd updates](https://indico.fnal.gov/event/16264/contribution/8/material/slides/0.pdf), <NAME>, [ROOT I/O Workshop, Early Spring 2018](https://indico.fnal.gov/event/16264/)

  * 21 Feb 2018 - [Parquet Data Format Performance](https://indico.fnal.gov/event/16264/contribution/1/material/slides/0.pdf), <NAME>, [ROOT I/O Workshop, Early Spring 2018](https://indico.fnal.gov/event/16264/)

  * 12 Feb 2018 - [GooFit Python bindings](https://indico.cern.ch/event/699424/contributions/2868678/attachments/1599216/2534704/Himadri.pdf), H<NAME>, [DIANA Meeting](https://indico.cern.ch/event/699424/)

  * 5 Feb 2018 - [Activities in Data Analysis](https://indico.cern.ch/event/702591/contributions/2881721/attachments/1594686/2525180/pivarski-data-analysis.pdf), <NAME>, [CMS Offline/Computing R&D](https://indico.cern.ch/event/702591/)

{'fraction_non_alphanumeric': 0.15447673277046928, 'fraction_numerical': 0.10445709797761633, 'mean_word_length': 4.7713881019830024, 'pattern_counts': {'":': 0, '<': 125, '<?xml version=': 0, '>': 125, 'https://': 132, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '330163', 'n_tokens_mistral': 8821, 'n_tokens_neox': 6847, 'n_words': 1502}