<commit_before>
.. blogpost::
    :title: Quelques articles intéressants
    :keywords: deep learning
    :date: 2022-07-17
    :categories: articles

    Deep Learning

    * `Training Deep Networks without Learning Rates Through Coin Betting
      <https://arxiv.org/abs/1705.07795>`_
    * `Model Comparison and Calibration Assessment: User Guide for 
      Consistent Scoring Functions in Machine Learning and Actuarial Practice
      <https://arxiv.org/abs/2202.12780>`_
    * `On Embeddings for Numerical Features in Tabular Deep Learning
      <https://arxiv.org/abs/2203.05556>`,
      `TABULAR DATA MODELING VIA CONTEXTUAL EMBEDDINGS
      <https://weasul.github.io/papers/7.pdf>`_
    * `MLGO: a Machine Learning Guided Compiler Optimizations Framework
      <https://arxiv.org/abs/2101.04808>`_
    * `Embedding Recycling for Language Models
      <https://arxiv.org/abs/2207.04993>`_
    * `Language Modelling with Pixels
      <https://arxiv.org/abs/2207.06991>`_
    * `XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
      <https://paperswithcode.com/paper/xls-r-self-supervised-cross-lingual-speech>`_
    * `Emerging Cross-lingual Structure in Pretrained Language Models
      <https://arxiv.org/abs/1911.01464>`_
    * `Word Translation Without Parallel Data
      <https://arxiv.org/abs/1710.04087>`_
    * `Very Deep Convolutional Networks for Text Classification
      <https://arxiv.org/abs/1606.01781>`_
    * `XTREME-S: Evaluating Cross-lingual Speech Representations
      <https://arxiv.org/abs/2203.10752>`_
    * `DeepSpeed Compression: A composable library for extreme compression and zero-cost quantization
      <https://www.microsoft.com/en-us/research/blog/deepspeed-compression-a-composable-library-for-extreme-compression-and-zero-cost-quantization/>`_

    Machine Learning

    * `On the consistency of supervised learning with missing values
      <https://arxiv.org/abs/1902.06931>`_
    * `Why do tree-based models still outperform deep learning on tabular data?
      <https://hal.archives-ouvertes.fr/hal-03723551>`_

    Ecologie

    * `Measuring the Carbon Intensity of AI in Cloud Instances
      <https://arxiv.org/abs/2206.05229>`_

    Un site

    * `paperwithcode <https://paperswithcode.com/>`_

    Librairies
    
    * `TorchArrow <https://github.com/pytorch/torcharrow>`_

    Un benchmark

    * `MIXED PRECISION TRAINING IN PRACTICE<commit_msg>Update 2022-07-17_library.rst<commit_after>
.. blogpost::
    :title: Quelques articles intéressants
    :keywords: deep learning
    :date: 2022-07-17
    :categories: articles

    Deep Learning

    * `Training Deep Networks without Learning Rates Through Coin Betting
      <https://arxiv.org/abs/1705.07795>`_
    * `Model Comparison and Calibration Assessment: User Guide for 
      Consistent Scoring Functions in Machine Learning and Actuarial Practice
      <https://arxiv.org/abs/2202.12780>`_
    * `On Embeddings for Numerical Features in Tabular Deep Learning
      <https://arxiv.org/abs/2203.05556>`,
      `TABULAR DATA MODELING VIA CONTEXTUAL EMBEDDINGS
      <https://weasul.github.io/papers/7.pdf>`_
    * `MLGO: a Machine Learning Guided Compiler Optimizations Framework
      <https://arxiv.org/abs/2101.04808>`_
    * `Embedding Recycling for Language Models
      <https://arxiv.org/abs/2207.04993>`_
    * `Language Modelling with Pixels
      <https://arxiv.org/abs/2207.06991>`_
    * `XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
      <https://paperswithcode.com/paper/xls-r-self-supervised-cross-lingual-speech>`_
    * `Emerging Cross-lingual Structure in Pretrained Language Models
      <https://arxiv.org/abs/1911.01464>`_
    * `Word Translation Without Parallel Data
      <https://arxiv.org/abs/1710.04087>`_
    * `Very Deep Convolutional Networks for Text Classification
      <https://arxiv.org/abs/1606.01781>`_
    * `XTREME-S: Evaluating Cross-lingual Speech Representations
      <https://arxiv.org/abs/2203.10752>`_
    * `DeepSpeed Compression: A composable library for extreme compression and zero-cost quantization
      <https://www.microsoft.com/en-us/research/blog/deepspeed-compression-a-composable-library-for-extreme-compression-and-zero-cost-quantization/>`_

    Machine Learning

    * `On the consistency of supervised learning with missing values
      <https://arxiv.org/abs/1902.06931>`_
    * `Why do tree-based models still outperform deep learning on tabular data?
      <https://hal.archives-ouvertes.fr/hal-03723551>`_
    * `Evaluating machine learning models and their diagnostic value
      <https://hal.inria.fr/hal-03682454/>`_

    Ecologie

    * `Measuring the Carbon Intensity of AI in Cloud Instances
      <https://arxiv.org/abs/2206.05229>`_

    Un site

    * `paperwithcode <https://paperswithcode.com/>`_

    Librairies
    
    * `TorchArrow <https://github.com/pytorch/torcharrow>`_

    Un benchmark

    * `MIXED PRECISION TRAINING IN PRACTICE
{'fraction_non_alphanumeric': 0.11057692307692307, 'fraction_numerical': 0.05328525641025641, 'mean_word_length': 3.245748299319728, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 37, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2887147', 'n_tokens_mistral': 1870, 'n_tokens_neox': 1536, 'n_words': 365}