<commit_before>% ROOT Version 6.14 Release Notes
% 2017-11-19

<a name="TopOfPage"></a>

## Introduction

ROOT version 6.14/00 is scheduled for release in 2018.

For more information, see:

[http://root.cern.ch](http://root.cern.ch)

The following people have contributed to this new version:

 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, UNL,\
 <NAME>, CERN/SFT,\
 <NAME>, FNAL,\
 <NAME>, ANL (SULI),\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, GSI,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, ANL,\
 <NAME>, Princeton/CMS,\
 <NAME>, UNL,\
 <NAME>, NIKHEF/Atlas, RooFit

## Removed interfaces

## Core Libraries
   - Optimize away redundant deserialization of template specializations. This reduces the memory footprint for hsimple by around 30% while improving the runtime performance for various cases by around 15%.
   - When ROOT is signaled with a SIGUSR2 (i.e. on Linux and MacOS X) it will now print a backtrace.
   - Move RStringView.h to ROOT/RStringView.hxx and always include ROOT/RStringView.hxx instead of RStringView.h for backward compatibility
   - In `TClingCallFunc`, support r-value reference parameters. This paves the way for the corresponding support in PyROOT (implemented now in the latest Cppyy).

## I/O Libraries
   - LZ4 (with compression level 4) is now the default compression algorithm for new ROOT files (LZ4 is lossless data compression algorithm that is focused on compression and decompression speed, while in ROOT case providing benefit in faster decompression at the price of a bit worse compression ratio comparing to ZLIB)
   - Implement reading of objects data from JSON
   - Provide TBufferJSON::ToJSON() and TBufferJSON::FromJSON() methods
   - Provide TBufferXML::ToXML() and TBufferXML::FromXML() methods

## TTree Libraries
   - Proxies are now properly re-used when multiple TTreeReader{Value,Array}s are associated to a single branch. Deserialisation is therefore performed once. This is an advantage for complex TDataFrame graphs.
   - Add TBranch::BackFill to allowing the addition of new branches to an existing tree and keep the new basket clustered in the same way as the rest of the TTree.  Use with the following pattern,
   make sure to to call BackFill for the same entry for all the branches consecutively:
```
  for(auto e = 0; e < tree->GetEntries(); ++e) { // loop over entries.
    for(auto branch : branchCollection) {
      ... Make change to the data associated with the branch ...
      branch->BackFill();
    }
  }
```
Since we loop over all the branches for each new entry all the baskets for a cluster are consecutive in the file.

### TDataFrame
   - Histograms and profiles returned by TDataFrame (e.g. by a Histo1D action) are now not associated to a ROOT directory (their fDirectory is a nullptr).
     The following still works as expected:
     ```
     auto h = tdf.Histo1D("x");
     TFile f(fname, "RECREATE");
     h->Write(); // event loop is run here and h is written to the TFile f
     ```

#### New features
   - The TDataSource interface changed. The `TDataSource::SetEntry` method now returns a boolean. If true the entry is processed within the event loop managed by the tdf, skipped otherwise.
   - The TLazyDS data source has been added. It allows to create a source starting from ResultProxies to vectors.
   - `TDataFrameInterface<T>::Report` returns a `TCutflowReport` object which can be inspected programmatically.
   - Add `Aggregate` action and implement `Reduce` in terms of it.
   - Add support for a more general leafname syntax such as "myBranch.myLeaf" - not available in jitted strings but only usable in lists of columns names.
   - Add the `ROOT::Experimental::TAdoptAllocator<T>`, an allocator which allows to adopt existing memory. If memory is adopted, upon allocation a copy is performed in the new, potentially more extended, memory region.
   - Add `ROOT::Experimental::VecOps::TVec<T>` a class which represents a contiguous array, inspired by Numpy arrays. `TVec` offer a convenient interface, almost identical to the one of `std::vector`. It can own or adopt its memory. As well as a set of tools which make analysis of collections easier, avoiding to loop over the individual elements of the collections. Basic arithmetic operations such as +,-,*,/,% between TVecs and scalars and TVecs are supported. Most popular math functions which act on TVecs are provided. Helpers to calculate basic quantities such as sum, mean, variance or standard deviation of TVecs are provided.
   A powerful and concise syntax for expressing cuts is available:
```
  // mu_pts_tvec and mu_etas_tvec are two equally sized TVecs holding kinematic properties of muons
  // a filter on muons pseudorapidities is applied considering a range in pseudo rapidity.
  filtered_mu_pts_tvec = mu_pts_tvec[abs(mu_etas_tvec) < 2)];
```
   - The `TArrayBranch` class has been removed and replaced by the more powerful `TVec`.
   - Columns on disk stored as C arrays should be read as `TVec`s, `std::vector` columns can be read as `TVec`s if requested. Jitted transformations and actions consider `std::vector` columns as well as C array columns `TVec`s.
   - In jitted transformations and actions, `std::vector` and C array columns are read as `TVec`s.
   - When snapshotting, columns read from trees which are of type `std::vector` or C array and read as TVecs are persistified on disk as a `std::vector` or C arrays respectively - no transformation happens. `TVec` columns, for example coming from `Define`s, are written as `std::vector<T, TAdoptAllocator<T>>`.

#### Fixes
   - Do not alphabetically order columns before snapshotting to avoid issues when writing C arrays the size of which varies and is stored in a separate branch.
   - Validate columns before writing datasets on disk.
   - Check the type of the columns via type info in CSV, ROOT and trivial data source.
   - Allow to snapshot a dataset read from a `TCsvDS`.
   - Snapshot and Cache now properly trigger column definitions.
   - Correctly deduce type of Float_t branches when jitting.
   - Do not rely on branches' titles for runtime type inference.
   - Do not loose an entry when using Range and multiple actions.

#### Other changes
   - Throw an exception if the type of a branch cannot be deduced.


## Histogram Libraries
   - Per object statsoverflow flag has been added. This change is required to prevent non reproducible behaviours in a multithreaded environments. For example, if several threads change the `TH1::fgStatOverflows` flag and fill histograms, the behaviour will be undefined.

## Math Libraries

## RooFit Libraries

## 2D Graphics Libraries
   - `TMultiGraph::GetHistogram` now works even if the multigraph is not drawn. Make sure
     it never returns a null pointer.
   - X11 line `width = 0` doesn't work on OpenSuSE Thumbleweed for non solid lines. Now fixed.
   - TCanvas::SetWindowsSize has been changed to get the same window size in interactive mode…and batch mode.
   - Change the `TGraph` default fill color to white to avoid black box in legend
     when `gPad->BuildLegend()` is called.
   - Auto-coloring for TF1 (drawing options PFC, PLC and PMC) is implemented.
   - Auto-coloring for TH1::DrawCopy (drawing options PFC, PLC and PMC) is implemented.
   - Improve the option management in `TF1::Draw` to allow to combine the option
     `SAME` with other drawing options.
   - `TGraph::Draw("AB")` was malfunctioning when using `TAxis::SetRangeUser`.
      It was reported [here](https://sft.its.cern.ch/jira/browse/ROOT-9144).
   - The errors end-caps size in `TLegend` follows the value set by `gStyle->SetEndErrorSize()`.
     For instance setting it to 0 allows to remove the end-caps both on the graph and the legend.
     It was requested [here](https://sft.its.cern.ch/jira/browse/ROOT-9184)
   - New color palette "cividis" implemented by <NAME>.
     This colormap aims to solve problems that people with color vision deficiency have
     with the common colormaps. For more details see:
     <NAME>, <NAME>, and <NAME>. Optimizing colormaps with consideration
     for color vision deficiency to enable accurate interpretation of scientific data.
     See the article [here](https://arxiv.org/abs/1712.01662)
   - New graphics style "ATLAS" from M.Sutton.
   - In `TGraphPainter` the fit parameters were painted too early. [In some cases graph's
     error bars overlapped the stat box](https://root-forum.cern.ch/t/hide-error-bars-behind-tpavestats/27996).
   - Implement the possibility to generate high definition bitmap pictures in `TImageDump`.
     This done via `gStyle->SetImageScaling(x);` `x` being a multiplication factor.
     This new feature is now used to generate the reference guide with `x=3`.
     Pictures in the reference guide are now much shaper and in particular the text.

## 3D Graphics Libraries
  - When a LEGO plot was drawn with Theta=90, the X and Y axis were misplaced.

## Geometry Libraries

## Database Libraries
  - Fix issue related to time stamps manipulation done by `TPgSQLStatement` as suggested [here](https://root-forum.cern.ch/t/please-correct-bug-reading-date-time-from-postgresql-tpgsqlstatement).

## Networking Libraries
   - New THttpWSHandler class should be used to work with websockets. It includes all necessary methods to handle multiple connections correctly. See in tutorials/http/ws.C how it can be used.
   - Interface of THttpWSEngine class was changed, all its instances handled internally in THttpWSHandler.

## GUI Libraries

## Montecarlo Libraries

## Parallelism
   - `TTree::GetEntry`: if IMT is enabled, run work in tasks if we have at least more than one top level branch.
   - Make EnableImplicitMT no-op if IMT is already on
   - Decompress `TTreeCache` in parallel if IMT is on (upgrade of the `TTreeCacheUnzip` class).
   - In `TTreeProcessorMT` delete friend chains after the main chain to avoid double deletes.


## Language Bindings

### Notebook integration
   - In the ROOT kernel, avoid import of unnecessary components.
   - In the ROOT kernel, optimise regexes involved in tab-completion which could take up to minutes to be executed

## JavaScript ROOT

## Code Examples

  - New graphics tutorial AtlasExample.C illustrating the ATLAS style.
  - New TLazyDS tutorial added tdf015_LazyDataSource.C.
  - Show how to inspect a `TCutFlowReport` object.

## Class Reference Guide
  - Replace low resolution images with bigger ones more suited for modern screens.

## Build, Configuration and Testing Infrastructure

The `python3` option to CMake has been removed. Python support is enabled by
default. To configure ROOT to use specific Python versions, there is a new
option called `python_version`. This is how to configure ROOT and Python for
the common use cases:

* Use the default Python interpreter:
  - `-Dpython=ON` (default)
* Search only for Python 2.x or only 3.x:
  - `-Dpython_version=2` or `-Dpython_version=3`
* Use a specific version of Python from `$PATH`:
  - `-Dpython_version=2.7` or `-Dpython_version=3.5`
* Use a specific Python interpreter, whatever the version:
  - `-DPYTHON_EXECUTABLE=/usr/local/bin/python`

Note: The use of `PYTHON_EXECUTABLE` requires the full path to the interpreter.

   - Reduce time taken by tests which takes too long to run ({One,Two}SidedFrequentistUpperLimitWithBands.C)
   - Disable PyROOT SQL tutorials (the C++ counterparts are since several releases).
<commit_msg>Release notes for TBufferJSON and JSROOT<commit_after>% ROOT Version 6.14 Release Notes
% 2017-11-19

<a name="TopOfPage"></a>

## Introduction

ROOT version 6.14/00 is scheduled for release in 2018.

For more information, see:

[http://root.cern.ch](http://root.cern.ch)

The following people have contributed to this new version:

 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, UNL,\
 <NAME>, CERN/SFT,\
 <NAME>, FNAL,\
 <NAME>, ANL (SULI),\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, GSI,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, CERN/SFT,\
 <NAME>, ANL,\
 <NAME>, Princeton/CMS,\
 <NAME>, UNL,\
 <NAME>, NIKHEF/Atlas, RooFit

## Removed interfaces

## Core Libraries
   - Optimize away redundant deserialization of template specializations. This reduces the memory footprint for hsimple by around 30% while improving the runtime performance for various cases by around 15%.
   - When ROOT is signaled with a SIGUSR2 (i.e. on Linux and MacOS X) it will now print a backtrace.
   - Move RStringView.h to ROOT/RStringView.hxx and always include ROOT/RStringView.hxx instead of RStringView.h for backward compatibility
   - In `TClingCallFunc`, support r-value reference parameters. This paves the way for the corresponding support in PyROOT (implemented now in the latest Cppyy).

## I/O Libraries
   - LZ4 (with compression level 4) is now the default compression algorithm for new ROOT files (LZ4 is lossless data compression algorithm that is focused on compression and decompression speed, while in ROOT case providing benefit in faster decompression at the price of a bit worse compression ratio comparing to ZLIB)
   - Implement reading of objects data from JSON
   - Provide TBufferJSON::ToJSON() and TBufferJSON::FromJSON() methods
   - Provide TBufferXML::ToXML() and TBufferXML::FromXML() methods
   - Converts NaN and Infinity values into null in JSON, there are no other direct equivalent

## TTree Libraries
   - Proxies are now properly re-used when multiple TTreeReader{Value,Array}s are associated to a single branch. Deserialisation is therefore performed once. This is an advantage for complex TDataFrame graphs.
   - Add TBranch::BackFill to allowing the addition of new branches to an existing tree and keep the new basket clustered in the same way as the rest of the TTree.  Use with the following pattern,
   make sure to to call BackFill for the same entry for all the branches consecutively:
```
  for(auto e = 0; e < tree->GetEntries(); ++e) { // loop over entries.
    for(auto branch : branchCollection) {
      ... Make change to the data associated with the branch ...
      branch->BackFill();
    }
  }
```
Since we loop over all the branches for each new entry all the baskets for a cluster are consecutive in the file.

### TDataFrame
   - Histograms and profiles returned by TDataFrame (e.g. by a Histo1D action) are now not associated to a ROOT directory (their fDirectory is a nullptr).
     The following still works as expected:
     ```
     auto h = tdf.Histo1D("x");
     TFile f(fname, "RECREATE");
     h->Write(); // event loop is run here and h is written to the TFile f
     ```

#### New features
   - The TDataSource interface changed. The `TDataSource::SetEntry` method now returns a boolean. If true the entry is processed within the event loop managed by the tdf, skipped otherwise.
   - The TLazyDS data source has been added. It allows to create a source starting from ResultProxies to vectors.
   - `TDataFrameInterface<T>::Report` returns a `TCutflowReport` object which can be inspected programmatically.
   - Add `Aggregate` action and implement `Reduce` in terms of it.
   - Add support for a more general leafname syntax such as "myBranch.myLeaf" - not available in jitted strings but only usable in lists of columns names.
   - Add the `ROOT::Experimental::TAdoptAllocator<T>`, an allocator which allows to adopt existing memory. If memory is adopted, upon allocation a copy is performed in the new, potentially more extended, memory region.
   - Add `ROOT::Experimental::VecOps::TVec<T>` a class which represents a contiguous array, inspired by Numpy arrays. `TVec` offer a convenient interface, almost identical to the one of `std::vector`. It can own or adopt its memory. As well as a set of tools which make analysis of collections easier, avoiding to loop over the individual elements of the collections. Basic arithmetic operations such as +,-,*,/,% between TVecs and scalars and TVecs are supported. Most popular math functions which act on TVecs are provided. Helpers to calculate basic quantities such as sum, mean, variance or standard deviation of TVecs are provided.
   A powerful and concise syntax for expressing cuts is available:
```
  // mu_pts_tvec and mu_etas_tvec are two equally sized TVecs holding kinematic properties of muons
  // a filter on muons pseudorapidities is applied considering a range in pseudo rapidity.
  filtered_mu_pts_tvec = mu_pts_tvec[abs(mu_etas_tvec) < 2)];
```
   - The `TArrayBranch` class has been removed and replaced by the more powerful `TVec`.
   - Columns on disk stored as C arrays should be read as `TVec`s, `std::vector` columns can be read as `TVec`s if requested. Jitted transformations and actions consider `std::vector` columns as well as C array columns `TVec`s.
   - In jitted transformations and actions, `std::vector` and C array columns are read as `TVec`s.
   - When snapshotting, columns read from trees which are of type `std::vector` or C array and read as TVecs are persistified on disk as a `std::vector` or C arrays respectively - no transformation happens. `TVec` columns, for example coming from `Define`s, are written as `std::vector<T, TAdoptAllocator<T>>`.

#### Fixes
   - Do not alphabetically order columns before snapshotting to avoid issues when writing C arrays the size of which varies and is stored in a separate branch.
   - Validate columns before writing datasets on disk.
   - Check the type of the columns via type info in CSV, ROOT and trivial data source.
   - Allow to snapshot a dataset read from a `TCsvDS`.
   - Snapshot and Cache now properly trigger column definitions.
   - Correctly deduce type of Float_t branches when jitting.
   - Do not rely on branches' titles for runtime type inference.
   - Do not loose an entry when using Range and multiple actions.

#### Other changes
   - Throw an exception if the type of a branch cannot be deduced.


## Histogram Libraries
   - Per object statsoverflow flag has been added. This change is required to prevent non reproducible behaviours in a multithreaded environments. For example, if several threads change the `TH1::fgStatOverflows` flag and fill histograms, the behaviour will be undefined.

## Math Libraries

## RooFit Libraries

## 2D Graphics Libraries
   - `TMultiGraph::GetHistogram` now works even if the multigraph is not drawn. Make sure
     it never returns a null pointer.
   - X11 line `width = 0` doesn't work on OpenSuSE Thumbleweed for non solid lines. Now fixed.
   - TCanvas::SetWindowsSize has been changed to get the same window size in interactive mode…and batch mode.
   - Change the `TGraph` default fill color to white to avoid black box in legend
     when `gPad->BuildLegend()` is called.
   - Auto-coloring for TF1 (drawing options PFC, PLC and PMC) is implemented.
   - Auto-coloring for TH1::DrawCopy (drawing options PFC, PLC and PMC) is implemented.
   - Improve the option management in `TF1::Draw` to allow to combine the option
     `SAME` with other drawing options.
   - `TGraph::Draw("AB")` was malfunctioning when using `TAxis::SetRangeUser`.
      It was reported [here](https://sft.its.cern.ch/jira/browse/ROOT-9144).
   - The errors end-caps size in `TLegend` follows the value set by `gStyle->SetEndErrorSize()`.
     For instance setting it to 0 allows to remove the end-caps both on the graph and the legend.
     It was requested [here](https://sft.its.cern.ch/jira/browse/ROOT-9184)
   - New color palette "cividis" implemented by <NAME>.
     This colormap aims to solve problems that people with color vision deficiency have
     with the common colormaps. For more details see:
     <NAME>, <NAME>, and <NAME>. Optimizing colormaps with consideration
     for color vision deficiency to enable accurate interpretation of scientific data.
     See the article [here](https://arxiv.org/abs/1712.01662)
   - New graphics style "ATLAS" from M.Sutton.
   - In `TGraphPainter` the fit parameters were painted too early. [In some cases graph's
     error bars overlapped the stat box](https://root-forum.cern.ch/t/hide-error-bars-behind-tpavestats/27996).
   - Implement the possibility to generate high definition bitmap pictures in `TImageDump`.
     This done via `gStyle->SetImageScaling(x);` `x` being a multiplication factor.
     This new feature is now used to generate the reference guide with `x=3`.
     Pictures in the reference guide are now much shaper and in particular the text.

## 3D Graphics Libraries
  - When a LEGO plot was drawn with Theta=90, the X and Y axis were misplaced.

## Geometry Libraries

## Database Libraries
  - Fix issue related to time stamps manipulation done by `TPgSQLStatement` as suggested [here](https://root-forum.cern.ch/t/please-correct-bug-reading-date-time-from-postgresql-tpgsqlstatement).

## Networking Libraries
   - New THttpWSHandler class should be used to work with websockets. It includes all necessary methods to handle multiple connections correctly. See in tutorials/http/ws.C how it can be used.
   - Interface of THttpWSEngine class was changed, all its instances handled internally in THttpWSHandler.

## GUI Libraries

## Montecarlo Libraries

## Parallelism
   - `TTree::GetEntry`: if IMT is enabled, run work in tasks if we have at least more than one top level branch.
   - Make EnableImplicitMT no-op if IMT is already on
   - Decompress `TTreeCache` in parallel if IMT is on (upgrade of the `TTreeCacheUnzip` class).
   - In `TTreeProcessorMT` delete friend chains after the main chain to avoid double deletes.


## Language Bindings

### Notebook integration
   - In the ROOT kernel, avoid import of unnecessary components.
   - In the ROOT kernel, optimise regexes involved in tab-completion which could take up to minutes to be executed

## JavaScript ROOT
 
Upgrade JSROOT to v5.4.1. Following new features implemented:

* New supported classes:
   - TDiamond
   - TArc
   - TCurlyLine
   - TCurlyArc
   - TCrown
* New draw options:
   - "RX" and "RY" for TGraph to reverse axis
   - "noopt" for TGraph to disable drawing optimization
   - "CPN" for TCanvas to create color palette from N last colors
   - "line" for TGraph2D
* New features:
   - support LZ4 compression
   - tooltips and zooming in TGraphPolar drawings
   - TPavesText with multiple underlying paves
   - implement all fill styles
   - draw borders for TWbox
   - draw all objects from TList/TObjArray as they appear in list of primitives
   - let enable/disable highlight of extra objects in geometry viewer
   - draw axis labels on both sides when pad.fTick[x/y] > 1
   - make drawing of TCanvas with many primitives smoother
   - add fOptTitle, fOptLogx/y/z fields in JSROOT.gStyle
* Behavior changes:
   - disable automatic frame adjustment, can be enabled with "&adjframe" parameter in URL
   - when drawing TH2/TH3 scatter plots, always generate same "random" pattern
   - use barwidth/baroffset parameters in lego plots
* Bug fixes:
   - use same number of points to draw lines and markers on the TGraph
   - correctly draw filled TArrow endings
   - let combine "L" or "C" TGraph draw option with others
   - correct positioning of custom axis labels
   - correctly toggle lin/log axes in lego plot
   - let correctly change marker attributes interactively 
   - monitoring mode in draw.htm page
   - zooming in colz palette
   - support both 9.x and 10.x jsdom version in Node.js (#149)
   - draw axis main line with appropriate attributes (#150)
   - use axis color when drawing grids lines (#150)
   - when set pad logx/logy, reset existing user ranges in pad
   - avoid too deep calling stack when drawing many graphs or histos (#154)
   - correctly (re)draw tooltips on canvas with many subpads


## Code Examples

  - New graphics tutorial AtlasExample.C illustrating the ATLAS style.
  - New TLazyDS tutorial added tdf015_LazyDataSource.C.
  - Show how to inspect a `TCutFlowReport` object.

## Class Reference Guide
  - Replace low resolution images with bigger ones more suited for modern screens.

## Build, Configuration and Testing Infrastructure

The `python3` option to CMake has been removed. Python support is enabled by
default. To configure ROOT to use specific Python versions, there is a new
option called `python_version`. This is how to configure ROOT and Python for
the common use cases:

* Use the default Python interpreter:
  - `-Dpython=ON` (default)
* Search only for Python 2.x or only 3.x:
  - `-Dpython_version=2` or `-Dpython_version=3`
* Use a specific version of Python from `$PATH`:
  - `-Dpython_version=2.7` or `-Dpython_version=3.5`
* Use a specific Python interpreter, whatever the version:
  - `-DPYTHON_EXECUTABLE=/usr/local/bin/python`

Note: The use of `PYTHON_EXECUTABLE` requires the full path to the interpreter.

   - Reduce time taken by tests which takes too long to run ({One,Two}SidedFrequentistUpperLimitWithBands.C)
   - Disable PyROOT SQL tutorials (the C++ counterparts are since several releases).

{'fraction_non_alphanumeric': 0.07604547622833518, 'fraction_numerical': 0.007195102560025441, 'mean_word_length': 3.8258200652215613, 'pattern_counts': {'":': 0, '<': 73, '<?xml version=': 0, '>': 82, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2742155', 'n_tokens_mistral': 7240, 'n_tokens_neox': 6905, 'n_words': 3328}