<commit_before>The Surprisingly ParalleL spArse Tensor Toolkit
========================================================

SPLATT is a library and C API for sparse tensor factorization. We support shared-memory parallelism with OpenMP and (soon) distributed-memory parallelism with MPI.


Building & Installing
---------------------
In short,

    $ ./configure && make

will build the SPLATT library and its executable. You can add a '--help' flag to configure to see additional build options. To install,

    $ make install

will suffice. The installation prefix can be chosen by adding a '--prefix=<DIR>' flag
to configure.


Executable
----------
After building, an executable will found in the build/ directory (or the installation prefix if SPLATT was installed). SPLATT builds a single executable which features a number of sub-commands:

* cpd
* check
* convert
* reorder
* stats

All SPLATT commands are executed in the form

    $ splatt CMD [OPTIONS]

You can execute

    $ splatt CMD --help

for usage information of each command.

**Example 1**

This runs splatt-check on 'mytensor.tns' and writes the fixed tensor to 'fixed.tns'. The splatt-check routine finds empty slices and duplicate nonzero entries. Empty slices are indices in any mode which do not have any nonzero entries associated with them. Some SPLATT routines (including CPD) expect there to be no empty slices, so running splatt-check
on any new tensors is recommended.

    $ splatt check mytensor.tns  --fix=fixed.tns

**Example 2**

This runs splatt-cpd on 'mytensor.tns' and finds a rank-25 CPD of the tensor. Adding '-t 4' instructs SPLATT to use four CPU threads during the computation. The matrix factors are written to 'mode<N>.mat' and lambda, the vector for scaling, is written to 'lambda.mat'. See the '--help' option to see available output formats.

    $ splatt cpd mytensor.tns -r 25 -t 4


API
---
Coming soon!


Licensing
---------
SPLATT is released under the MIT License. Please see the 'LICENSE' file for details.<commit_msg>cleaned up README formatting<commit_after>The Surprisingly ParalleL spArse Tensor Toolkit
========================================================

SPLATT is a library and C API for sparse tensor factorization. We support
shared-memory parallelism with OpenMP and (soon) distributed-memory parallelism
with MPI.


Building & Installing
---------------------
In short,

    $ ./configure && make

will build the SPLATT library and its executable. You can add a '--help' flag
to configure to see additional build options. To install,

    $ make install

will suffice. The installation prefix can be chosen by adding a
'--prefix=<DIR>' flag to configure.


Executable
----------
After building, an executable will found in the build/ directory (or the
installation prefix if SPLATT was installed). SPLATT builds a single executable
which features a number of sub-commands:

* cpd
* check
* convert
* reorder
* stats

All SPLATT commands are executed in the form

    $ splatt CMD [OPTIONS]

You can execute

    $ splatt CMD --help

for usage information of each command.

**Example 1**

This runs splatt-check on 'mytensor.tns' and writes the fixed tensor to
'fixed.tns'. The splatt-check routine finds empty slices and duplicate nonzero
entries. Empty slices are indices in any mode which do not have any nonzero
entries associated with them. Some SPLATT routines (including CPD) expect there
to be no empty slices, so running splatt-check
on any new tensors is recommended.

    $ splatt check mytensor.tns  --fix=fixed.tns

**Example 2**

This runs splatt-cpd on 'mytensor.tns' and finds a rank-25 CPD of the tensor.
Adding '-t 4' instructs SPLATT to use four CPU threads during the computation.
The matrix factors are written to 'mode<N>.mat' and lambda, the vector for
scaling, is written to 'lambda.mat'. See the '--help' option to see available
output formats.

    $ splatt cpd mytensor.tns -r 25 -t 4


API
---
Coming soon!


Licensing
---------
SPLATT is released under the MIT License. Please see the 'LICENSE' file for
details.
{'fraction_non_alphanumeric': 0.10637773947303619, 'fraction_numerical': 0.003939916276779119, 'mean_word_length': 3.6159090909090907, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '153118', 'n_tokens_mistral': 1148, 'n_tokens_neox': 1061, 'n_words': 558}