<commit_before>.. _nmrglue_tutorial:

########
Tutorial
########

Introduction
============

nmrglue is a python module for reading, writing, and interacting with the 
spectral data stored in a number of common NMR data formats.  This tutorial 
provides an overview of some of the features of nmrglue.  A basic 
understanding of python is assumed which can be obtained by reading some
of the `python documentation <http://docs.python.org/>`_.  The examples in 
this tutorial can be run interactively from the python shell but the use of an
enhanced python shell which provides non-blocking control of GUI threads, 
for example  `ipython <http://www.ipython.org>`_, is 
recommended when trying the examples which use matplotlib.  The sample data
using in this tutorial is 
`available <http://code.google.com/p/nmrglue/downloads/list>`_ is you wish to 
follow along with the same files.

The software can also be used in Google Colabs - see details below_.

Reading NMR files
=================

nmrglue can read and write to a number of common NMR file formats.  To see 
how simple this can be let's read a 2D NMRPipe file. (Note: If you need an example 
dataset, use the one provided `here <https://github.com/jjhelmus/nmrglue/tree/master/tests/pipe_proc_tests>`_, and replace ``test.fid`` or ``test.ft`` with the names
of the files ending in ``.fid`` or ``.dat``).

    >>> import nmrglue as ng
    >>> dic,data = ng.pipe.read("test.fid")

Here we have imported the ``nmrglue`` module and opened the NMRPipe file 
``test.fid``.  nmrglue contains a number of modules for reading and writing NMR
files and all of these modules have a ``read`` function which opens a file
or directory containing NMR data, reads in any necessary information, and loads 
the spectral data into memory.  The ``read`` function returns a 2-tuple 
containing a python dictionary with file and spectral parameters and a 
`numpy <https://www.numpy.org/>`_ array object containing the numeric 
spectral data.  Currently the following file formats are supported by nmrglue
with the associated module:

=======  ======================== ==================
Module   File Format              Reference 
=======  ======================== ==================
bruker   Bruker                   https://www.bruker.com/service/information-communication/user-manuals/nmr.html 
pipe     NMRPipe                  https://www.ibbr.umd.edu/nmrpipe/index.html
sparky   Sparky                   https://nmrfam.wisc.edu/nmrfam-sparky-distribution/
varian   Varian/Agilent           
rnmrtk   Rowland NMR Toolkit      https://rnmrtk.uchc.edu/rnmrtk/RNMRTK.html
jcampdx  JCAMP-DX                 http://www.jcamp-dx.org/
nmrml    NMR Markup Language      https://github.com/nmrML/nmrML
simpson  Simpson                  https://inano.au.dk/about/research-centers/nmr/software/simpson/
tecmag   Technology for Magnetic  https://www.tecmag.com/
         Resonance
=======  ======================== ==================

Examining the data object in more detail:

    >>> data.ndim
    2
    >>> data.shape
    (332, 1500)
    >>> data.dtype
    dtype('complex64')

We can see that this is a two dimensional data set with 1500 complex points
in the direct dimension and 332 points in the indirect dimension.  nmrglue 
takes care of converting the raw data in the file into an array of appropriate 
type, dimensionality, and quadrature.  For complex data the last axis, 
typically the direct dimension, is converted to a complex data type.  The other
axes are not converted.

In some cases, not all of the information needed to represent the spectral data 
as a well formed numpy array is stored in the file, or the values determined 
automatically are incorrect. In many of these cases, this information can be <commit_msg>Add spinsolve to supported formats in tutorial.rst<commit_after>.. _nmrglue_tutorial:

########
Tutorial
########

Introduction
============

nmrglue is a python module for reading, writing, and interacting with the 
spectral data stored in a number of common NMR data formats.  This tutorial 
provides an overview of some of the features of nmrglue.  A basic 
understanding of python is assumed which can be obtained by reading some
of the `python documentation <http://docs.python.org/>`_.  The examples in 
this tutorial can be run interactively from the python shell but the use of an
enhanced python shell which provides non-blocking control of GUI threads, 
for example  `ipython <http://www.ipython.org>`_, is 
recommended when trying the examples which use matplotlib.  The sample data
using in this tutorial is 
`available <http://code.google.com/p/nmrglue/downloads/list>`_ is you wish to 
follow along with the same files.

The software can also be used in Google Colabs - see details below_.

Reading NMR files
=================

nmrglue can read and write to a number of common NMR file formats.  To see 
how simple this can be let's read a 2D NMRPipe file. (Note: If you need an example 
dataset, use the one provided `here <https://github.com/jjhelmus/nmrglue/tree/master/tests/pipe_proc_tests>`_, and replace ``test.fid`` or ``test.ft`` with the names
of the files ending in ``.fid`` or ``.dat``).

    >>> import nmrglue as ng
    >>> dic,data = ng.pipe.read("test.fid")

Here we have imported the ``nmrglue`` module and opened the NMRPipe file 
``test.fid``.  nmrglue contains a number of modules for reading and writing NMR
files and all of these modules have a ``read`` function which opens a file
or directory containing NMR data, reads in any necessary information, and loads 
the spectral data into memory.  The ``read`` function returns a 2-tuple 
containing a python dictionary with file and spectral parameters and a 
`numpy <https://www.numpy.org/>`_ array object containing the numeric 
spectral data.  Currently the following file formats are supported by nmrglue
with the associated module:

=======    ======================== ==================
Module     File Format              Reference
=======    ======================== ==================
bruker     Bruker                   https://www.bruker.com/service/information-communication/user-manuals/nmr.html
pipe       NMRPipe                  https://www.ibbr.umd.edu/nmrpipe/index.html
sparky     Sparky                   https://nmrfam.wisc.edu/nmrfam-sparky-distribution/
varian     Varian/Agilent
rnmrtk     Rowland NMR Toolkit      https://rnmrtk.uchc.edu/rnmrtk/RNMRTK.html
jcampdx    JCAMP-DX                 http://www.jcamp-dx.org/
nmrml      NMR Markup Language      https://github.com/nmrML/nmrML
simpson    Simpson                  https://inano.au.dk/about/research-centers/nmr/software/simpson/
spinsolve  Magritek or JCAMP-DX
tecmag     Technology for Magnetic  https://www.tecmag.com/
           Resonance
=======    ======================== ==================

Examining the data object in more detail:

    >>> data.ndim
    2
    >>> data.shape
    (332, 1500)
    >>> data.dtype
    dtype('complex64')

We can see that this is a two dimensional data set with 1500 complex points
in the direct dimension and 332 points in the indirect dimension.  nmrglue 
takes care of converting the raw data in the file into an array of appropriate 
type, dimensionality, and quadrature.  For complex data the last axis, 
typically the direct dimension, is converted to a complex data type.  The other
axes are not converted.

In some cases, not all of the information needed to represent the spectral data 
as a well formed numpy array is stored in the file, or the values determined 
automatically are incorrect. In many of these cases, this information can be 
{'fraction_non_alphanumeric': 0.114900314795383, 'fraction_numerical': 0.004984260230849948, 'mean_word_length': 3.5145056246299586, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 43, 'https://': 18, 'lorem ipsum': 0, 'www.': 12, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '735323', 'n_tokens_mistral': 2221, 'n_tokens_neox': 2071, 'n_words': 910}