<commit_before>.. meta::
   :robots: index,follow
   :description: ndtypes datashape specification
   :keywords: ndtypes, datashape, specification

.. sectionauthor:: <NAME> <skrah at bytereef.org>


.. _types:

#####
Types
#####

The set of all types comprises :ref:`dtypes <dtypes>` and :ref:`arrays <arrays>`.

The rest of this document assumes that the :py:mod:`ndtypes` module has been
imported:

.. testcode::

   from ndtypes import ndt

.. _dtypes:

******
Dtypes
******

An important notion in datashape is the ``dtype``, which roughly translates to
the element type of an array.  In datashape, the ``dtype`` can be of arbitrary
complexity and can contain e.g. tuples, records and functions.


.. _scalars:

=======
Scalars
=======

Scalars are the primitive C/C++ types. Most scalars are fixed-size and platform
independent.


Fixed size
----------

Datashape offers a number of fixed-size scalars. Here's how to construct a simple
:c:type:`int64_t` type:

.. doctest::

   >>> ndt('int64')
   ndt("int64")


All fixed-size scalars:

   +-----------+-----------------+------------+--------------+---------------+-----------------------+
   |   void    |     boolean     | signed int | unsigned int |  float [#f2]_ |        complex        |
   +===========+=================+============+==============+===============+=======================+
   | ``void``  | ``bool`` [#f1]_ |   ``int8`` |   ``uint8``  |  ``float16``  | ``complex32``         |
   +-----------+-----------------+------------+--------------+---------------+-----------------------+
   |           |                 |  ``int16`` |  ``uint16``  |  ``float32``  | ``complex64`` [#f3]_  |
   +-----------+-----------------+------------+--------------+---------------+-----------------------+
   |           |                 |  ``int32`` |  ``uint32``  |  ``float64``  | ``complex128`` [#f4]_ |
   +-----------+-----------------+------------+--------------+---------------+-----------------------+
   |           |                 |  ``int64`` |  ``uint64``  |               |                       |
   +-----------+-----------------+------------+--------------+---------------+-----------------------+

   .. [#f1] implemented as :c:type:`char`
   .. [#f2] IEEE 754-2008 binary floating point types
   .. [#f3] implemented as :c:type:`complex\<float32\>`<commit_msg>Add bfloat16 to the docs.<commit_after>.. meta::
   :robots: index,follow
   :description: ndtypes datashape specification
   :keywords: ndtypes, datashape, specification

.. sectionauthor:: <NAME> <skrah at bytereef.org>


.. _types:

#####
Types
#####

The set of all types comprises :ref:`dtypes <dtypes>` and :ref:`arrays <arrays>`.

The rest of this document assumes that the :py:mod:`ndtypes` module has been
imported:

.. testcode::

   from ndtypes import ndt

.. _dtypes:

******
Dtypes
******

An important notion in datashape is the ``dtype``, which roughly translates to
the element type of an array.  In datashape, the ``dtype`` can be of arbitrary
complexity and can contain e.g. tuples, records and functions.


.. _scalars:

=======
Scalars
=======

Scalars are the primitive C/C++ types. Most scalars are fixed-size and platform
independent.


Fixed size
----------

Datashape offers a number of fixed-size scalars. Here's how to construct a simple
:c:type:`int64_t` type:

.. doctest::

   >>> ndt('int64')
   ndt("int64")


All fixed-size scalars:

   +-----------+-----------------+------------+--------------+---------------+-----------------------+
   |   void    |     boolean     | signed int | unsigned int |  float [#f2]_ |        complex        |
   +===========+=================+============+==============+===============+=======================+
   | ``void``  | ``bool`` [#f1]_ |   ``int8`` |   ``uint8``  |  ``float16``  | ``complex32``         |
   +-----------+-----------------+------------+--------------+---------------+-----------------------+
   |           |                 |  ``int16`` |  ``uint16``  |  ``float32``  | ``complex64`` [#f3]_  |
   +-----------+-----------------+------------+--------------+---------------+-----------------------+
   |           |                 |  ``int32`` |  ``uint32``  |  ``float64``  | ``complex128`` [#f4]_ |
   +-----------+-----------------+------------+--------------+---------------+-----------------------+
   |           |                 |  ``int64`` |  ``uint64``  |  ``bfloat16`` | ``bcomplex32``        |
   +-----------+-----------------+------------+--------------+---------------+-----------------------+

   .. [#f1] implemented as :c:type:`char`
   .. [#f2] IEEE 754-2008 binary floating point types
   .. [#f3] implemented as :c:type:`complex\<float32\>`
{'fraction_non_alphanumeric': 0.3789944610140605, 'fraction_numerical': 0.022155943757988922, 'mean_word_length': 1.4052254098360655, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 19, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7352052', 'n_tokens_mistral': 1374, 'n_tokens_neox': 1200, 'n_words': 337}