<commit_before>Data
====

InfluxDB
--------

All data is stored in `InfluxDB`_.  The following sections describe how the
data is stored.

Measurement
~~~~~~~~~~~

.. epigraph::

   The part of InfluxDB’s structure that describes the data stored in the
   associated fields. Measurements are strings.

   -- `InfluxDB`_ `measurement`_ documentation

Tags
~~~~

.. epigraph::

   The key-value pair in InfluxDB’s data structure that records metadata. Tags
   are an optional part of InfluxDB’s data structure but they are useful for
   storing commonly-queried metadata; tags are indexed so queries on tags are
   performant. Query tip: Compare tags to fields; fields are not indexed.

   -- `InfluxDB`_ `tag`_ documentation

.. note::

   The `tag`_ values are always interpreted as strings.

Each `tag`_ value should have very few possible values which yields a low
`series cardinality`_.

Fields
~~~~~~

.. epigraph::

   The key-value pair in InfluxDB’s data structure that records metadata and
   the actual data value. Fields are required in InfluxDB’s data structure and
   they are not indexed - queries on field values scan all points that match
   the specified time range and, as a result, are not performant relative to
   tags. Query tip: Compare fields to tags; tags are indexed

   -- `InfluxDB`_ `field`_ documentation

Structure
---------

``den`` records structure data in a measurement named ``structure``.

.. _structure-data-model:

Data Model
~~~~~~~~~~

An example of the structures data model, limited to thermostat read/write
permissions, from the Nest `Data Model Viewer`_ (as `JSON`_):

.. code-block:: js

   {
       "structures": {
           "VqFabWH21nwVyd4RWgJgNb292wa7hG_dUwo2i2SG7j3-BOLY0BA4sw": {
               "structure_id": "VqFabWH21nwVyd4RWgJgNb292wa7hG_dUwo2i2SG7j3-BOLY0BA4sw",
               "thermostats": [ "peyiJNo0IldT2YlIVtYaGQ", ... ],
               "smoke_co_alarms": [ "RTMTKxsQTCxzVcsySOHPxKoF4OyCifrs", ... ],
               "cameras": [ "awJo6rH…", ... ],
               "devices": {
               },
               "away": "home",
               "name": "Home",
               "country_code": "US",
               "postal_code": "94304",
               "peak_period_start_time": "2016-10-31T23:59:59.000Z",
               "peak_period_end_time": "2016-10-31T23:59:59.000Z",
               "time_zone": "America/Los_Angeles",
               "eta": {
               },
               "rhr_enrollment": true,
               "wheres": {
                   "Fqp6wJI...": {
                   }
               }
           }
       }
   }

`InfluxDB`_ Point
~~~~~~~~~~~~~~~~~

The :ref:`structure-data-model` example transformed into a list containing a
single `InfluxDB`_ `point`_ (as `Python`_):

.. code-block:: python

   [
       {
           "measure": "structure",
           "tags": {
               "away": "home",
               "country_code": "US",
               "name": "Home",
               "postal_code": "94304",
               "structure_id": "VqFabWH21nwVyd4RWgJgNb292wa7hG_dUwo2i2SG7j3-BOLY0BA4sw",
               "thermostat_id": "peyiJNo0IldT2YlIVtYaGQ",
               "time_zone": "America/Los_Angeles"
           },
           "fields": {
               "is_away": 0
           }
       }
   ]

.. note::

   The :ref:`structure-data-model` ``thermostats`` list is denormalized into a
   point per thermostat id.

Measurement
~~~~~~~~~~~

``den`` records structure data in a measurement named ``structure``.


Tags
~~~~

#. `away`_
#. `country_code`_
#. `name`_
#. `postal_code`_
#. `structure_id`_
#. `thermostat_id`_
#. `time_zone`_

Fields
~~~~~~

#. ``is_away`` is a numeric representation of the ``away`` tag, i.e., ``home``
   as ``0``, ``away`` as ``1``

Thermostat
----------

``den`` records thermostat data in `InfluxDB`_.

.. _thermostat-data-model:

Data Model
~~~~~~~~~~

An example of the devices data model, limited to thermostat read/write
permissions, from the Nest `Data Model Viewer`_ (as `JSON`_):

.. code-block:: js

   {
       "devices": {
           "thermostats": {
               "peyiJNo0IldT2YlIVtYaGQ": {
                   "ambient_temperature_c": 21.5,
                   "ambient_temperature_f": 72,
                   "away_temperature_high_c": 24.5,
                   "away_temperature_high_f": 80,
                   "away_temperature_low_c": 19.5,
                   "away_temperature_low_f": 65,
                   "can_cool": true,
                   "can_heat": true,
                   "device_id": "peyiJNo0IldT2YlIVtYaGQ",
                   "eco_temperature_high_c": 24.5,
                   "eco_temperature_high_f": 80,
                   "eco_temperature_low_c": 19.5,<commit_msg>docs - Cleanup.<commit_after>Data
====

InfluxDB
--------

All data is stored in `InfluxDB`_.  The following sections describe how the
data is stored.

Measurement
~~~~~~~~~~~

.. epigraph::

   The part of InfluxDB’s structure that describes the data stored in the
   associated fields. Measurements are strings.

   -- `InfluxDB`_ `measurement`_ documentation

Tags
~~~~

.. epigraph::

   The key-value pair in InfluxDB’s data structure that records metadata. Tags
   are an optional part of InfluxDB’s data structure but they are useful for
   storing commonly-queried metadata; tags are indexed so queries on tags are
   performant. Query tip: Compare tags to fields; fields are not indexed.

   -- `InfluxDB`_ `tag`_ documentation

.. note::

   The `tag`_ values are always interpreted as strings.

Each `tag`_ value should have very few possible values which yields a low
`series cardinality`_.

Fields
~~~~~~

.. epigraph::

   The key-value pair in InfluxDB’s data structure that records metadata and
   the actual data value. Fields are required in InfluxDB’s data structure and
   they are not indexed - queries on field values scan all points that match
   the specified time range and, as a result, are not performant relative to
   tags. Query tip: Compare fields to tags; tags are indexed

   -- `InfluxDB`_ `field`_ documentation

Structure
---------

.. _structure-data-model:

Data Model
~~~~~~~~~~

An example of the structures data model, limited to thermostat read/write
permissions, from the Nest `Data Model Viewer`_ (as `JSON`_):

.. code-block:: js

   {
       "structures": {
           "VqFabWH21nwVyd4RWgJgNb292wa7hG_dUwo2i2SG7j3-BOLY0BA4sw": {
               "structure_id": "VqFabWH21nwVyd4RWgJgNb292wa7hG_dUwo2i2SG7j3-BOLY0BA4sw",
               "thermostats": [ "peyiJNo0IldT2YlIVtYaGQ", ... ],
               "smoke_co_alarms": [ "RTMTKxsQTCxzVcsySOHPxKoF4OyCifrs", ... ],
               "cameras": [ "awJo6rH…", ... ],
               "devices": {
               },
               "away": "home",
               "name": "Home",
               "country_code": "US",
               "postal_code": "94304",
               "peak_period_start_time": "2016-10-31T23:59:59.000Z",
               "peak_period_end_time": "2016-10-31T23:59:59.000Z",
               "time_zone": "America/Los_Angeles",
               "eta": {
               },
               "rhr_enrollment": true,
               "wheres": {
                   "Fqp6wJI...": {
                   }
               }
           }
       }
   }

`InfluxDB`_ Point
~~~~~~~~~~~~~~~~~

The :ref:`structure-data-model` example transformed into a list containing a
single `InfluxDB`_ `point`_ (as `Python`_):

.. code-block:: python

   [
       {
           "measure": "structure",
           "tags": {
               "away": "home",
               "country_code": "US",
               "name": "Home",
               "postal_code": "94304",
               "structure_id": "VqFabWH21nwVyd4RWgJgNb292wa7hG_dUwo2i2SG7j3-BOLY0BA4sw",
               "thermostat_id": "peyiJNo0IldT2YlIVtYaGQ",
               "time_zone": "America/Los_Angeles"
           },
           "fields": {
               "is_away": 0
           }
       }
   ]

.. note::

   The :ref:`structure-data-model` ``thermostats`` list is denormalized into a
   point per thermostat id.

Measurement
~~~~~~~~~~~

``den`` records structure data in a measurement named ``structure``.


Tags
~~~~

#. `away`_
#. `country_code`_
#. `name`_
#. `postal_code`_
#. `structure_id`_
#. `thermostat_id`_
#. `time_zone`_

Fields
~~~~~~

#. ``is_away`` is a numeric representation of the ``away`` tag, i.e., ``home``
   as ``0``, ``away`` as ``1``

Thermostat
----------

.. _thermostat-data-model:

Data Model
~~~~~~~~~~

An example of the devices data model, limited to thermostat read/write
permissions, from the Nest `Data Model Viewer`_ (as `JSON`_):

.. code-block:: js

   {
       "devices": {
           "thermostats": {
               "peyiJNo0IldT2YlIVtYaGQ": {
                   "ambient_temperature_c": 21.5,
                   "ambient_temperature_f": 72,
                   "away_temperature_high_c": 24.5,
                   "away_temperature_high_f": 80,
                   "away_temperature_low_c": 19.5,
                   "away_temperature_low_f": 65,
                   "can_cool": true,
                   "can_heat": true,
                   "device_id": "peyiJNo0IldT2YlIVtYaGQ",
                   "eco_temperature_high_c": 24.5,
                   "eco_temperature_high_f": 80,
                   "eco_temperature_low_c": 19.5,
{'fraction_non_alphanumeric': 0.13128160981383838, 'fraction_numerical': 0.02582589045518132, 'mean_word_length': 2.137744767049291, 'pattern_counts': {'":': 88, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3655070', 'n_tokens_mistral': 3264, 'n_tokens_neox': 2955, 'n_words': 761}