<commit_before>.. cumulative_residual_entropy.rst
.. py:module:: dit.other.cumulative_residual_entropy

***************************
Cumulative Residual Entropy
***************************

The cumulative residual entropy :cite:`Rao2004` is an alternative to the
differential Shannon entropy. The differential entropy has many issues,
including that it can be negative even for simple distributions such as the
uniform distribution; and that if one takes discrete estimates that limit to the
continuous distribution, the discrete entropy does not limit to the differential
(continuous) entropy. It also attempts to provide meaningful differences between
numerically different random variables, such as a die labeled [1, 2, 3, 4, 5, 6]
and one lebeled [1, 2, 3, 4, 5, 100].

.. note::

   The Cumulative Residual Entropy is unrelated to :doc:`residual_entropy`.

.. math::

   \CRE[X] = -\int_{0}^{\infty} p(|X| > x) \log_{2} p(|X| > x) dx

The genearlized form integrates over the intire set of reals rather than just<commit_msg>fix a reference between documents<commit_after>.. cumulative_residual_entropy.rst
.. py:module:: dit.other.cumulative_residual_entropy

***************************
Cumulative Residual Entropy
***************************

The cumulative residual entropy :cite:`Rao2004` is an alternative to the
differential Shannon entropy. The differential entropy has many issues,
including that it can be negative even for simple distributions such as the
uniform distribution; and that if one takes discrete estimates that limit to the
continuous distribution, the discrete entropy does not limit to the differential
(continuous) entropy. It also attempts to provide meaningful differences between
numerically different random variables, such as a die labeled [1, 2, 3, 4, 5, 6]
and one lebeled [1, 2, 3, 4, 5, 100].

.. note::

   The Cumulative Residual Entropy is unrelated to
   :doc:`../multivariate/residual_entropy`.

.. math::

   \CRE[X] = -\int_{0}^{\infty} p(|X| > x) \log_{2} p(|X| > x) dx

The genearlized form integrates over the intire set of reals rather than just
{'fraction_non_alphanumeric': 0.1279236276849642, 'fraction_numerical': 0.01909307875894988, 'mean_word_length': 5.022988505747127, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2489652', 'n_tokens_mistral': 621, 'n_tokens_neox': 550, 'n_words': 259}