<commit_before>.. _reduction_toolbox:

Reduction toolbox
=================

.. note::

    This is not intended to be an introduction to image reduction. While
    performing the steps presented here may be the correct way to reduce data
    in some cases, it is not correct in all cases.

    A much more detailed guide to CCD data reduction is
    `available <https://mwcraig.github.io/ccd-as-book/00-00-Preface>`_

Logging in `ccdproc`
--------------------

All logging in `ccdproc` is done in the sense of recording the steps performed
in image metadata. if you want to do `logging in the python sense of the word
<https://docs.python.org/library/logging.html>`_ please see those docs.

There are basically three logging options:

1. Implicit logging: No setup or keywords needed, each of the functions below adds a note to the metadata when it is performed.
2. Explicit logging: You can specify what information is added to the metadata using the ``add_keyword`` argument for any of the functions below.
3. No logging: If you prefer no logging be done you can "opt-out" by calling each function with ``add_keyword=None``.

.. _create_deviation:

Gain correct and create deviation image
----------------------------------------

Uncertainty
+++++++++++

An uncertainty can be calculated from your data with
`~ccdproc.create_deviation`:

    >>> from astropy import units as u
    >>> import numpy as np
    >>> from astropy.nddata import CCDData
    >>> import ccdproc
    >>> img = np.random.normal(loc=10, scale=0.5, size=(100, 232))
    >>> data = CCDData(img, unit=u.adu)
    >>> data_with_deviation = ccdproc.create_deviation(
    ...                           data, gain=1.5 * u.electron/u.adu,
    ...                           readnoise=5 * u.electron)
    >>> data_with_deviation.header['exposure'] = 30.0  # for dark subtraction

The uncertainty, :math:`u_{ij}`, at pixel :math:`(i,~j)` with value
:math:`p_{ij}` is calculated as

.. math::

    u_{ij} = \left(g * p_{ij} + \sigma_{rn}^2\right)^{\frac{1}{2}},

where :math:`\sigma_{rn}` is the read noise. Gain is only necessary when the
image units are different than the units of the read noise, and is used only
to calculate the uncertainty. The data itself is not scaled by this function.

As with all of the functions in `ccdproc`, *the input image is not modified*.

In the example above the new image ``data_with_deviation`` has its uncertainty
set.

Gain
++++

To apply a gain to an image, do:

    >>> gain_corrected = ccdproc.gain_correct(data_with_deviation, 1.5*u.electron/u.adu)

The result ``gain_corrected`` has its data *and uncertainty* scaled by the gain
and its unit updated.

There are several ways to provide the gain, among them as an
`astropy.units.Quantity`, as in the example above, as a `ccdproc.Keyword`.
See to documentation for `~ccdproc.gain_correct` for details.

Clean image
-----------

There are two ways to clean an image of cosmic rays. One is to use clipping to
create a mask for a stack of images, as described in :ref:`clipping`.

The other is to replace, in a single image, each pixel that is several
standard deviations from a central value in a region surrounding that pixel.
The methods below describe how to do that.

LACosmic
++++++++

The lacosmic technique identifies cosmic rays by identifying pixels based on a
variation of the Laplacian edge detection.  The algorithm is an implementation
of the code describe in van Dokkum (2001) [1]_ as implemented
in [astroscrappy](https://github.com/astropy/astroscrappy) [2]_.

Use this technique with `~ccdproc.cosmicray_lacosmic`:

    >>> cr_cleaned = ccdproc.cosmicray_lacosmic(gain_corrected, sigclip=5)

.. note::

    A significant error in `~ccdproc.cosmicray_lacosmic` in version 2.1 of
    ccdproc. By default, `~ccdproc.cosmicray_lacosmic` multiplies the image by
    the gain; prior to version 2.1 it did so without changing the units of
    the image.

    There are two ways to correctly invoke `~ccdproc.cosmicray_lacosmic`:

    + Supply a gain-corrected image, in units of ``electron``, and set ``gain=1.0``
      (the default value) in `~ccdproc.cosmicray_lacosmic`.
    + Supply an image in ``adu`` and set the ``gain`` argument of
      `~ccdproc.cosmicray_lacosmic` to the appropriate value for your
      instrument. Ideally, pass in a ``gain`` with units, but if units are
      omitted the will be assumed to be ``electron/adu``.

median
++++++<commit_msg>Modify documentation note in response to review<commit_after>.. _reduction_toolbox:

Reduction toolbox
=================

.. note::

    This is not intended to be an introduction to image reduction. While
    performing the steps presented here may be the correct way to reduce data
    in some cases, it is not correct in all cases.

    A much more detailed guide to CCD data reduction is
    `available <https://mwcraig.github.io/ccd-as-book/00-00-Preface>`_

Logging in `ccdproc`
--------------------

All logging in `ccdproc` is done in the sense of recording the steps performed
in image metadata. if you want to do `logging in the python sense of the word
<https://docs.python.org/library/logging.html>`_ please see those docs.

There are basically three logging options:

1. Implicit logging: No setup or keywords needed, each of the functions below adds a note to the metadata when it is performed.
2. Explicit logging: You can specify what information is added to the metadata using the ``add_keyword`` argument for any of the functions below.
3. No logging: If you prefer no logging be done you can "opt-out" by calling each function with ``add_keyword=None``.

.. _create_deviation:

Gain correct and create deviation image
----------------------------------------

Uncertainty
+++++++++++

An uncertainty can be calculated from your data with
`~ccdproc.create_deviation`:

    >>> from astropy import units as u
    >>> import numpy as np
    >>> from astropy.nddata import CCDData
    >>> import ccdproc
    >>> img = np.random.normal(loc=10, scale=0.5, size=(100, 232))
    >>> data = CCDData(img, unit=u.adu)
    >>> data_with_deviation = ccdproc.create_deviation(
    ...                           data, gain=1.5 * u.electron/u.adu,
    ...                           readnoise=5 * u.electron)
    >>> data_with_deviation.header['exposure'] = 30.0  # for dark subtraction

The uncertainty, :math:`u_{ij}`, at pixel :math:`(i,~j)` with value
:math:`p_{ij}` is calculated as

.. math::

    u_{ij} = \left(g * p_{ij} + \sigma_{rn}^2\right)^{\frac{1}{2}},

where :math:`\sigma_{rn}` is the read noise. Gain is only necessary when the
image units are different than the units of the read noise, and is used only
to calculate the uncertainty. The data itself is not scaled by this function.

As with all of the functions in `ccdproc`, *the input image is not modified*.

In the example above the new image ``data_with_deviation`` has its uncertainty
set.

Gain
++++

To apply a gain to an image, do:

    >>> gain_corrected = ccdproc.gain_correct(data_with_deviation, 1.5*u.electron/u.adu)

The result ``gain_corrected`` has its data *and uncertainty* scaled by the gain
and its unit updated.

There are several ways to provide the gain, among them as an
`astropy.units.Quantity`, as in the example above, as a `ccdproc.Keyword`.
See to documentation for `~ccdproc.gain_correct` for details.

Clean image
-----------

There are two ways to clean an image of cosmic rays. One is to use clipping to
create a mask for a stack of images, as described in :ref:`clipping`.

The other is to replace, in a single image, each pixel that is several
standard deviations from a central value in a region surrounding that pixel.
The methods below describe how to do that.

LACosmic
++++++++

The lacosmic technique identifies cosmic rays by identifying pixels based on a
variation of the Laplacian edge detection.  The algorithm is an implementation
of the code describe in van Dokkum (2001) [1]_ as implemented
in [astroscrappy](https://github.com/astropy/astroscrappy) [2]_.

Use this technique with `~ccdproc.cosmicray_lacosmic`:

    >>> cr_cleaned = ccdproc.cosmicray_lacosmic(gain_corrected, sigclip=5)

.. note::

    By default, `~ccdproc.cosmicray_lacosmic` multiplies the image by
    the gain; prior to version 2.1 it did so without changing the units of
    the image which could result in incorrect results.

    There are two ways to correctly invoke `~ccdproc.cosmicray_lacosmic`:

    + Supply a gain-corrected image, in units of ``electron``, and set ``gain=1.0``
      (the default value) in `~ccdproc.cosmicray_lacosmic`.
    + Supply an image in ``adu`` and set the ``gain`` argument of
      `~ccdproc.cosmicray_lacosmic` to the appropriate value for your
      instrument. Ideally, pass in a ``gain`` with units, but if units are
      omitted the will be assumed to be ``electron/adu``.

median
++++++
{'fraction_non_alphanumeric': 0.1102077687443541, 'fraction_numerical': 0.009033423667570008, 'mean_word_length': 3.4732323232323234, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 67, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4120248', 'n_tokens_mistral': 2678, 'n_tokens_neox': 2537, 'n_words': 1126}