<jupyter_start><jupyter_text>Calibrate dark imagesDark images, like any other images, need to be calibrated. Depending on the datayou have and the choices you have made in reducing your data, the steps toreducing your images may include:1. Subtracting overscan (only if you decide to subtract overscan from allimages).2. Trim the image (if it has overscan, whether you are using the overscan ornot).3. Subtract bias (if you need to scale the calibrated dark frames to a differentexposure time).<jupyter_code>from pathlib import Path

from astropy.nddata import CCDData
from ccdproc import ImageFileCollection
import ccdproc as ccdp<jupyter_output><empty_output><jupyter_text>Example 1: Overscan subtracted, bias not removed  Take a look at what images you haveFirst we gather up some information about the raw images and the reduced imagesup to this point. These examples have darks stored in a subdirectory of thefolder with the rest of the images, so we create an `ImageFileCollection` foreach.<jupyter_code>ex1_path_raw = Path('example-cryo-LFC')

ex1_images_raw = ImageFileCollection(ex1_path_raw)
ex1_darks_raw = ImageFileCollection(ex1_path_raw / 'darks')

ex1_path_reduced = Path('example1-reduced')
ex1_images_reduced = ImageFileCollection(ex1_path_reduced)<jupyter_output><empty_output><jupyter_text>Raw images, everything except the darks<jupyter_code>ex1_images_raw.summary['file', 'imagetyp', 'exptime', 'filter']<jupyter_output><empty_output><jupyter_text>Raw dark frames<jupyter_code>ex1_darks_raw.summary['file', 'imagetyp', 'exptime', 'filter']<jupyter_output><empty_output><jupyter_text>Decide which calibration  steps to takeThis example is, again, one of the chips of the LFC camera at Palomar. Inearlier notebooks we have seen that the chip has a [useful overscan region](01.08-Overscan.ipynbCase-1:-Cryogenically-cooled-Large-Format-Camera-(LFC)-at-Palomar), has little dark current except for some hot pixels, and sensor glow inone corner of the chip.Looking at the list of non-dark images (i.e., the flat and light images) showsthat for each exposure time in the non-dark images there is a set of darkexposures that has a matching, or very close to matching, exposure time.To be more explicit, there are flats with exposure times of 7.0 sec and 70.011sec and darks with exposure time of 7.0 and 70.0 sec. The dark and flat exposuretimes are close enough that there is no need to scale them.  The two images ofan object are each roughly 300 sec, matching the darks with exposure time 300sec. The very small difference in exposure time, under 0.1 sec, does not need tobe compensated for.Given this, we will:1. Subtract overscan from each of the darks. The useful overscan region is XXX(see LINK).2. Trim the overscan out of the dark images.We will *not* subtract bias from these images because we will *not* need torescale them to a different exposure time.  Calibrate the individual dark frames<jupyter_code>for ccd, file_name in ex1_darks_raw.ccds(imagetyp='DARK',            # Just get the dark frames
                                         ccd_kwargs={'unit': 'adu'}, # CCDData requires a unit for the image if 
                                                                     # it is not in the header
                                         return_fname=True           # Provide the file name too.
                                        ):    
    # Subtract the overscan
    ccd = ccdp.subtract_overscan(ccd, overscan=ccd[:, 2055:], median=True)
    
    # Trim the overscan
    ccd = ccdp.trim_image(ccd[:, :2048])
    
    # Save the result
    ccd.write(ex1_path_reduced / file_name)<jupyter_output><empty_output><jupyter_text>Reduced images (so far)<jupyter_code>ex1_images_reduced.refresh()
ex1_images_reduced.summary['file', 'imagetyp', 'exptime', 'filter', 'combined']<jupyter_output><empty_output><jupyter_text>Example 2: Overscan not subtracted, bias is removed<jupyter_code>ex2_path_raw = Path('example-thermo-electric')

ex2_images_raw = ImageFileCollection(ex2_path_raw)

ex2_path_reduced = Path('example2-reduced')
ex2_images_reduced = ImageFileCollection(ex2_path_reduced)<jupyter_output><empty_output><jupyter_text>We begin by looking at what exposure times we have in this data.<jupyter_code>ex2_images_raw.summary['file', 'imagetyp', 'exposure'].show_in_notebook()<jupyter_output><empty_output><jupyter_text>Decide what steps to take nextIn this case the only dark frames have exposure time 90 sec. Though that matchesthe exposure time of the science images, the flat field images are much shorterexposure time, ranging from 1 sec to 1.21 sec. This type of range of exposure istypical when twilight flats are taken. Since these are a much differentexposure time than the darks, the dark frames will need to be scaled.Recall that for this camera the overscan is not useful and should betrimmed off.Given this, we will:1. Trim the overscan from each of the dark frames.2. Subtract calibration bias from the dark frames so that we can scale the darksto a different exposure time.  Calibration the individual dark frames First, we read the combined bias image created in the previous notebook. Thoughwe could do this based on the file name, using a systematic set of headerkeywords to keep track of which images have been combined is less likely to leadto errors.<jupyter_code>combined_bias = CCDData.read(ex2_images_reduced.files_filtered(imagetyp='bias', 
                                                               combined=True, 
                                                               include_path=True)[0])
for ccd, file_name in ex2_images_raw.ccds(imagetyp='DARK',            # Just get the bias frames
                                          return_fname=True           # Provide the file name too.
                                         ):
        
    # Trim the overscan
    ccd = ccdp.trim_image(ccd[:, :4096])
    
    # Subtract bias
    ccd = ccdp.subtract_bias(ccd, combined_bias)
    # Save the result
    ccd.write(ex2_path_reduced / file_name)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06540049545829893, 'fraction_numerical': 0.013047068538398017, 'mean_word_length': 3.662047729022325, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 15, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '52600', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1775, 'n_tokens_neox': 1695, 'n_words': 653, 'path': 'notebooks/03-05-Calibrate-dark-images.ipynb', 'repo_name': 'programatt/ccd-reduction-and-photometry-guide', 'size': '6055'}