<jupyter_start><jupyter_text>Compute the Galaxy lightcurves assuming large mask Save metadataHere we compute the galaxy lightcurves assuming a simple, large, mask we derive.<jupyter_code># %load /Users/obsidian/Desktop/defaults.py
import numpy as np
%matplotlib inline
%config InlineBackend.figure_format = 'retina'

from lightkurve import KeplerTargetPixelFile
import glob<jupyter_output><empty_output><jupyter_text>You'll need to download all the C16 data first:  `wget -r http://quicklook.keplerscience.org/c16-galaxies/`    Once you have that, assemble all the filenames into an iterable list.<jupyter_code>fns = glob.glob('/Volumes/Truro/C16/quicklook.keplerscience.org/c16-galaxies/*.fits.gz')
len(fns)<jupyter_output><empty_output><jupyter_text>We could assemble an $n_{\mathrm{cadence}} \times n_{\mathrm{tpfs}} = (3888 \times 8380)$ array of lightcurves, then perform PCA.  Alternatively, we could also assemble subsets of PCA grouped by channels, since we anticipate that nearby lightcurves share similar motion-induced noise characteristics.Let's start with the "full-focal-plane" PCA first.<jupyter_code>n_tpfs = len(fns)
n_cadences = 3888

# x, y centroids and lightcurve flux
x0s_array = np.zeros((n_cadences, n_tpfs))
y0s_array = np.zeros((n_cadences, n_tpfs))
lc_array = np.zeros((n_cadences, n_tpfs))
sigma_clip_array = np.zeros((n_cadences, n_tpfs))

# Channel and number of pixels in aperture and background
n_pix_ap = np.zeros(n_tpfs)
n_pix_bg = np.zeros(n_tpfs)
ch_array = np.zeros(n_tpfs)

#An array that tracks the TPFs that fail for any reason:
fail_array = np.zeros(n_tpfs, dtype=np.bool)  
cadn = np.arange(0, n_cadences, 1)<jupyter_output><empty_output><jupyter_text>First we simply need to acquire the lightcurves.  We only have TPFs available, so we need to pick a mask.  We're looking for *transients* that can occur anywhere in the galaxy's disk, so I recommend a **large** mask: all pixels except the edge pixels.  We will exclude pixels that were not downloaded (nan-everywhere).<jupyter_code>from astropy.utils.console import ProgressBar
%%time

with ProgressBar(n_tpfs, ipython_widget=True) as bar:
    for i,fn in enumerate(fns[0:n_tpfs]):
        bar.update()
        
        try:
            # Get x, y, centroids
            tpf = KeplerTargetPixelFile(fn)
            xs, ys = tpf.centroids(aperture_mask='all')
            x0s_array[:, i] = xs
            y0s_array[:, i] = ys

            # Assemble mask
            finite_mask = np.alltrue(tpf.flux==tpf.flux, axis=0)
            core_mask = tpf.pipeline_mask * False
            core_mask[1:-1, 1:-1] = True
            core_mask = core_mask & finite_mask
            bg_mask = tpf.pipeline_mask * False
            bg_mask[0, :] = True
            bg_mask[-1, :] = True
            bg_mask[:, 0] = True
            bg_mask[:, -1] = True
            bg_mask = bg_mask & finite_mask
            n_pix_ap[i] = core_mask.sum()
            n_pix_bg[i] = bg_mask.sum()

            # Compute the lightcurve
            core_lc = tpf.to_lightcurve(aperture_mask=core_mask) / core_mask.sum()
            bg_lc = tpf.to_lightcurve(aperture_mask=bg_mask) / bg_mask.sum()
            target_lc = core_lc - bg_lc.flux
            lc_array[:, i] = target_lc.flux
            
            # Identify routinely outlier flux cadences
            _, outlier_mask_sigma_clip = target_lc.flatten().remove_outliers(return_mask=True, sigma=3)
            sigma_clip_array[:, i] = outlier_mask_sigma_clip

            ch_array[i] = tpf.channel
        except:
            print("{:04d} Fail".format(i))
            fail_array[i] = True
lc_array.shape<jupyter_output><empty_output><jupyter_text>The results are $n_{\mathrm{cadence}} \times n_{\mathrm{tpf}}$ arrays.  Save the arrays locally for later use.<jupyter_code>#! mkdir ../data/C16_raw
np.save('../data/C16_raw/x0s_array.npy', x0s_array)
np.save('../data/C16_raw/y0s_array.npy', y0s_array)
np.save('../data/C16_raw/lc_array.npy', lc_array)
np.save('../data/C16_raw/sigma_clip_array.npy', sigma_clip_array)
np.save('../data/C16_raw/n_pix_ap.npy', n_pix_ap)
np.save('../data/C16_raw/n_pix_bg.npy', n_pix_bg)
np.save('../data/C16_raw/ch_array.npy', ch_array)
np.save('../data/C16_raw/fail_array.npy', fail_array)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11472723015687193, 'fraction_numerical': 0.014282369468508546, 'mean_word_length': 3.454640250260688, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '152599', 'license': 'MIT', 'n_tokens_mistral': 1590, 'n_tokens_neox': 1513, 'n_words': 341, 'path': 'notebooks/K2_C16_PCA_pre_analysis.ipynb', 'repo_name': 'gully/goldenrod', 'size': '4271'}