<jupyter_start><jupyter_text>Time Series DemoWritten by <NAME>This notebook will demonstrate the use of `deeplenstronomy` to create a time series of images.<jupyter_code>import deeplenstronomy.deeplenstronomy as dl<jupyter_output><empty_output><jupyter_text>For this example, we'll use the `demo_timeseries.yaml` config file, which already has timeseries functionality built in. It is the same demo file we have been using, but we have updated the entries for `SPECIES.POINTSOURCE_2` and `CONFIGURATION_3` to include the temporal evolution of the supernova in that system.<jupyter_code>config_file = 'data/demo_timeseries.yaml'
! cat data/demo_timeseries.yaml<jupyter_output>DATASET:
    NAME: MyDeeplenstronomyDataset
    PARAMETERS:
        SIZE: 100
        OUTDIR: MySimulationResultsTimeSeries

COSMOLOGY:
    PARAMETERS:
        H0: 70
        Om0: 0.3

IMAGE:
    PARAMETERS:
        exposure_time: 
            DISTRIBUTION:
                NAME: uniform
                PARAMETERS:
                    minimum: 60
                    maximum: 120
        numPix: 100
        pixel_scale: 0.263
        psf_type: 'GAUSSIAN'
        read_noise: 7
        ccd_gain: 6.083

SURVEY:
    PARAMETERS:
        BANDS: g,r,i,z,Y
        seeing: 0.9
        magnitude_zero_point: 30.0
        sky_brightness: 23.5
        num_exposures: 1

SPECIES:
    GALAXY_1:
        NAME: LENS
        LIGHT_PROFILE_1:
            NAME: SERSIC_ELLIPSE
            PARAMETERS:
                magnitude: 19.5
                center_x: 0.0
                center_y: 0.0
                R_sersic: 10
                n_sersic: 4
                e1: [...]<jupyter_text>_Note_: `SURVEY.num_exposures` must be set to 1 (to better reflect real time-series data) and cannot be drawn from a distribution if you are using any time-series features.  Configuration File Keywords - MODELCheck out the information in the `SPECIES` section for `POINTSOURCE_2`. You will notice the `MODEL` keyword is new and has not been used in other examples. Next, take a look at `CONFIGURATION_3`; you should see an entry for `TIMESERIES`. These two additions to the config file are all you need to utilize the time series features of `deeplenstronomy`.Let's start with the `MODEL` keyword in the `SPECIES` section. The `MODEL` points `deeplenstronomy` to the type of temporal behavior you want to add to the object. Time series toy models The basic options for time series models are:- `MODEL: flat` for no evolution of brightness in time, but `deeplenstronomy` will still simulate the image under different observing conditions in each of the nites requested- `MODEL: flatnoise` for no real evolution of brightness in time, but a small amount of noise is added to the measured brightness of the object on each nite requested- `MODEL: variable` a central magnitude is chosen from the range 13-23 mag, a small color is intruduced between the different bands, and the magnitudee each nite is chosen from a $\pm$ 1 mag interval around the central magnitude- `MODEL: variablenoise` the variable model is used, but a small amount of noise is added to the measured brightness on top of the variation. Time series spectral modelsAside from these toy time-series models, `deeplenstronomy` has real objects built in as well. You also have the ability to simulate supernova and kilonova light curves. Under the hood, `deeplenstronomy` has a time series spectral energy distribution (SED) for type-Ia supernovae, core-collapse supernovae, and the kilonova counterpart to GW170817. When these models are called, `deeplenstronomy` redshifts the spectra, integrates the spectra through the bands of the survey, and produces a magnitude in each band on each nite.Specifically, these SEDs are automatically downloaded when you request time-series data for the first time. They are:<jupyter_code>! echo "SNe-CC SED filenames\n" & cat seds/cc/NON1A.LIST
! echo "SNe-Ia SED filename\n" & ls seds/ia<jupyter_output>SNe-Ia SED filename\n
salt2_template_0.dat     snflux_1a_Nugent2002.dat<jupyter_text>The GW170817 model is just a single file located in `seds/kn/kn.SED` The reason for diving into the raw files is that `deeplenstronomy` allows you to either choose randomly from all the available SEDs for a given object, or allows you to specify the particular file of interest.- `MODEL: ia_random` will choose a single SED file at random from the available SNe-Ia SEDs and use it- `MODEL: cc_random` will choose a single SED file at random from the available SNe-CC SEDs and use itTo use a specific SNe-CC or SNe-Ia model, you replace `random` with the filename you want:- `MODEL: ia_salt2-template-0.dat` for the well-established SALT2 SN-Ia model,- `MODEL: cc_SDSS-003818.SED` for a particular SN-IIP observed by SDSS, etc.The GW170817 kilonova model is targeted by `MODEL: kn_kn.SED` to match the syntax of the other models, even though there is currently only one model.  User-specified time series modelsLike other features in `deeplenstronomy` (image background, probability distributions, etc.) the user has the ability to incorporate their own data into the simulations. To use your own file, format a SED file like the built-in `deeplenstronomy` files:<jupyter_code>! head seds/ia/salt2-template-0.dat<jupyter_output>head: seds/ia/salt2-template-0.dat: No such file or directory<jupyter_text>The columns (from left to right) are days relative to the date of peak flux, the wavelength in angstroms, and the flux in erg / cm$^2$ / sec.If you are not simulating data using Dark Energy Survey bands, you will need to also supply transmission curevs for all filters in your survey in the `filters/` directory while following all formatting of the provided transmission curves.To target your model from the `deeplenstronomy` configuration file, you use the `MODEL` keyword as follows:- `MODEL: user_`  Configuration File Keywords - TIMESERIESThe final step in including time series behavior in your dataset is to specify which object in which configuration you want to "time-series-ify", and the temporal spacing of the observations.In the `demo_timeseries.yaml` file, we have done:```    CONFIGURATION_3:        NAME: LENSED_SNE        FRACTION: 0.25        PLANE_1:            OBJECT_1: LENS            PARAMETERS:                REDSHIFT: 0.2                            PLANE_2:            OBJECT_1: SOURCE            OBJECT_2: SUPERNOVA            PARAMETERS:                REDSHIFT: 0.7              NOISE_SOURCE_1: POISSON_NOISE           TIMESERIES:            NITES: [-10, -5, 0, 5, 10]            OBJECTS: ['SUPERNOVA']```The name `SUPERNOVA` is the `NAME` of `POINTSOURCE_2` in the `SPECIES` section, and listed as `OBJECT_2` in `PLANE_2` of `CONFIGURATION_3`. `deeplenstronomy` uses this `NAME` value to connect the properties of the object to its temporal behavior. Making sure the `NAME` value is connected across these sections is essential for `deeplenstronomy` to work properly.The remaining `TIMESERIES` attributes have the following meanings:- `NITES` Suppy a list of times (in units of days relative to the date of peak flux) to define the temporal spacing of the observations- `OBJECTS` Supply a list of names (as strings) of entries in the `SPECIES` section for which the `MODEL` key has already been specified  What About Quasars???Searching for lensed quasars is a popular science case at the interface of time-series data and strong lensing. However, `deeplenstronomy`'s time-series functionality is mainly built to simulate explosive objects, not persistent variable objects, so the use of time-series functionality to simulate lensed quasars is discouraged.That being said, detections of lens quasars from images are usually made by detecting the system in a single image and then obtaining further observations to characterize the light curve of the potential quasar images, as opposed to detecting them only through a time-series set of images. For this purpose, `deeplenstronomy`'s `USERDIST` functionality is exactly what you're looking for!Through specifying a `USERDIST` you can readily put quasar color distributions into your distributions to get realistic single images of lensed quasars.  Generating the DatasetIf this is your first time simulating a particular class of time series object, `deeplenstronomy` will need to quickly download some data. It will automatically get what it needs from my GitHub, and it will target my [deeplenstronomy_data](https://github.com/rmorgan10/deeplenstronomy_data) repo.The data it will download are time-series spectra and the transmission curves for whichever filters you specified.When you run in time series mode, `deeplenstronomy` will build up a light curve bank while processing your configuration file and before it begins simulating images. This extra step may take a few minutes to run, but will only need to occur at the beginning of the simulation.<jupyter_code>dataset = dl.make_dataset(config_file, verbose=True)<jupyter_output>Generating time series data for CONFIGURATION_3
Entering main organization loop
Organizing CONFIGURATION_1
Organizing CONFIGURATION_2
Organizing CONFIGURATION_3
Organizing CONFIGURATION_4
Generating images for CONFIGURATION_1
	Progress: 100.0 %  ---  Elapsed Time: 0 H 0 M 5 S
Generating images for CONFIGURATION_2
	Progress: 100.0 %  ---  Elapsed Time: 0 H 0 M 4 S
Generating images for CONFIGURATION_3
	Progress: 100.0 %  ---  Elapsed Time: 0 H 0 M 26 S
Generating images for CONFIGURATION_4
	Progress: 100.0 %  ---  Elapsed Time: 0 H 0 M 34 S<jupyter_text>In the config file, the time series objects were placed in `CONFIGURATION_3`, so let's focus there for this example. The images for the entire configuration will all be stacked in the `dataset.CONFIGURATION_3_images`, so therefore it is a good idea to use the `OBJID` columns in `dataset.CONFIGURATION_3_metadata` to track the images that are part of the same time series.<jupyter_code>print(dataset.CONFIGURATION_3_metadata['OBJID-g'].values)
import numpy as np
# Get the indices of all the images corresponding to OBJID = 10
obj_indices = (dataset.CONFIGURATION_3_metadata['OBJID-g'].values == 10).nonzero()[0]
print(obj_indices)<jupyter_output>[50 51 52 53 54]<jupyter_text>Now let's visualize the time-evolution of the object<jupyter_code>from deeplenstronomy.visualize import view_image
for idx in obj_indices:
    print("NITE: ", dataset.CONFIGURATION_3_metadata['PLANE_2-OBJECT_2-nite-g'].values[idx])
    print("     g         r         i         z         Y")
    view_image(dataset.CONFIGURATION_3_images[idx])<jupyter_output>NITE:  -10
     g         r         i         z         Y<jupyter_text>We can also access and plot the light curve of the point source like this:<jupyter_code>import matplotlib.pyplot as plt
plt.figure()

color_dict = {'g': '#984ea3', 'r': '#377eb8', 'i': '#4daf4a', 'z': '#ff7f00', 'Y': '#e41a1c'}

for band in ['g', 'r', 'i', 'z', 'Y']:
    times = dataset.CONFIGURATION_3_metadata['PLANE_2-OBJECT_2-nite-{0}'.format(band)].values[obj_indices]
    mags = dataset.CONFIGURATION_3_metadata['PLANE_2-OBJECT_2-magnitude-{0}'.format(band)].values[obj_indices]
    
    plt.plot(times, mags, color=color_dict[band], label=band)
    
plt.legend(fontsize=12, ncol=5)
plt.ylim(22, 16)
plt.xticks(fontsize=14)
plt.yticks(fontsize=14)
plt.xlabel("Days relative to peak flux", fontsize=14)
plt.ylabel("Magnitude", fontsize=14)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06599738105630729, 'fraction_numerical': 0.019991270187690965, 'mean_word_length': 3.8624787775891343, 'pattern_counts': {'":': 0, '<': 28, '<?xml version=': 0, '>': 28, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '431270', 'license': 'MIT', 'n_tokens_mistral': 3549, 'n_tokens_neox': 3253, 'n_words': 1408, 'path': 'Notebooks/TimeSeriesDemo.ipynb', 'repo_name': 'erikzaborowski/deeplenstronomy', 'size': '11462'}