<jupyter_start><jupyter_text>This notebook is a short and sweet demo showing how we can use roboscheduler's cadence methods to load a set of cadences from a .cfg file, and write them out as a FITS file. In the future, we may want to edit to show how we can even load them straight into targetdb, but not yet.First step - import packages we'll need:<jupyter_code>##### import roboscheduler, and its cadence method
import roboscheduler
from roboscheduler import cadence

##### also import fitsio, to write out the cadences to a FITS file.
import fitsio
import csv

##### and import sdssdb to load back to the database
import sdssdb
from sdssdb.peewee.sdss5db import database

##### import numpy to construct the arrays for the rm cadence
import numpy as np<jupyter_output><empty_output><jupyter_text>next step: open a blank cadence list that we'll use to hold the cadences<jupyter_code>clist = cadence.CadenceList()
clist.reset()<jupyter_output><empty_output><jupyter_text>Now, use the 'fromcfg' method to import the cadences from the existing [defaultCadences.cfg](https://github.com/sdss/roboDocs/blob/main/cadences/defaultCadences.cfg) file<jupyter_code>clist.fromcfg('defaultCadences_v1.cfg')
### uncomment if you want to convince yourself that bright_8x1 and bright_8x4 are incompatible as 
### recorded in brokenCadences_v0.5.cfg
#clist.fromcfg('brokenCadences_v0.5.cfg')<jupyter_output><empty_output><jupyter_text>now export the cadences to an array, and print out the first element in that array to check that it makes sense.<jupyter_code>cadenceArray = clist.toarray()
print(cadenceArray[0])<jupyter_output>('bright_1x1_v1', 1, [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [b'bright_time', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b'', b''], 'bright_1x1', '_v1')<jupyter_text>Despite the large number of zeros, that is actually what we'd expect -- that cadence format consists of:the cadence name: 'bright_1x1'the number of epochs: 1the array of delta values associated with each epoch: '[ 0., nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]'(the nan values are placeholders to signal that the epoch is not active/meaningful, and thus to ignore all the delta min/max etc. info)the array of sky brightness limits for each epoch: '[1., nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]'the array of delta_min values for each epoch: '[0., nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]'the array of delta_max values for each epoch: '[0., nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]'the array listing the number of exposures requested for each epoch: '[1, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]'the array of max_length values for each epoch: '[0., nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]'Why are the arrays so long for a 1x1 cadence?  Because the toarray() method has to size the array according to the longest cadence it contains (in this case, a bright_18x4 cadence, so all arrays have spaces for 18 epochs, even if most of those epochs are non-active) Now let's check some cadences for consistency.<jupyter_code>print(cadenceArray[17])
print(cadenceArray[19])
clist.cadence_consistency('bright_8x1_v1', 'bright_8x4_v1')
clist.cadence_consistency('bright_9x1_v1', 'dark_10x4_4yr_v1')
clist.cadence_consistency('bright_18x1_v1', 'mixed2_single_22x1_v1')<jupyter_output><empty_output><jupyter_text>ah, right -- to keep the file simple, we only have the target level cadences in the defaultCadences_v0.5.cfg file.  If we want to load the field level cadences like the mixed2_single or bright_single cadences, we'll need to load those separately.  Thankfully <NAME> has written a handy script (mixed2_single_cadence_generator.py) that we can use to generate the mixed2_single_cadences_v0.5.1.cfg file.  Let's read that in:<jupyter_code>clist.fromcfg('mixed2_single_cadences_v1.cfg')<jupyter_output><empty_output><jupyter_text>now let's re-do that test from above...<jupyter_code>clist.cadence_consistency('bright_18x1_v1', 'mixed2_single_22x1_v1')<jupyter_output><empty_output><jupyter_text>and it works now!  Great.  Let's also read in the bright single cadences (which we can generate using a only slightly hacked version of Joleen's script, saved in bright_single_cadence_generator.py):<jupyter_code>clist.fromcfg('bright_single_cadences_v1.cfg')<jupyter_output><empty_output><jupyter_text>and now let's test that the bright single epochs fit within a mixed2 cadence that is slightly longer (since the bright cadence won't fit in the dark epochs of the mixed cadence).<jupyter_code>clist.cadence_consistency('bright_single_20x1_v1', 'mixed2_single_22x1_v1')<jupyter_output><empty_output><jupyter_text>now let's set up a default RM cadence.  We'll try to make this follow a front-loaded cadence structure, just to show how we can make a non-uniform cadence, but robostrategy won't sweat the details, as long as the number of epochs are correct.<jupyter_code>def RM_obsmode(n_epochs, obsmode):
    
    obsmodes = np.array(obsmode, dtype=str)
    
    for i in range(n_epochs-1):
        obsmodes = np.append(obsmodes, obsmode)
        #print(obsmodes)
    #print(len(obsmodes))
    return obsmodes
#begin building a default (front-loaded) RM cadence

rm_label_root = 'dark_174x8'
rm_label_version = '_v1'

rm_obsmodes = RM_obsmode(174, 'dark_rm')

#set up the skybrightness and nexp constraints (which are always 0.35 and 8, respectively)
RM_bright = np.ones(174)*0.35
RM_exps = np.ones(174, dtype=np.int32)*8

print(RM_exps)
def RM_timing(n_dense, dense_gap, n_sparse, sparse_gap, seasonal_gap):
    
    #initially, n_dense = 61, dense_gap = 3., n_sparse = 22, sparse_gap = 7, seasonal gap = 183
    
    #set up a season of epochs separated by three days 
    #(in the future we'll have to add gaps for bright runs, but let's keep it simple for now)
    RMdeltas_denselyPacked = np.ones(n_dense)*dense_gap
    RMdeltaMins_denselyPacked = np.ones(n_dense)*dense_gap/2.
    RMdeltaMaxes_denselyPacked = np.ones(n_dense)*dense_gap*1.5
    
    #add on the seasonal gap
    RMdeltas = np.append(RMdeltas_denselyPacked, seasonal_gap)
    RMdeltaMins = np.append(RMdeltaMins_denselyPacked, seasonal_gap-3)
    RMdeltaMaxes = np.append(RMdeltaMaxes_denselyPacked, seasonal_gap+3)

    #now add on another season (with the gap!))
    RMdeltas = np.append(RMdeltas, RMdeltas)
    RMdeltaMins = np.append(RMdeltaMins, RMdeltaMins)
    RMdeltaMaxes = np.append(RMdeltaMaxes, RMdeltaMaxes)

    #now fill in the maxlengths for both seasons (123 epochs)
    maxlengths = np.ones((n_dense+1)*2)*(dense_gap-0.5)
    
    
    #now add on some sparser seasons
    RMdeltas_looselyPacked = np.ones(n_sparse)*sparse_gap
    RMdeltaMins_looselyPacked = np.ones(n_sparse)*sparse_gap/2.
    RMdeltaMaxes_looselyPacked = np.ones(n_sparse)*(sparse_gap+3)

    #and append first of two sparsely packed seasons
    RMdeltas = np.append(RMdeltas, RMdeltas_looselyPacked)
    RMdeltaMins = np.append(RMdeltaMins, RMdeltaMins_looselyPacked)
    RMdeltaMaxes = np.append(RMdeltaMaxes, RMdeltaMaxes_looselyPacked)

    #add on the seasonal gap
    RMdeltas = np.append(RMdeltas, seasonal_gap)
    RMdeltaMins = np.append(RMdeltaMins, seasonal_gap-3)
    RMdeltaMaxes = np.append(RMdeltaMaxes, seasonal_gap+3)

    #and append another sparsely packed season
    RMdeltas = np.append(RMdeltas, RMdeltas_looselyPacked)
    RMdeltaMins = np.append(RMdeltaMins, RMdeltaMins_looselyPacked)
    RMdeltaMaxes = np.append(RMdeltaMaxes, RMdeltaMaxes_looselyPacked)

    #add on the seasonal gap
    RMdeltas = np.append(RMdeltas, seasonal_gap)
    RMdeltaMins = np.append(RMdeltaMins, seasonal_gap-3)
    RMdeltaMaxes = np.append(RMdeltaMaxes, seasonal_gap+3)

    #add the longer max lengths
    more_maxlengths = np.ones( (n_sparse+1)*2)*(sparse_gap+2)
    RM_maxlengths = np.append(maxlengths, more_maxlengths)

    
    #and fill out the rest of the array
    n_tail = 4
    tail_gap = 30.
    
    #and finally another 4 epochs spaced one month apart
    RMdeltas_loosestPacked = np.ones(n_tail)*tail_gap
    RMdeltaMins_loosestPacked = np.ones(n_tail)*tail_gap/2.
    RMdeltaMaxes_loosestPacked = np.ones(n_tail)*tail_gap*1.5

    #and append the last season
    RMdeltas = np.append(RMdeltas, RMdeltas_loosestPacked)
    RMdeltaMins = np.append(RMdeltaMins, RMdeltaMins_loosestPacked)
    RMdeltaMaxes = np.append(RMdeltaMaxes, RMdeltaMaxes_loosestPacked)

    last_maxlengths = np.ones(n_tail)*tail_gap
    RM_maxlengths = np.append(RM_maxlengths, last_maxlengths)

    #check that everything turned out the right length
    #print(len(RM_maxlengths))
    #print(len(RMdeltas))
    
    return RMdeltas, RMdeltaMins, RMdeltaMaxes, RM_maxlengths
n_dense = 61
dense_gap = 3.
n_sparse = 22
sparse_gap = 7.
seasonal_gap = 183.

RMdeltas, RMdeltaMins, RMdeltaMaxes, RM_maxlengths = RM_timing(n_dense, dense_gap, n_sparse, sparse_gap, seasonal_gap)
#now let's add this RM cadence to our cadence list
clist.add_cadence(name=rm_label_root+rm_label_version, nepochs = 174, nexp = RM_exps, skybrightness = RM_bright, 
                            delta = RMdeltas, delta_min = RMdeltaMins, delta_max = RMdeltaMaxes, 
                            max_length = RM_maxlengths, obsmode_pk = rm_obsmodes, 
                            label_root = rm_label_root, label_version = rm_label_version)
clist.cadence_consistency('dark_10x4_4yr_v1', 'dark_174x8_v1')
#now lets add an RM lite cadence

rm_lite_label_root = 'dark_100x8'
rm_lite_label_version = '_v1'

rm_lite_obsmodes = RM_obsmode(100, 'dark_rm')


#set up the skybrightness and nexp constraints (which are always 0.35 and 8, respectively)
RM_lite_bright = np.ones(100)*0.35
RM_lite_exps = np.ones(100, dtype=np.int32)*8

#set up the timing constraints
n_lite_dense = 32
dense_lite_gap = 5.
n_lite_sparse = 14
sparse_lite_gap = 10.
seasonal_lite_gap = 183.

#fill out the arrays
RMdeltas_lite, RMdeltaMins_lite, RMdeltaMaxes_lite, RM_maxlengths_lite = RM_timing(n_lite_dense, dense_lite_gap, n_lite_sparse, sparse_lite_gap, seasonal_lite_gap)

#now let's add this RM cadence to our cadence list
clist.add_cadence(name=rm_lite_label_root+rm_lite_label_version, nepochs = 100, nexp = RM_lite_exps, 
                  skybrightness = RM_lite_bright, delta = RMdeltas_lite, delta_min = RMdeltaMins_lite, 
                  delta_max = RMdeltaMaxes_lite, max_length = RM_maxlengths_lite, obsmode_pk = rm_lite_obsmodes, 
                            label_root = rm_lite_label_root, label_version = rm_lite_label_version)
clist.cadence_consistency('dark_100x8_v1', 'dark_174x8_v1')
clist.cadence_consistency('bright_5x1_v1', 'dark_10x4_4yr_v1')<jupyter_output><empty_output><jupyter_text>Now let's write this cadence array out to a FITS file for easy transportation:<jupyter_code>FinalCadenceArray = clist.toarray()

print(len(FinalCadenceArray))

#now let's replace all the non-active zero deltas with -1
print(FinalCadenceArray[39])

#np.shape(FinalCadenceArray)
#TrimmedArray = np.delete(FinalCadenceArray, 9, 1)
#print(FinalCadenceArray[41])
fitsio.write("DefaultCadences_v1.fits", FinalCadenceArray)<jupyter_output><empty_output><jupyter_text>Success!  This command should produce the [DefaultCadences.fits file](https://github.com/sdss/roboDocs/blob/main/cadences/DefaultCadences.fits) that lives next to this notebook.As telegraphed above, we could then use some of the functionality roboscheduler provides to load these cadences into the targeting database, but that would require getting credentials etc. correct, so let's leave that for later... Now let's print to a csv so that Pramod can import easily...<jupyter_code>clist.tocsv('defaultCadences_v1.csv')<jupyter_output>"bright_time"
"bright_time"
"bright_time"
"bright_time"
"bright_time"
"bright_time"
"bright_time"
"bright_time"
"bright_time","bright_time"
"bright_time","bright_time","bright_time"
"bright_time","bright_time","bright_time"
"bright_time","bright_time","bright_time"
"bright_time","bright_time","bright_time","bright_time"
"bright_time","bright_time","bright_time","bright_time","bright_time"
"bright_time","bright_time","bright_time","bright_time","bright_time","
 bright_time"
"bright_time","bright_time","bright_time","bright_time","bright_time","
 bright_time"
"bright_time","bright_time","bright_time","bright_time","bright_time","
 bright_time"
"bright_time","bright_time","bright_time","bright_time","bright_time","
 bright_time","bright_time","bright_time"
"bright_time","bright_time","bright_time","bright_time","bright_time","
 bright_time","bright_time","bright_time"
"bright_time","bright_time","bright_time","bright_time","bright_time","
 bright_time","bright_time","bright_time"
"bright_[...]
{'fraction_non_alphanumeric': 0.11273769996981588, 'fraction_numerical': 0.02618472683368548, 'mean_word_length': 5.09894155545329, 'pattern_counts': {'":': 0, '<': 48, '<?xml version=': 0, '>': 48, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 28, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '458957', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 5287, 'n_tokens_neox': 4693, 'n_words': 1301, 'path': 'schema/sdss5db/targetdb/data_table/cadence/LoadDefaultCadences_v1.ipynb', 'repo_name': 'sdss/sdssdb', 'size': '13261'}