<jupyter_start><jupyter_text>LST Coverage Shows the LSTs (in hours) and JDs for which data is collected. Green represents data, red means no data.<jupyter_code>utils.plot_lst_coverage(uvd)<jupyter_output><empty_output><jupyter_text>Delay spectrum Delay spectrum CLEANed using uvtools.dspec.high_pass_fourier_filter with 7th-order Blackman-Harris window function. Odd/even visibilities are used to remove noise bias.<jupyter_code>_data_cleaned_sq, d_even, d_odd = utils.clean_ds(bls, uvd, uvd_diff, N_threads=14)<jupyter_output><empty_output><jupyter_text>Waterfalls of delay spectra for autocorrelation These plots show autocorrelation delay spectrum waterfalls of each antenna that is active and whose status qualifies for this notebook. For nn/ee polarization, the autocorrelation delay spectrum is normalized by the max of the delay spectrum. For ne polarization, the autocorrelation delay spectrum is normalized by max(sqrt(|nn| * |ee|)). ne and en are the same for autocorrelations, and thus only ne is shown here. The delay spectra are presented in dB with 20*log10($|\tilde{V}|$).For each node, antennas are ordered by SNAP number, and within that by SNAP input number. The antenna number label color corresponds to the a priori status of that antenna.  nn polarization<jupyter_code>utils.plot_wfds(uvd, _data_cleaned_sq, 0)<jupyter_output><empty_output><jupyter_text>ee polarization<jupyter_code>utils.plot_wfds(uvd, _data_cleaned_sq, 1)<jupyter_output><empty_output><jupyter_text>ne polarization<jupyter_code>utils.plot_wfds(uvd, _data_cleaned_sq, 2)<jupyter_output><empty_output><jupyter_text>Analysis of 2700ns features in delay spectra This plot shows the relative amplitude at 2700 ns feature. The relative amplitude is calculated in dB with the mean amplitude at 2500-3000 ns compared to the mean amplitude at 2000-2500 ns. Larger values of relative feature amplitude indicate higher probability of detecting the peak at 2700 ns. Antennas in the same node are grouped by the shaded region.<jupyter_code>utils.plot_antFeatureMap_2700ns(uvd, _data_cleaned_sq, JD, pol='nn')
utils.plot_antFeatureMap_2700ns(uvd, _data_cleaned_sq, JD, pol='ee')<jupyter_output><empty_output><jupyter_text>This plot shows a matrix representing the 2700ns feature correlation of each baseline. The color bar indicates the amplitude of 2700ns (mean amplitude of 2500-3000ns delay spectrum) in dB which is the same as that in the above plot.<jupyter_code>utils.CorrMatrix_2700ns(uvd, HHfiles, difffiles, flagfile, JD, N_threads=14)<jupyter_output><empty_output><jupyter_text>Analysis of noise floor in delay spectra This plot shows the ratio of delay spectrum to noise floor (averaged over 1000-4000ns). Near 1 indicates the delay spectrum reaches to the noise floor, which may mean good.<jupyter_code>utils.plot_antFeatureMap_noise(uvd_diff, d_even, d_odd, JD, pol='nn')
utils.plot_antFeatureMap_noise(uvd_diff, d_even, d_odd, JD, pol='ee')
# get the ratio of delay spectum to noise for different freqeuncy bands and pols
ds_noise_ratio = utils.get_ds_noise_ratio(uvd, uvd_diff, bls)

nodes, antDict, inclNodes = utils.generate_nodeDict(uvd)
ants = uvd.get_ants()
# build dataframe
to_show = {'Ant': ants, 'Node': [int(antDict[ant]['node']) for ant in ants], 'Snap': [int(antDict[ant]['snapLocs'][0]) for ant in ants]}
df = pd.DataFrame(to_show)
 
cols_ratio = []
for key in ds_noise_ratio.keys():
    if(key[0] == 40):
        col = r'Full '
    else:
        col = r'{}-{} MHz '.format(key[0], key[1])
    col += key[2]
    df[col] = ds_noise_ratio[key]
    cols_ratio.append(col)
    

# sort by node number and then by antenna number within nodes
df.sort_values(['Node', 'Ant'], ascending=True)

ratio_cut = 3
# style dataframe
table = df.style.hide_index() \
          .applymap(lambda val: 'color: red' if val > ratio_cut else '', subset=cols_ratio) \
          .set_table_styles([dict(selector="th",props=[('max-width', f'70pt')])])<jupyter_output><empty_output><jupyter_text>This table shows the ratio of the delay spectrum to the noise level from diff files for different frequency bands and pols. The ratio > 3 is colored in red<jupyter_code>HTML(table.render())
csv_file = os.path.join(nb_outdir, 'ds_noise_ratio_{}.csv'.format(JD))
df.to_csv(csv_file, index=False)<jupyter_output><empty_output><jupyter_text>Delay spectrum and autocorrelation plot per baseline per polarization for a given frequency (sub-)band Left panel: time averaged delay spectum of autocorrelation in dB with 20*log10($|\tilde{V}|$) (blue) and noise from diff file representing the expected variance of the delay spectrum (red)    Right panel: time averaged autocorrelations w/o (orange) and w/ xRFI flags (blue). Flagged one is shifted from the unflagged one for clarity<jupyter_code>utils.interactive_plots_dspec(bls, uvd, uvd_diff, JD)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09004543577034284, 'fraction_numerical': 0.01900041305245766, 'mean_word_length': 6.132547864506628, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 43, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '41450', 'license': 'MIT', 'n_tokens_mistral': 1648, 'n_tokens_neox': 1510, 'n_words': 513, 'path': 'notebooks/delay_spectrum_inspect.ipynb', 'repo_name': 'HERA-Team/hera_notebook_templates', 'size': '4842'}