<jupyter_start><jupyter_text>Scan log alignment summaryThis notebook takes the `'%s_map.yml'` outpute file from `'48spot alignment-Two_Mantas_load_scan_data-datetime.ipynb'` and tidies it into DataFrames for GG, GR, and RR multiscans. Note that `'Batch 48spot alignment.ipynb'` must be run before running this notebook.<jupyter_code>import ruamel.yaml as ruamel_yaml
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from cycler import cycler
import palettable as pal
import seaborn as sns
%matplotlib inline
sns.set_style('whitegrid')
colors = pal.colorbrewer.qualitative.Dark2_8
colors.show_as_blocks(50)
plt.rc('axes', prop_cycle=(cycler('color', colors.mpl_colors)))<jupyter_output><empty_output><jupyter_text>List of scan dataThe scan parameters are univoque but other scan data such as sigmas and max counts depends on data selection.Median is used to minimize effect of outliers and because we want to maximaze an effect for the majority of spots.Moreover, filtering along appropriate "features" is used to remove strong outliersIn this series of scan data, 'offset' is used to exclude outliers for sigma computation.<jupyter_code>filenames = ['two_mantas_map.yml']
d_list = []
for filename in filenames:
    with open(filename) as f:
        scan_data = ruamel_yaml.load(f)
        for v in scan_data.values():
            del v['fit_result_text']
    dx = pd.DataFrame.from_dict(scan_data, orient='index')
    d_list.append(dx)
dx.tail()
dx.columns
dx<jupyter_output><empty_output><jupyter_text>Concatenate Results<jupyter_code>input_col = ['center_x', 'center_y', 'f_min', 'integration_ms', 'nspots_x', 'nspots_y', 'offset_min', 
             'pitch_x', 'pitch_y','range', 'residue_max', 'step', 'z']
output_col = ['center_x_fit', 'center_y_fit', 'max_counts_corr', 'n_inliers', 
              'pitch_x_fit', 'pitch_y_fit', 'pitch_x_fitrot', 'pitch_y_fitrot', 
              'sigma_x', 'sigma_y', 'intensity', 'z', 'datetime']
dfg = (pd.concat(d_list)
      .assign(intensity=lambda x: x.max_counts_corr / x.integration_ms)
      .assign(datetime=lambda x: pd.to_datetime(x.datetime, format="%Y-%m-%d %H-%M-%S"))
      .query('ch == "G"')
      .query('laser == "G"')
     ).iloc[-3:]
dfg['intensity'] = dfg.max_counts_corr / dfg.integration_ms
dfr = (pd.concat(d_list)
      .assign(intensity=lambda x: x.max_counts_corr / x.integration_ms)
      .assign(datetime=lambda x: pd.to_datetime(x.datetime, format="%Y-%m-%d %H-%M-%S"))
      .query('ch == "R"')
      .query('laser == "G"')       
     ).iloc[-3:]
dfr['intensity'] = dfr.max_counts_corr / dfr.integration_ms
dfrr = (pd.concat(d_list)
      .assign(intensity=lambda x: x.max_counts_corr / x.integration_ms)
      .assign(datetime=lambda x: pd.to_datetime(x.datetime, format="%Y-%m-%d %H-%M-%S"))
      .query('ch == "R"')
      .query('laser == "R"')       
     ).iloc[-3:]
dfrr['intensity'] = dfrr.max_counts_corr / dfrr.integration_ms
dfg[input_col]
dfg[output_col]
dfr[input_col]
dfr[output_col]
dfrr[output_col]<jupyter_output><empty_output><jupyter_text>Alignment plots  Donor SPAD fitted (x,y) pitch Donor SPAD under 532nm excitation.<jupyter_code>dfg.pitch_x_fit.plot(marker='o', rot=45)
dfg.pitch_y_fit.plot(marker='s', rot=45)
plt.ylim(23, 24.5)
plt.legend()
plt.title('Donor SPAD Fitted pitch');<jupyter_output><empty_output><jupyter_text>Acceptor SPAD fitted (x,y) pitch Acceptor SPAD under 532nm excitation.<jupyter_code>dfr.pitch_x_fit.plot(marker='o', rot=45)
dfr.pitch_y_fit.plot(marker='s', rot=45)
plt.ylim(23, 24.5)
plt.legend()
plt.title('Acceptor SPAD Fitted pitch');<jupyter_output><empty_output><jupyter_text>Acceptor SPAD fitted (x,y) pitch Acceptor SPAD under 628nm excitation.<jupyter_code>dfrr.pitch_x_fit.plot(marker='o', rot=45)
dfrr.pitch_y_fit.plot(marker='s', rot=45)
plt.ylim(22.5, 23)
plt.legend()
plt.title('Acceptor SPAD Fitted pitch @628nm');<jupyter_output><empty_output><jupyter_text>Fitted LCOS positions  Pitch X, Green LCOS<jupyter_code>accept_pitch_x = dfr.pitch_x_fit.mean()
accept_pitch_x
accept_pitch_xrot = dfr.pitch_x_fitrot.mean()
accept_pitch_xrot
donor_pitch_x = dfg.pitch_x_fit.mean()
donor_pitch_x<jupyter_output><empty_output><jupyter_text>Mean X-pitchAn average of two pitches from fitted multiscan results.<jupyter_code>donor_pitch_xrot = dfg.pitch_x_fitrot.mean()
donor_pitch_xrot
np.round(0.5*(donor_pitch_x + accept_pitch_x), 2)<jupyter_output><empty_output><jupyter_text>Pitch Y, Green LCOS<jupyter_code>accept_pitch_y = dfr.pitch_y_fit.mean()
accept_pitch_y
accept_pitch_yrot = dfr.pitch_y_fitrot.mean()
accept_pitch_yrot
donor_pitch_y = dfg.pitch_y_fit.mean()
donor_pitch_y
donor_pitch_yrot = dfg.pitch_y_fitrot.mean()
donor_pitch_yrot
np.round(0.5*(donor_pitch_y + accept_pitch_y), 2)<jupyter_output><empty_output><jupyter_text>Red LCOS<jupyter_code>Raccept_pitch_x = dfrr.pitch_x_fit.mean()
Raccept_pitch_x
Raccept_pitch_xrot = dfrr.pitch_x_fitrot.mean()
Raccept_pitch_xrot
Raccept_pitch_y = dfrr.pitch_y_fit.mean()
Raccept_pitch_y
Raccept_pitch_yrot = dfrr.pitch_y_fitrot.mean()
Raccept_pitch_yrot<jupyter_output><empty_output><jupyter_text>RotationRotation of the SPADs over time<jupyter_code>dfg.rotation.plot(marker='o', rot=45)
dfr.rotation.plot(marker='o', rot=45)
plt.title('SPADs Rotation @ 532nm');
plt.ylim(1, 2)
plt.legend(['Donor SPAD', 'Acceptor SPAD'])
dfg.rotation.mean(), dfr.rotation.mean(), dfrr.rotation.mean()
np.mean([dfg.rotation.mean(), dfr.rotation.mean()])
dfrr.rotation.plot(marker='o', rot=45)
plt.title('A SPAD Rotation @ 628nm');
plt.ylim(-1, -2)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.138104657435713, 'fraction_numerical': 0.012947311634598094, 'mean_word_length': 6.0227272727272725, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '411723', 'license': 'MIT', 'n_tokens_mistral': 2316, 'n_tokens_neox': 2147, 'n_words': 398, 'path': 'alignment/2017-05-02/Alignment Summary.ipynb', 'repo_name': 'tritemio/48-spot-smFRET-PAX-analysis', 'size': '5561'}