<jupyter_start><jupyter_text>IPython Import and Processing of SFG Spectra Locate the directory where your Python files are stored.<jupyter_code>import sys
sys.path.append('/Users/geiger/Box Sync/Science/Experimental/Software/SFGprocessing/Python')<jupyter_output><empty_output><jupyter_text>Set directory to write files too:<jupyter_code>outputPath = '/Users/geiger/Box Sync/Science/Data/SFG/Solstice/2017/10162017'<jupyter_output><empty_output><jupyter_text>1. PS Calibration Locate the polystyrene calibration file by adjusting the path below. Upon execution, the polystyrene calibration file will be plotted:<jupyter_code>#locate ps calibration file here
psCalibFile = '/Users/geiger/Box Sync/Science/Data/SFG/Solstice/2017/10162017/goldenduse/ps_3400.spe'

#import
from pscalib import PScalib
ps = PScalib(psCalibFile)

#plot pscalibration
ps.plot()<jupyter_output><empty_output><jupyter_text>**Fit the first peak centered around 2850 wavenumbers:**<jupyter_code>#starting and ending values of segment to fit
val1 = 2825
val2 = 2860

#fit it with gaussian, calculate shift
ps.fitPeak(val1,val2,0)<jupyter_output>Peak shift from peak 0 is -8.96<jupyter_text>**Fit the second peak centered around 3060 wavenumbers:**<jupyter_code>#starting and ending values of segment to fit
val1 = 3040
val2 = 3060

#fit it with gaussian, calculate shift
ps.fitPeak(val1,val2,1)<jupyter_output>Peak shift from peak 1 is -11.16<jupyter_text>**Evaluate total shift as average of the two shifts:**<jupyter_code>calib = ps.evaluateShift()<jupyter_output>Average shift is  -10.06<jupyter_text>2. Import Gold Reference Spectrum **Input individual gold spectra:**<jupyter_code>#folder where the gold data is stored
inputgold = '/Users/geiger/Box Sync/Science/Data/SFG/Solstice/2017/10162017/goldenduse'

from spectrum import Spectrum
import os

#create object, loads each sample and background DFG
gold = Spectrum(inputgold,shift = calib)

#subtract the appropriate background DFG from each sample DFG
gold.subtractBGs()

#plot the imported sample DFGs
gold.plotDFGs()

#plot the imported background DFGs
gold.plotBGs()<jupyter_output>Importing DFGs and BGs...
Has dfgs:
2800
2900
3000
3100
3200
3300
3400
3500
Has bgs:
2800_bg
2900_bg
3000_bg
3100_bg
3400_bg
3500_bg
Subtracting BGs from DFGs...
For dfg 2800 found 2800_bg
For dfg 2900 found 2900_bg
For dfg 3000 found 3000_bg
For dfg 3100 found 3100_bg
For dfg 3200 found 3400_bg
For dfg 3300 found 3400_bg
For dfg 3400 found 3400_bg
For dfg 3500 found 3500_bg<jupyter_text>**Align individual DFG spectra along wavenumber axis:**<jupyter_code>#pad the dfgs with zeros so they align and can be summed up
gold.padDFGs()

#smooth the DFGs with a gaussian filter
gold.smoothDFGs(5)

#plot the smoothed and raw DFGs
gold.plotSmoothRawDFGs()<jupyter_output>Padding DFGs with Zeros...
Smoothing DFGs...<jupyter_text>**Truncate each spectrum where IR power dies off:**<jupyter_code>#threshold for truncation
trunc = 0.05

#find the indices of where the reference signal falls off to guide truncation
gold.findTruncateIndices(trunc)

#truncate 
gold.truncateFullDFGs(gold)

#plot the truncated DFGs
gold.plotTruncatedDFGs()<jupyter_output>Finding truncation thresholds at 0.05 ...
Truncating DFGs...<jupyter_text>**Sum the truncated spectra:**<jupyter_code>#sum the truncated DFGs
gold.sumTruncatedDFGs()

#plot the summed, truncated DFGS
gold.plotSumTruncatedDFG()<jupyter_output>Summing truncated DFGs...<jupyter_text>**Export Gold Data to File:**<jupyter_code>#write final summed, smoothed, padded, truncated spectrum to file
name = 'goldSummedTruncated.txt'

os.chdir(outputPath)
gold.writeSumTruncatedDFG(name)<jupyter_output>Truncated, summed wave written to goldSummedTruncated.txt<jupyter_text>3. Load Sample Spectra<jupyter_code>#folder where files are stored
samplepath = '/Users/geiger/Box Sync/Science/Data/SFG/Solstice/2017/10162017/lowsalt/run1'
#name
name = 'lowsalt1.txt'

cosmicrayThreshold = 100

spec = Spectrum(samplepath,shift=calib)

#plot after cosmic ray removal
spec.plotDFGs()
spec.plotBGs()

#remove cosmic rays
spec.removeCRs(cosmicrayThreshold)

#plot after cosmic ray removal
spec.plotDFGs()
spec.plotBGs()

#subtract background
spec.subtractBGs()

#plot after background subtraction
spec.plotDFGs()

#pad the dfgs with zeros so they align and can be summed up
spec.padDFGs()

#sum the padded DFGs
spec.sumFullDFGs()

#truncate the DFGs according to the gold reference spectrum
spec.truncateFullDFGs(gold)

#plot the truncated DFGs
spec.plotTruncatedDFGs()

#summ the truncated DFGs
spec.sumTruncatedDFGs()

#plot the summed, truncated spectrum
spec.plotSumTruncatedDFG()

#write the summed, truncated spectrum to file
spec.writeSumTruncatedDFG(name)<jupyter_output>Importing DFGs and BGs...
Has dfgs:
2800
2900
3000
3100
3200
3300
3400
3500
Has bgs:
2800_bg
2900_bg
3000_bg
3100_bg
3400_bg
3500_bg
Removing cosmic rays from spectra...
No spikes found in 2800
No spikes found in 2900
No spikes found in 3000
No spikes found in 3100
No spikes found in 3200
No spikes found in 3300
No spikes found in 3400
No spikes found in 3500
No spikes found in 2800_bg
No spikes found in 2900_bg
No spikes found in 3000_bg
No spikes found in 3100_bg
No spikes found in 3400_bg
No spikes found in 3500_bg
Subtracting BGs from DFGs...
For dfg 2800 found 2800_bg
For dfg 2900 found 2900_bg
For dfg 3000 found 3000_bg
For dfg 3100 found 3100_bg
For dfg 3200 found 3400_bg
For dfg 3300 found 3400_bg
For dfg 3400 found 3400_bg
For dfg 3500 found 3500_bg
Padding DFGs with Zeros...
Summing full DFGs...
Truncating DFGs...
Summing truncated DFGs...
Truncated, summed wave written to lowsalt1.txt
{'fraction_non_alphanumeric': 0.07535098631597655, 'fraction_numerical': 0.07304069664119424, 'mean_word_length': 5.880195599022005, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '90220', 'license': 'MIT', 'n_tokens_mistral': 2392, 'n_tokens_neox': 1939, 'n_words': 626, 'path': 'pythonmodules/pySfgProcess/sfgWorkup.ipynb', 'repo_name': 'MLackner/NUSFSpectra', 'size': '5627'}