<jupyter_start><jupyter_text>2019-07-26: Testing standardize-specta Authors* <NAME> (<EMAIL>) Notes* In this Jupyter notebook, we check that standardize spectra is correctly filling in missing values for spectrum id=2714  - ChapterM_Minerals/splib07a_Chlorite_SMR-13.c_45-60um_BECKa_AREF.txt  Preparations<jupyter_code># --- Imports

# Standard libraries
import os

# External packages
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

# SpectraML
from spectra_ml import data
from spectra_ml import io
# --- Configuration Parameters

# Data directories
data_dir = os.environ['DATA_DIR']
splib07a_dir = os.path.join(data_dir, 'ASCIIdata_splib07a')
splib07a_standardized_dir = os.path.join(data_dir, 'ASCIIdata_splib07a.standardized')
spectrometers_dir = os.path.join(data_dir, 'spectrometers')

# Spectrometers
spectrometers = io.load_spectrometers(spectrometers_dir, splib07a_dir)

# Test Spectrum
raw_spectrum_path = os.path.join(splib07a_dir, 'ChapterM_Minerals', 'splib07a_Chlorite_SMR-13.c_45-60um_BECKa_AREF.txt')
standardized_spectrum_path = os.path.join(splib07a_standardized_dir, '2714.csv')<jupyter_output><empty_output><jupyter_text>Read and Plot Spectra Data<jupyter_code># --- Read spectra data

raw_spectrum = pd.read_csv(raw_spectrum_path)
spectrometer_wavelengths = spectrometers['BECK']['x-axis']['wavelength']['values']

standardized_spectrum = pd.read_csv(standardized_spectrum_path)
standardized_wavelengths = standardized_spectrum.wavelength

# --- Plot raw and filled-in spectra

plt.figure()
plt.subplot(2, 1, 1)
plt.plot(spectrometer_wavelengths, raw_spectrum)
plt.show()
    
plt.subplot(2, 1, 2)
plt.plot(standardized_wavelengths, standardized_spectrum.reflectance)
    
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1012514220705347, 'fraction_numerical': 0.02957906712172924, 'mean_word_length': 7.181395348837209, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '629175', 'license': 'Apache-2.0', 'n_tokens_mistral': 678, 'n_tokens_neox': 624, 'n_words': 116, 'path': 'lab-notebook/kchu/2019-07-26-KTC-testing_standardize_spectra.ipynb', 'repo_name': 'velexi-corporation/spectra-ml', 'size': '1770'}