<commit_before>%% This file is part of the Annie's Lasso project.
%% Copyright 2015 the authors.  All rights reserved.

\documentclass[12pt,preprint]{aastex}
\usepackage{amsmath,amssymb}

\newcommand{\project}[1]{\textsl{#1}}
\newcommand{\apogee}{\project{\textsc{apogee}}}
\newcommand{\logg}{\log g}
\newcommand{\Teff}{T_{\mathrm{eff}}}
\newcommand{\argmin}[1]{\underset{#1}{\operatorname{argmin}}\,}

\begin{document}

\title{The Cannon 2: A data-driven model \\ for detailed chemical abundance analyses}
\author{AC, DWH, MKN, others}

\begin{abstract}
% context
In previous work, we have shown that it is possible to train a generative
probabilistic model for stellar spectra using a training set of stars, each with known
parameters ($\logg$ and $\Teff$) and chemical abundance, and then use that model to infer labels for
unlabeled stars, even stars with lower signal-to-noise observations.
% aims
Here we ask whether this is possible when the dimensionality of the chemical
abundance space is large (15 abundances: Mg, Fe, etc, etc, Al)
and the model is non-linear in its response to abundance and parameter changes.
% method
We adopt ideas from compressed sensing to limit overall model complexity (number
of non-zero parameters) while retaining model freedom.
The training set is a set of YYY red-giant stars with high signal-to-noise
spectroscopic observations and stellar parameters and abundances taken from the
\apogee\ Survey.
% results
We find that we can successfully train and use a model with 17 stellar labels.
Cross-validation shows that the model does a good job of inferring all 17 labels
(with the exception of XXX), even when we degrade the signal-to-noise of the
validation set by discarding some of the spectroscopic observing time.
The model dependencies make sense; the derivatives of the spectral mean model
with respect to abundances correlate well with known atomic lines.
We deliver open-source code and also stellar parameters and 15 abundances for a
set of ZZZ stars.
\end{abstract}

\section{Introduction}

Hello World!

\section{Method}

The model is
\begin{eqnarray}
  y_{jn} &=& v(\ell_n)\cdot\theta_j + e_{jn}
  \label{eq:model}\quad ,
\end{eqnarray}
where $y_{jm}$ is the data for star $n$ at wavelength pixel $j$,
$v(\ell_n)$ is a function that takes as input
the label vector $\ell_n$ of length $K$ for star $n$
and outputs a vector of length $D>K$,
$\theta_j$ is a vector of length $D$ of parameters controlling the model at wavelength pixel $j$,
and $e_{jn}$ is a noise draw or residual.
Inasmuch as the model is good, the noise values $e_{jn}$ can be taken to be
drawn from a Gaussian with zero mean and variance $\sigma^2_{jn}+s^2_j$,
where $\sigma^2_{jn}$ is the pipeline-reported uncertainty variance on datum
$y_{jn}$ and $s^2_j$ is a parameter describing excess variance at wavelength pixel $j$.

Two comments about the model (\ref{eq:model}).
The first is that, because the $e_{jn}$ are thought of as being drawn from a 
probability density function (pdf), it is a probabilistic model for the spectral
data $y_{jn}$.
The second is that the output of the function $v(\ell)$ can be thought
of as a row of the ``design matrix'' that defines the possible freedom
given to the spectrum expectation model.

In the \emph{training step}, we fix the $K$-vectors of labels $\ell_n$
for all training-set stars $n$.
We seek, at each wavelength pixel $j$, the $[D+1]$ parameters
$\theta_j,s^2_j$ that optimize a penalized likelihood:
\begin{eqnarray}
  \theta_j,s^2_j &\leftarrow& \argmin{\theta,s^2}\left[
    \sum_{n=1}^N \frac{y_{jn}-v(\ell_n)\cdot\theta}{\sigma^2_{jn}+s^2}
    + \sum_{n=1}^N \ln(\sigma^2_{jn}+s^2)
    + \Lambda\,||\theta||_1^1
    \right]
  \\
  ||\theta||_1^1 &\equiv& \sum_{d=1}^D |\theta_d|
  \quad ,
\end{eqnarray}
where $\Lambda$ is a regularization parameter, and $||\theta||_1^1$ is
the L1-norm of $\theta$ or the sum of the absolute values of the
components $\theta_d$ of the $D$-vector $\theta$.

In the \emph{test step}, we fix the parameters $\theta_j,s^2_j$ at all
wavelength pixels $j$.
We seek, for each test-set star $m$, the $K$-vector of labels $\ell_m$
that optimizes the likelihood:
\begin{eqnarray}
  \ell_m &\leftarrow& \argmin{\ell}\left[
    \sum_{j=1}^J \frac{y_{jm}-v(\ell)\cdot\theta_j}{\sigma^2_{jm}+s^2_j}
    \right]
  \quad .
\end{eqnarray}

\section{Experiments}

\section{Results}

\end{document}<commit_msg>outline of intro<commit_after>%% This file is part of the Annie's Lasso project.
%% Copyright 2015 the authors.  All rights reserved.

% To-Do
% -----
% - get to zeroth draft
% - search for all occurrences of DWH, AC, or MKN in the text and fix them.

\documentclass[12pt,preprint]{aastex}
\usepackage{amsmath,amssymb}

\newcommand{\project}[1]{\textsl{#1}}
\newcommand{\thecannon}{\project{The~Cannon}}
\newcommand{\apogee}{\project{\textsc{apogee}}}
\newcommand{\aspcap}{\project{\textsc{aspcap}}}
\newcommand{\logg}{\log g}
\newcommand{\Teff}{T_{\mathrm{eff}}}
\newcommand{\argmin}[1]{\underset{#1}{\operatorname{argmin}}\,}

\begin{document}

\title{\textsl{The Cannon 2:} A data-driven model \\ for detailed chemical abundance analyses}
\author{AC, DWH, MKN, others}

\begin{abstract}
% context
In previous work, we have shown that it is possible to train a generative
probabilistic model for stellar spectra using a training set of stars, each with known
parameters ($\logg$ and $\Teff$) and chemical abundance, and then use that model to infer labels for
unlabeled stars, even stars with lower signal-to-noise observations.
% aims
Here we ask whether this is possible when the dimensionality of the chemical
abundance space is large (15 abundances: Mg, Fe, etc, etc, Al)
and the model is non-linear in its response to abundance and parameter changes.
% method
We adopt ideas from compressed sensing to limit overall model complexity (number
of non-zero parameters) while retaining model freedom.
The training set is a set of YYY red-giant stars with high signal-to-noise
spectroscopic observations and stellar parameters and abundances taken from the
\apogee\ Survey.
% results
We find that we can successfully train and use a model with 17 stellar labels.
Cross-validation shows that the model does a good job of inferring all 17 labels
(with the exception of XXX), even when we degrade the signal-to-noise of the
validation set by discarding some of the spectroscopic observing time.
The model dependencies make sense; the derivatives of the spectral mean model
with respect to abundances correlate well with known atomic lines.
We deliver open-source code and also stellar parameters and 15 abundances for a
set of ZZZ stars.
\end{abstract}

\section{Introduction}

We built \thecannon, and it rocked.
DWH: Summarize what it does and what it can be used for.
Cite some relevant literature.

DWH: Define the word ``label''.

DWH: Define the words ``train'', ``validate'', and ``test''.

DWH: Labels for data at wavelengths where there are no good labels.

DWH: Labels that are consistent across surveys and wavelengths.

\thecannon\ is a data-driven model, but it differs from standard machine-learning
approaches because it contains an explicit noise model.
This means that it can transfer labels from high signal-to-noise training-set
stars to low signal-to-noise test-set stars; that is, the training set and 
the test set do not need to be statistically identical.
This is related to the fact that \thecannon\ is an interpretable model;
the internals of the model are the dependencies of the spectral expetation
and variance on wavelength and physical parameters of the star.

DWH: Should we be saying things about the fact that the model is probabilistic?
It takes as input stellar labels and gives as output a pdf for stellar spectra?

DWH: In the first work we did with \thecannon, we only used a small
number of labels (three in the original work, and four or five in late
work; CITE).
Here we were guided by thoughts related to density estimation:
Sampling well a $K$-dimensional label space takes a training set the
size of which scales exponentially (or worse) with $K$.
Subsequent experiments, however, did not bear this out:
We found that we can transfer many labels from the training set to
the test set, with training sets of thousands of stars.
The fundamental reason is that \thecannon\ is \emph{not} a density estimator!
It is more like an \emph{interpolator}, which effectively finds stars in
the training set that are close to the test star, and transfers labels,
using the smooth polynomial model as a kind of regularizer.

DWH: Here we exploit this to the fullest.
We consider the \emph{entire} 17-dimensional label spece produced by
the \apogee\ \aspcap\ pipeline.
We (for very good reasons) believe the \aspcap\ labels for the highest
signal-to-noise stars, and adopt these stars and their labels as the
training set.
We show, by very conservative cross-validation, that we can transfer these
labels to much lower signal-to-noise stars, with reduced precision but no
strong biases.
We then use the system to label all of the stars in the \apogee\ DRXX data set.

DWH: Etc!

\section{Method}

DWH: What are we going to assume?

The model is
\begin{eqnarray}
  y_{jn} &=& v(\ell_n)\cdot\theta_j + e_{jn}
  \label{eq:model}\quad ,
\end{eqnarray}
where $y_{jm}$ is the data for star $n$ at wavelength pixel $j$,
$v(\ell_n)$ is a function that takes as input
the label vector $\ell_n$ of length $K$ for star $n$
and outputs a vector of length $D>K$,
$\theta_j$ is a vector of length $D$ of parameters controlling the model at wavelength pixel $j$,
and $e_{jn}$ is a noise draw or residual.
Inasmuch as the model is good, the noise values $e_{jn}$ can be taken to be
drawn from a Gaussian with zero mean and variance $\sigma^2_{jn}+s^2_j$,
where $\sigma^2_{jn}$ is the pipeline-reported uncertainty variance on datum
$y_{jn}$ and $s^2_j$ is a parameter describing excess variance at wavelength pixel $j$.

Two comments about the model (\ref{eq:model}).
The first is that, because the $e_{jn}$ are thought of as being drawn from a 
probability density function (pdf), it is a probabilistic model for the spectral
data $y_{jn}$.
The second is that the output of the function $v(\ell)$ can be thought
of as a row of the ``design matrix'' that defines the possible freedom
given to the spectrum expectation model.

In the \emph{training step}, we fix the $K$-vectors of labels $\ell_n$
for all training-set stars $n$.
We seek, at each wavelength pixel $j$, the $[D+1]$ parameters
$\theta_j,s^2_j$ that optimize a penalized likelihood:
\begin{eqnarray}
  \theta_j,s^2_j &\leftarrow& \argmin{\theta,s^2}\left[
    \sum_{n=1}^N \frac{y_{jn}-v(\ell_n)\cdot\theta}{\sigma^2_{jn}+s^2}
    + \sum_{n=1}^N \ln(\sigma^2_{jn}+s^2)
    + \Lambda\,||\theta||_1^1
    \right]
  \\
  ||\theta||_1^1 &\equiv& \sum_{d=1}^D |\theta_d|
  \quad ,
\end{eqnarray}
where $\Lambda$ is a regularization parameter, and $||\theta||_1^1$ is
the L1-norm of $\theta$ or the sum of the absolute values of the
components $\theta_d$ of the $D$-vector $\theta$.

In the \emph{test step}, we fix the parameters $\theta_j,s^2_j$ at all
wavelength pixels $j$.
We seek, for each test-set star $m$, the $K$-vector of labels $\ell_m$
that optimizes the likelihood:
\begin{eqnarray}
  \ell_m &\leftarrow& \argmin{\ell}\left[
    \sum_{j=1}^J \frac{y_{jm}-v(\ell)\cdot\theta_j}{\sigma^2_{jm}+s^2_j}
    \right]
  \quad .
\end{eqnarray}

\section{Training, validation, and test data}

MKN: Describe \apogee\ data here.

MKN: What is different about our continuum normalization from everyone else's?

DWH: Describe the random integer system here.

DWH: How do integers map on to train, validate, and test?

\section{Experiments}

\section{Results}

\end{document}
{'fraction_non_alphanumeric': 0.10193306650770671, 'fraction_numerical': 0.007834454568679212, 'mean_word_length': 4.472506989748369, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1428856', 'n_tokens_mistral': 3615, 'n_tokens_neox': 3438, 'n_words': 1617}