<commit_before>\documentclass[article]{jss}
\usepackage{amsmath}
\usepackage[utf8]{inputenc}
\usepackage{graphicx}
\usepackage{natbib}
\usepackage{dsfont}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% declarations for jss.cls %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%% almost as usual
\author{<NAME>\\The University of Auckland\And 
        <NAME>\\University of Washington}
\title{\pkg{DNest4}: Diffusive Nested Sampling in
\proglang{C++} and \proglang{Python}}

%% for pretty printing and a nice hypersummary also set:
\Plainauthor{<NAME>, <NAME>} %% comma-separated
\Plaintitle{DNest4: An implementation of Diffusive Nested Sampling in
C++11 and Python} %% without formatting
\Shorttitle{\pkg{DNest4}: Diffusive Nested Sampling} %% a short title (if necessary)

%% an abstract and keywords
\Abstract{In probabilistic (Bayesian) inferences, we typically want to compute
properties of the posterior distribution, describing knowledge of
unknown quantities in the context of a particular dataset and the assumed
prior information. The marginal likelihood (also known as the ``evidence'')
is a key quantity in Bayesian model averaging. The Diffusive Nested Sampling algorithm, a variant of Nested Sampling, is a powerful tool for generating
posterior samples and estimating marginal likelihoods. It is effective at
solving complex problems including many where the posterior distribution is
multimodal or has strong dependencies between variables. The key requirement
is that the likelihood function be fast to evaluate.
\pkg{DNest4} is a multi-threaded implementation of this algorithm in
modern \proglang{C++} (specifically, \proglang{C++11}),
along with associated utilities including: i)
a \pkg{Python} package allowing basic use without \proglang{C++} coding;
ii) \pkg{RJObject}, a class template for finite mixture models;
iii) Experimental support for models implemented in \proglang{Julia};
iv) An experimental tool \proglang{Python} tool to
generate \pkg{C++} code from a model specification.
In this paper we demonstrate \pkg{DNest4} usage through examples including
simple Bayesian data analysis, finite mixture models, and Approximate
Bayesian Computation (ABC).
}

\Keywords{bayesian inference, markov chain monte carlo,
metropolis algorithm, bayesian computation, nested sampling, \proglang{c++11},
\proglang{python}}
\Plainkeywords{bayesian inference, markov chain monte carlo,
metropolis algorithm, bayesian computation, nested sampling, c++11, python} %% without formatting
%% at least one keyword must be supplied

%% publication information
%% NOTE: Typically, this can be left commented and will be filled out by the technical editor
%% \Volume{50}
%% \Issue{9}
%% \Month{June}
%% \Year{2012}
%% \Submitdate{2012-06-04}
%% \Acceptdate{2012-06-04}

%% The address of (at least) one author should be given
%% in the following format:
\Address{
  <NAME>\\
  Department of Statistics\\
  The University of Auckland\\
  Private Bag 92019\\
  Auckland, 1142\\
  New Zealand\\
  E-mail: \email{<EMAIL>}\\
  URL: \url{https://www.stat.auckland.ac.nz/~brewer/}
}

\newcommand{\params}{\theta}
\newcommand{\data}{D}
\newcommand{\dobs}{D_{\rm obs}}

\begin{document}
\maketitle

% Need this after the abstract
\setlength{\parindent}{0pt}
\setlength{\parskip}{8pt}

\section{Bayesian computation and Nested Sampling}

In very compact notation, the posterior
posterior distribution for parameters $\params$ given data $\data$, within
context $I$, is
\begin{align}
p(\params | \data, I) &=
\frac{p(\params | I)p(\data | \params, I)}{p(\data | I)}
\end{align}
or
\begin{align}
\textnormal{posterior} &=
\frac{\textnormal{prior} \times \textnormal{likelihood}}
     {\textnormal{marginal likelihood}}.
\end{align}
When discussing computational matters, the prior is
often written $\pi(\theta)$, the likelihood $L(\theta)$,
and the marginal likelihood $Z$.

\subsection{Markov Chain Monte Carlo}
In the Metropolis algorithm, the acceptance probability $\alpha$
is given by
\begin{align}
\alpha &= \min\left(1,
\frac{q(\params'|\params)}{q(\params | \params')}
\frac{\pi(\params')}{\pi(\params)}\frac{L(\params')}{L(\params)}
\right)
\end{align}
where $q(\theta' | \theta)$ is the proposal distribution used to generate
a new position $\theta'$ from the current position $\theta$. Often,
$q$ is {\em symmetric} so the $q$ terms cancel in the acceptance probability.

In Nested Sampling, the target distribution is not the posterior but rather
the constrained prior. Hence, the appropriate acceptance probability is
\begin{align}
\alpha &= \min\left[1,
\frac{q(\params'|\params)}{q(\params | \params')}
\frac{\pi(\params')}{\pi(\params)}
\mathds{1}\left(L(\params') > L^*\right)
\right]
\end{align}
where $L^*$ is the threshold value of the likelihood.

The \pkg{DNest4} \code{Sampler} class handles the likelihood check,
while the \code{perturb(DNest4::RNG\&)} member function handles the
rest of the terms. The value returned from \code{perturb(DNest4::RNG\&)}
must be
$\ln\left[q(\params'|\params)/q(\params'|\params)\times\pi(\params')/\pi(\params)\right]$ for your problem.



%\subsection{Diffusive Nested Sampling}



\section{Basic usage of DNest4}
The

\section{Advanced usage}

%\subsection{Multithreading}

\section{Implementing models}
The ``classic'' method of implementing models in \pkg{DNest4} is by
writing a \proglang{C++} class, an object of which represents a
point in your parameter space.

To demonstrate, consider a simple linear regression situation where the
sampling distribution is
\begin{align}
y_i | m, b, \sigma &\sim \textnormal{Normal}(mx_i + b, \sigma^2)
\end{align}
and the priors are
\begin{align}
m &\sim \textnormal{Normal}(0, 1000^2)\\
b &\sim \textnormal{Normal}(0, 1000^2)\\
\ln\sigma &\sim \textnormal{Uniform}(-10, 10)
\end{align}
These are naive diffuse priors and we do not claim any special status for them.
<commit_msg>List of member functions required<commit_after>\documentclass[article]{jss}
\usepackage{amsmath}
\usepackage[utf8]{inputenc}
\usepackage{graphicx}
\usepackage{natbib}
\usepackage{dsfont}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% declarations for jss.cls %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%% almost as usual
\author{<NAME>\\The University of Auckland\And 
        <NAME>\\University of Washington}
\title{\pkg{DNest4}: Diffusive Nested Sampling in
\proglang{C++} and \proglang{Python}}

%% for pretty printing and a nice hypersummary also set:
\Plainauthor{<NAME>, <NAME>} %% comma-separated
\Plaintitle{DNest4: An implementation of Diffusive Nested Sampling in
C++11 and Python} %% without formatting
\Shorttitle{\pkg{DNest4}: Diffusive Nested Sampling} %% a short title (if necessary)

%% an abstract and keywords
\Abstract{In probabilistic (Bayesian) inferences, we typically want to compute
properties of the posterior distribution, describing knowledge of
unknown quantities in the context of a particular dataset and the assumed
prior information. The marginal likelihood (also known as the ``evidence'')
is a key quantity in Bayesian model averaging. The Diffusive Nested Sampling algorithm, a variant of Nested Sampling, is a powerful tool for generating
posterior samples and estimating marginal likelihoods. It is effective at
solving complex problems including many where the posterior distribution is
multimodal or has strong dependencies between variables. The key requirement
is that the likelihood function be fast to evaluate.
\pkg{DNest4} is a multi-threaded implementation of this algorithm in
modern \proglang{C++} (specifically, \proglang{C++11}),
along with associated utilities including: i)
a \pkg{Python} package allowing basic use without \proglang{C++} coding;
ii) \pkg{RJObject}, a class template for finite mixture models;
iii) Experimental support for models implemented in \proglang{Julia};
iv) An experimental \proglang{Python} tool to
generate \pkg{C++} code from a model specification.
In this paper we demonstrate \pkg{DNest4} usage through examples including
simple Bayesian data analysis, finite mixture models, and Approximate
Bayesian Computation (ABC).
}

\Keywords{bayesian inference, markov chain monte carlo,
metropolis algorithm, bayesian computation, nested sampling, \proglang{c++11},
\proglang{python}}
\Plainkeywords{bayesian inference, markov chain monte carlo,
metropolis algorithm, bayesian computation, nested sampling, c++11, python} %% without formatting
%% at least one keyword must be supplied

%% publication information
%% NOTE: Typically, this can be left commented and will be filled out by the technical editor
%% \Volume{50}
%% \Issue{9}
%% \Month{June}
%% \Year{2012}
%% \Submitdate{2012-06-04}
%% \Acceptdate{2012-06-04}

%% The address of (at least) one author should be given
%% in the following format:
\Address{
  <NAME>\\
  Department of Statistics\\
  The University of Auckland\\
  Private Bag 92019\\
  Auckland, 1142\\
  New Zealand\\
  E-mail: \email{<EMAIL>}\\
  URL: \url{https://www.stat.auckland.ac.nz/~brewer/}
}

\newcommand{\params}{\theta}
\newcommand{\data}{D}
\newcommand{\dobs}{D_{\rm obs}}

\begin{document}
\maketitle

% Need this after the abstract
\setlength{\parindent}{0pt}
\setlength{\parskip}{8pt}

\section{Bayesian computation and Nested Sampling}

In very compact notation, the posterior
posterior distribution for parameters $\params$ given data $\data$, within
context $I$, is
\begin{align}
p(\params | \data, I) &=
\frac{p(\params | I)p(\data | \params, I)}{p(\data | I)}
\end{align}
or
\begin{align}
\textnormal{posterior} &=
\frac{\textnormal{prior} \times \textnormal{likelihood}}
     {\textnormal{marginal likelihood}}.
\end{align}
When discussing computational matters, the prior is
often written $\pi(\theta)$, the likelihood $L(\theta)$,
and the marginal likelihood $Z$.

\subsection{Markov Chain Monte Carlo}
In the Metropolis algorithm, the acceptance probability $\alpha$
is given by
\begin{align}
\alpha &= \min\left(1,
\frac{q(\params'|\params)}{q(\params | \params')}
\frac{\pi(\params')}{\pi(\params)}\frac{L(\params')}{L(\params)}
\right)
\end{align}
where $q(\theta' | \theta)$ is the proposal distribution used to generate
a new position $\theta'$ from the current position $\theta$. Often,
$q$ is {\em symmetric} so the $q$ terms cancel in the acceptance probability.

In Nested Sampling, the target distribution is not the posterior but rather
the constrained prior. Hence, the appropriate acceptance probability is
\begin{align}
\alpha &= \min\left[1,
\frac{q(\params'|\params)}{q(\params | \params')}
\frac{\pi(\params')}{\pi(\params)}
\mathds{1}\left(L(\params') > L^*\right)
\right]\\
&= \min\left[1,
e^{\ln(H)}
\mathds{1}\left(L(\params') > L^*\right)
\right]\label{eqn:log_hastings}
\end{align}
where $L^*$ is the threshold value of the likelihood.

The \pkg{DNest4} \code{Sampler} class handles the likelihood check,
while the \code{perturb(DNest4::RNG\&)} member function handles the
rest of the terms. The value returned from \code{perturb(DNest4::RNG\&)}
must be
$\ln\left[q(\params'|\params)/q(\params'|\params)\times\pi(\params')/\pi(\params)\right]$
for your problem.



%\subsection{Diffusive Nested Sampling}



\section{Basic usage of DNest4}
The

\section{Advanced usage}

%\subsection{Multithreading}

\section{Implementing models}
The ``classic'' method of implementing models in \pkg{DNest4} is by
writing a \proglang{C++} class, an object of which represents a
point in your parameter space.
To demonstrate, consider a simple linear regression situation where the
sampling distribution is
\begin{align}
y_i | m, b, \sigma &\sim \textnormal{Normal}(mx_i + b, \sigma^2)
\end{align}
and the priors are
\begin{align}
m &\sim \textnormal{Normal}(0, 1000^2)\\
b &\sim \textnormal{Normal}(0, 1000^2)\\
\ln\sigma &\sim \textnormal{Uniform}(-10, 10)
\end{align}
These are naive diffuse priors and we do not claim any special status for them.

Running \pkg{DNest4} on this problem requires a class, which we call
{\tt StraightLine}, which has the following member functions:
\begin{itemize}
\item \code{void from\_prior(DNest4::RNG\& rng)}, which generates parameter
        values from the prior;
\item \code{double perturb(DNest4::RNG\& rng)}, which proposes a
        change to the parameter values, and returns $\ln(H)$ as defined in
        Equation~\ref{eqn:log_hastings};
\item \code{double log\_likelihood() const}, which evaluates the log of
        the likelihood function;
\item \code{void print(std::ostream\& out) const}, which prints parameters of
        interest to the given output stream; and
\item \code{std::string description() const}, which returns a \proglang{C++}
        string naming the parameters printed by \code{print(std::ostream\&) const}.
\end{itemize}

{'fraction_non_alphanumeric': 0.1406924707544948, 'fraction_numerical': 0.011855224935228077, 'mean_word_length': 5.083094555873926, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 18, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '858563', 'n_tokens_mistral': 3948, 'n_tokens_neox': 3667, 'n_words': 1375}