The role of positron polarization for the inital $250$ GeV stage of the International Linear Collider

Abstract: The International Linear Collider is now proposed with a staged machine design, with the first stage at $\sqrt{s}=$~250 GeV and an integrated luminosity goal of 2~ab$^{-1}$. One of the questions for the machine design is the importance of positron polarization. In this report, we review the impact of positron polarization on the physics goals of the $250$ GeV stage of the ILC and demonstrate that positron polarization has distinct advantages.

\section{\@startsection{section}{0}{\z@}{5.5ex plus .5ex minus
 1.5ex}{2.3ex plus .2ex}{\large\bf}}
\def\subsection{\@startsection{subsection}{1}{\z@}{3.5ex plus .5ex minus
 1.5ex}{1.3ex plus .2ex}{\normalsize\bf}}
\def\subsubsection{\@startsection{subsubsection}{2}{\z@}{-3.5ex plus
-1ex minus  -.2ex}{2.3ex plus .2ex}{\normalsize\sl}}

\renewcommand{\@makecaption}[2]{%
   \vskip 10pt
   \setbox\@tempboxa\hbox{\small #1: #2}
   \ifdim \wd\@tempboxa >\hsize
       \small #1: #2\par
     \else
       \hbox to\hsize{\hfil\box\@tempboxa\hfil}
   \fi}

 \def\citenum#1{{\def\@cite##1##2{##1}\cite{#1}}}

\newcount\@tempcntc
\def\@citex[#1]#2{\if@filesw\immediate\write\@auxout{\string\citation{#2}}\fi
  \@tempcnta\z@\@tempcntb\m@ne\def\@citea{}\@cite{\@for\@citeb:=#2\do
    {\@ifundefined
       {b@\@citeb}{\@citeo\@tempcntb\m@ne\@citea\def\@citea{,}{\bf ?}\@warning
       {Citation `\@citeb' on page \thepage \space undefined}}%
    {\setbox\z@\hbox{\global\@tempcntc0\csname b@\@citeb\endcsname\relax}%
     \ifnum\@tempcntc=\z@ \@citeo\@tempcntb\m@ne
       \@citea\def\@citea{,}\hbox{\csname b@\@citeb\endcsname}%
     \else
      \advance\@tempcntb\@ne
      \ifnum\@tempcntb=\@tempcntc
      \else\advance\@tempcntb\m@ne\@citeo
      \@tempcnta\@tempcntc\@tempcntb\@tempcntc\fi\fi}}\@citeo}{#1}}
\def\@citeo{\ifnum\@tempcnta>\@tempcntb\else\@citea\def\@citea{,}%
  \ifnum\@tempcnta=\@tempcntb\the\@tempcnta\else
  {\advance\@tempcnta\@ne\ifnum\@tempcnta=\@tempcntb \else\def\@citea{--}\fi
    \advance\@tempcnta\m@ne\the\@tempcnta\@citea\the\@tempcntb}\fi\fi}
\makeatother

\section{Introduction}

Recently, the plan for the International Linear Collider (ILC) has been revised to a staged machine design with the first stage at $\sqrt{s}=250$\,GeV~\cite{Evans:2017rvt}. The physics for such a staged machine has been sumarized in~\cite{Fujii:2017vwa}, based on electron and positron beam polarization, with 80\% polarization of the electron beam and 30\% polarization of the positron beam, as foreseen in the ILC  Technical Design Report~\cite{Adolphsen:2013kya}. Electron polarization is essential for all of the physics goals of the ILC.  It plays an important role in the measurements proposed for every physics topic that will be studied at this machine~\cite{ILCTDR_VOL2}, and we will not comment further on the role of electron polarization in this document.

The baseline design of the $250$\,GeV stage of the ILC described in~\cite{Evans:2017rvt} includes a polarized positron source. However, because there exist alternative concepts with complementary strengths and weaknesses, the Linear Collider Collaboration (LCC) requested a survey of the importance of positron polarization to meet the physics goals of the $250$\,GeV stage.
This report is intended to address that request.

The role of positron polarization at future $e^+e^-$ colliders has been reviewed
in great detail in the past~\cite{MoortgatPick:2005cw}, and  updated for the case of $30$\% positron polarization~\cite{POWER_SB2009}. These reports identified three main benefits of positron polarization.  In this report, we will trace the influence of these through the physics topics of the $250$\,GeV stage of the ILC.

There are three main effects of positron beam polarization which will be discussed in the context of specific physics examples in this note:
\begin{enumerate}
\item
Positron polarization allows us to obtain subsamples of the data with higher rates for interesting physics processes and lower rates for backgrounds. Since  sensitivities do not combine as a linear sum,
the combination of results from e.g.\ two data sets with small and large signal-to-background ratio, respectively, is more sensitive than a single data set with the same total number of signal and background events.
\item Positron polarization offers four distinct data sets instead of the two available if only the electron beam can be polarized. Most important reactions can be studied with the opposite-sign polarization modes only, but there are measurements in which the two like-sign polarization states give additional or even unique information. The flexibility in choosing between these configurations (and possibly even five more when considering parts of the data to be taken with zero longitudinal polarisation) is a unique asset of the ILC.
\item The likely most important effect is the control of systematic uncertainties: The precisions aimed for at the ILC can only be reached if all relevant systematic uncertainties are controlled to the same level as the statistical uncertainties or better. This requires sufficient experimental redundancy in order to determine all relevant nuisance parameters in-situ. While it is difficult to estimate reliably systematic uncertainties in the absence of real detectors and real data, we present one example based on ILC Monte-Carlo studies below. In addition we discuss a real-life example from the SLC which is very instructive also for the ILC.
\end{enumerate}

As we survey the various physics topics, we will conclude that the first of these advantages can in many cases be compensated by an increase of the running time by $2-3$ years. The second advantage is a qualitative one,
and the loss in degrees of freedom from $4$ to $2$ independent data-sets\footnote{or, when including the case of zero longitudinal polarisation by either rotating to transverse polarisation or by active depolarisation from $9$ to $3$ data-sets} will be paid in terms of less model-independence of the measurements and their interpretation.

The third advantage that we have cited here is less easily quantified.  However, the advantage of positron polarization for systematic control in precision experiments is a very important one for the ILC program. In particular in the event that the ILC discovers an anomaly with respect to the Standard Model in a precision observable, the additional measurements made possible by positron polarization will be important to give confidence in the presence of this effect with respect to possible systematic uncertainties.

On this basis, we find that positron polarization has an important role to play in the ILC program.

This note is structured as follows: we will begin with a recap of polarization formalism in Section~\ref{sec:polbasics} and discuss polarimetry, with special emphasis on the case \ensuremath{\P_{e^+}}=0 in Section~\ref{sec:polarimetry}. In Section~\ref{sec:sm}, we will illustrate the importance of positron polarization for the control of systematic uncertainties by the example of 2- and 4-fermion processes including $W$ boson pair production, and explain the role of positron polarization in the search for new sources of CP violation. Section~\ref{sec:higgs} addresses the influence of positron polarization on precision Higgs boson measurements. Section~\ref{sec:bsm}  discusses positron polarization in the context of searches for new phenomena. We finally give our conclusions in Section~\ref{sec:conclusions}.

\section{Polarization at $e^+e^-$ Colliders}
\label{sec:polbasics}

In order to discuss the effects listed above in more detail, we recall a few general considerations. As weak interactions are chiral, i.e.\ $W^{\pm}$ and $Z$ bosons couple differently to left-handed and right-handed fermions, and only left-handed fermions take part in charged weak interactions, polarization effects play an important role in strategies to extract information from $e^+e^-$ reactions. Since all the strong motivations to search for physics beyond the Standard Model (SM) of particles physics relate to its electroweak sector, there is every reason to expect that also new phenomena will depend on the chirality of the involved particles.

Any real particle beam will contain a mixture of $N_L$ left- and $N_R$ right-handed particles, given by the longitudinal beam polarization
\begin{equation}
{\cal P} = \frac{N_R - N_L}{N_R + N_L}
\end{equation}
Depending on the orientation of the full polarization vector, there can also be transverse polarization, which will play a role later when we discuss CP violating effects. The spin rotator systems of the ILC~\cite{Malysheva:2016jdr}, which are needed to turn the polarization into the vertical before the damping rings and back into the longitudinal direction afterwards, have been designed carefully to allow
any orientation of the polarization vectors at the $e^+e^-$ interaction point. In the following, however, ``polarization'' refers by default to longitudinal polarization, unless transvese polarization is explicitly mentioned.

For electron beam polarizations \ensuremath{\P_{e^-}}\ and positron beam polarization \ensuremath{\P_{e^+}} , the cross section of any reaction is computed from the four possible pure chiral cross sections (with $\sigma_{\rm LR}$ for left-handed electron and right-handed positron etc.) as
\begin{eqnarray}
\sigma(\ensuremath{\P_{e^-}},\ensuremath{\P_{e^+}}) &=& \frac{1}{4}\bigl\{
  (1+\ensuremath{\P_{e^-}})(1+\ensuremath{\P_{e^+}})\sigma_{\rm RR}
+ (1-\ensuremath{\P_{e^-}})(1-\ensuremath{\P_{e^+}})\sigma_{\rm LL} \nonumber \\
&& + (1+\ensuremath{\P_{e^-}})(1-\ensuremath{\P_{e^+}})\sigma_{\rm RL}
+ (1-\ensuremath{\P_{e^-}})(1+\ensuremath{\P_{e^+}})\sigma_{\rm LR} \bigr\},
\end{eqnarray}
The unpolarized cross section $\sigma_0$ is given by
\begin{equation}
\sigma_0 = \frac{1}{4}\bigl\{\sigma_{\rm RR} + \sigma_{\rm LL} + \sigma_{\rm RL} + \sigma_{\rm LR} \bigr\}.
\end{equation}
Further important quantities are the left-right asymmetry $A_{\rm LR}$, the effective luminosity \ensuremath{\L_{\eff}}, and the effective polarization \ensuremath{\P_{\eff}}:
\begin{eqnarray}
A_{\rm LR} &=& \frac{(\sigma_{\rm LR} - \sigma_{\rm RL})}{(\sigma_{\rm LR} + \sigma_{\rm RL})} \\
\ensuremath{\L_{\eff}} &=& \frac{1}{2} (1-\ensuremath{\P_{e^-}} \ensuremath{\P_{e^+}} ) {\cal L} \\
\ensuremath{\P_{\eff}} &=& \frac{\ensuremath{\P_{e^-}} -\ensuremath{\P_{e^+}}}{1 - \ensuremath{\P_{e^-}} \ensuremath{\P_{e^+}}}.
\end{eqnarray}
The quantity \ensuremath{\P_{\eff}}\ can be substantially closer to $\pm 1$ than achievable single beam polarizations. For example, for $\ensuremath{\P_{e^+}} = +0.3$ and $\ensuremath{\P_{e^-}} = -0.8$, $\ensuremath{\L_{\eff}} = 0.62 {\cal L}$ and $\ensuremath{\P_{\eff}} = -0.89$, while in case of $\ensuremath{\P_{e^+}} = 0$, $\ensuremath{\L_{\eff}} = 0.5 {\cal L}$ and $\ensuremath{\P_{\eff}} = \ensuremath{\P_{e^-}}$.
The drop in $\ensuremath{\L_{\eff}}$ means that descoping from \ensuremath{\P_{e^+}}=30\% to \ensuremath{\P_{e^+}}=0 is for most processes equivalent to a 24\% loss in luminosity, and at the same time the reduction of $\ensuremath{\P_{\eff}}$ translates into a 10\% reduction of analysing power for left-right asymmetries, which are important observables for electroweak and Higgs physics.

For the ILC physics programme, several distinct types of process are of particular importance:
\begin{itemize}
\item {\bf $s$-channel $Z/\gamma$ exchange:} For the $s$-channel exchange of a vector boson, the spin of the incoming particles have to add up to a total spin-1 configuration, therefore only $\sigma_{LR}$ and $\sigma_{RL}$ are non-zero. In this case, the polarized cross section simplifies to:
\begin{equation}
\sigma(\ensuremath{\P_{e^-}},\ensuremath{\P_{e^+}}) = 2 \sigma_0 (\ensuremath{\L_{\eff}} / {\cal L}) [1- \ensuremath{\P_{\eff}} A_{\rm LR}]
\end{equation}
Important examples are Higgs production via Higgs-strahlung and fermion-anti-fermion production, e.g.\ $b\bar{b}$ production.
\item{\bf $t$-channel $W$ or $\nu_e$ exchange:} Since only left-handed fermions and right-handed anti-fermions take part in the charged weak interaction, only $\sigma_{LR}$ is non-zero in this case, or in other words $A_{\rm LR} = 1$, thus
\begin{equation}
\sigma(\ensuremath{\P_{e^-}},\ensuremath{\P_{e^+}}) = 2 \sigma_0 (\ensuremath{\L_{\eff}} / {\cal L}) [1 - \ensuremath{\P_{\eff}}]
\end{equation}
Important examples are Higgs production via $WW$ fusion and $W$ pair production, but also neutrino pair production as background for missing energy signatures.
In this case, the impact of \ensuremath{\L_{\eff}}\ and \ensuremath{\P_{\eff}}\ is even more striking: for $\ensuremath{\P_{e^+}} = +0.3$ and $\ensuremath{\P_{e^-}} = -0.8$, the cross section increases by 30\% w.r.t.\ $\ensuremath{\P_{e^+}} = 0$ and $\ensuremath{\P_{e^-}} = -0.8$. In the opposite sign configuration, which in case of e.g.\ $WW$ measurements serves as in-situ background determination,  $\ensuremath{\P_{e^+}} = -0.3$ and $\ensuremath{\P_{e^-}} = +0.8$ gives a 30\% reduction of the remaining signal pollution w.r.t.\ $\ensuremath{\P_{e^+}} = 0$ and $\ensuremath{\P_{e^-}} = +0.8$.
\item{\bf single $W$ production:} In this case, only one of the beam particles emits a $W$ boson, which scatters on a photon (or a $Z$) from the other beam. Since the photon and the $Z$ boson couple to left- and right-handed fermions, $\sigma_{LR}$ and $\sigma_{LL}$ are allowed for $W^-$ production, while $\sigma_{LR}$ and $\sigma_{RR}$ are responsible for $W^+$ production. So in these cases, the polarized cross sections are given by:
\begin{equation}
\sigma^{\pm}(\ensuremath{\P_{e^-}},\ensuremath{\P_{e^+}}) = \sigma_0 (1 \pm \ensuremath{\P_{e^{\pm}}}) [1 - \ensuremath{\P_{e^{\mp}}} A_{\rm RR/LL}]
\end{equation}
with the upper signs for $W^+$ production and the lower ones for $W^-$ production. The relevant asymmetries are $A_{\rm RR} = (\sigma_{\rm LR} - \sigma_{\rm RR})/(\sigma_{\rm LR} + \sigma_{\rm RR})$ and $A_{\rm LL} = (\sigma_{\rm LR} - \sigma_{\rm LL})/(\sigma_{\rm LR} + \sigma_{\rm LL})$, respectively.
\item{\bf new physics:}
Depending on the kind of new physics, in principle all four chiral cross sections can be relevant.
The measurement of the full set of chiral cross sections delivers important information on the interaction of the new particles with SM particles. We will discuss WIMP dark matter as a specific example below.
\end{itemize}

In order to evaluate the role of positron polarization quantitatively, we need to refer to a specific
operation scenario. Throughout this document, we will assume a total integrated luminosity of $2$\,ab$^{-1}$ to be collected at a center-of-mass energy of $250$\,GeV. For the case of $|\ensuremath{\P_{e^+}}| = 30\%$, we assume that $45\%$ of the data will be collected with $\ensuremath{\P_{e^-}} = -80\%$ and $\ensuremath{\P_{e^+}} = +30\%$, another $45\%$ with reversed signs, and $5\%$ on each of the like-sign configurations, in accordance with~\cite{Fujii:2017vwa}. In the absence
of positron polarization, we assume half of the data to be taken with $\ensuremath{\P_{e^-}} = -80\%$, and the other half with $\ensuremath{\P_{e^-}} = +80\%$. These specific combinations will be referred to as ``\ensuremath{\P_{e^+}} = 30\%'' and ``\ensuremath{\P_{e^+}} = 0'' scenarios in the remainder of the document.

\section{Determination of the Beam Polarization}
\label{sec:polarimetry}
The polarimeters~\cite{Boogert:2009ir} in the beam delivery system of the ILC will provide fast online measurements of the beam polarization  $\sim 1.7$\,km before and $\sim 150$\,m behind the $e^+e^-$ interaction point with precisions of $0.25\%$. However the luminosity-weighted average polarization during the $e^+e^-$ collisions, which can differ from the polarimeter measurements due to spin transport and depolarization effects~\cite{Beckmann:2014mka}, will ultimately be obtained by a combination of the polarimeter measurements with $e^+e^-$ cross section measurements. For the case of \ensuremath{\P_{e^+}} = 30\% it has recently been shown that the goal of 0.1\% precision on the luminosity-weighted average polarization can be reached~\cite{Karl:2017xra} for each of the datasets foreseen in the H-20 running scenario~\cite{Barklow:2015tja}, even when treating the absolute values of the beam polarizations for positive and negative signs as independent parameters, thus allowing for an imperfect helicity reversal.

Before we discuss the effect of the positron polarization on Higgs and SM measurements as well as on BSM searches in the next sections, it is important to address the following question: Can we, in the case of an unpolarized positron source, rely under all circumstances on $\ensuremath{\P_{e^+}} \equiv 0$ without in-situ control of this assumption? Or should \ensuremath{\P_{e^+}}\ rather be included as nuisance parameter, possibly constrained within the polarimeter uncertainty?

While there is no obvious origin of polarization (other than the polarized sources) in the design of the ILC, we need to consider what
would happen if we indeed observe a discrepancy from the Standard Model prediction, e.g.\ in some cross-section asymmetry. Before being able to claim an observation of physics beyond the SM, it would be our duty to exclude any other explanation for the observed discrepancy, however unlikely it might seem. Depending on how well this will then be possible {\em a posteriori}, any remaining doubt would shadow the ILC's discovery potential.

Actually there is an example of exactly such a situation in the measurement of the effective weak mixing angle via $A^e_{\rm LR}$ by the SLD experiment and via $A^b_{\rm FB}$ at LEP, which differ by more than $3\,\sigma$~\cite{ALEPH:2005ab}. While the SLD measurement of course relied on the SLC electron beam being polarized, the SLC positron beam was nominally unpolarized and not equipped with a polarimeter. In view of the above mentioned discrepancy, the SLD collaboration in the end undertook a considerable effort in order to measure the positron polarization {\em a posteriori} to a precision of $\pm 0.0007$ in a dedicated experiment at SLAC's End Station A~\cite{SLDnote268} --- despite the fact that nobody had a mechanism for producing non-zero positron polarization from the SLC positron target!

We conclude from this lesson than even for the \ensuremath{\P_{e^+}} = 0 scenario, the
positron polarization should be treated as a nuisance parameter in global fits, and that the polarimeters for the positron beam are essential even if the nominal polarization is zero.

\section{Standard Model Precision Measurements}
\label{sec:sm}

Precision measurements of all kinds of electroweak observables are at the heart of the ILC's physics program, and will be the basis for the precise and model-independent characterization of the Higgs bosons. Observables like total and differential cross sections, left-right and forward-backward asymmetries etc.\ enable us to probe energy scales far beyond the center-of-mass energy of the collider. See, e.g.,~\cite{Fujii:2017vwa} and references therein.

Higher precision of the measurements will thereby allow to probe higher energy scales. In order to reduce the impact of systematic uncertainties
to a minimum, it will be necessary to constrain the actual observables of interest simultaneously with many so-called nuisance parameters, which model possible systematic effects. Classic examples comprise the
luminosity, the beam polarizations, selection efficiencies as well as theoretical or parametric uncertainties.

\subsection{Cross section and asymmetry measurements}
\label{sec:sm:xs}
Only recently a study to demonstrate the simultaneous extraction of total cross sections, left-right asymmetries and beam polarizations from differential distributions of all kinds of electroweak processes at the ILC was started. The results will be reported in this document for the first time, based on an extension of the framework described in~\cite{Karl:2017xra} for the beam polarization extraction\footnote{
A further extension of this framework to include also anomalous triple gauge couplings is in progress.}. This study is currently by far the most comprehensive attempt at such a global interpretation, and directly targets ILC operation at 250\,GeV.

The study makes use of the errors projected for ILC on the following observables:  for $W$ pair production, single W production, the total cross sections (for each setting of the beam polarizations) and the (binned) differential cross sections with respect to the $W$ production and decay angles;  for single $W$ production, the total cross sections and differential cross sections with respect to the $W$ decay angles; for 2-fermion processes, the total cross sections and differential cross sections with respect to the fermion production angle.  Measurements of the beam polarizations by the polarimeters to an accuracy of 0.0025 are also included.  The analysis also includes the possibility of an undetected bias between the polarization measured at the IP. A fit is performed and the following parameters are extracted:   the total unpolarized cross section for each process, the left-right asymmetry for each process, and the beam polarizations separately for $e^-$ and $e^+$, a total of 16 parameters.  In the case \ensuremath{\P_{e^+}}=0, only one parameter is considered for the positron polarization, giving 15 parameters in total.

While a full description of the procedure and it's results would go far beyond the scope of this document, we summarize here the findings which are of highest relevance to the impact of positron polarization. A full documentation will be available in early 2018 in the PhD thesis of R.~Karl~\cite{Karl_thesis}. All numbers in the following are lower limits since detector inefficiencies and instrumental backgrounds have not yet been included. We therefore stress most the {\em relative} changes between different configurations, which are not expected to be significantly affected by these simplifications.

\begin{itemize}
\item In all the configurations studied so far, the electron polarization was always well determined to sub-per mille precision.
\item Without a constraint on the positron polarization from the polarimeters, the positron polarization can only be constrained to
0.5\% in the \ensuremath{\P_{e^+}}=0 case. This is a factor 5 worse than usually assumed as systematic uncertainty, e.g., in the Higgs boson studies described in Sec.~\ref{sec:higgs}. At the same time, the resulting uncertainties on the total cross sections and on the left-right asymmetries grow by typically one order of magnitude compared to the case of \ensuremath{\P_{e^+}}=30\%.
\item When adding the polarimeter constraint for the positron polarization, assuming no bias, the effects are partially mitigated:
The polarimeter uncertainty propagates nearly one-to-one onto the positron polarization obtained from the fit, so with the polarimeter measuring $\ensuremath{\P_{e^+}} = 0 \pm 0.0025$, the final uncertainty is still 0.0024.
The same applies for the total cross sections for $W$ pair and single-$W$ processes and the left-right asymmetries for the 2-fermion processes, which still are a factor 2-3 worse than in the \ensuremath{\P_{e^+}}=30\% case.
\item If now a bias of 2.5 per mille is assumed between the polarimeter and the IP, thus  $\ensuremath{\P_{e^+}} = 0.0025 \pm 0.0025$, we observe the following effects: In the case of \ensuremath{\P_{e^+}} = 30\%, the fitted cross sections and asymmetries receive a bias of typically $-0.5 \sigma$, thus covered by the uncertainty. A larger discrepancy could be revealed by comparison with the results from a fit without the polarimeter constraint, which in case of \ensuremath{\P_{e^+}} = 30\% is, for sufficiently large data-sets, of similar precision. In the case of \ensuremath{\P_{e^+}} = 0, however, all total cross sections and three out of the six considered asymmetries
receive biases between $-1$ and $-1.5\,\sigma$. Since the uncertainties without the polarimeter constraint are an order of magnitude larger, as described in the previous bullet, the opportunity of an independent consistency check does not exist in the case of \ensuremath{\P_{e^+}} = 0.
\end{itemize}

Although this study is still work in progress, it illustrates that positron polarization plays an important role in beating down systematic uncertainties for all kinds of cross section and asymmetry measurements --- including Higgs observables.

\subsection{$CP$ violation}
\label{sec:sm:cp}
$CP$ violation is one of the key ingredients needed to explain the baryon-antibaryon asymmetry of the universe, which is fundamental to our existence. It is  well established that the $CP$ violation observed in the quark sector of the SM is too small to explain this asymmetry. Thus, we hope to discover additional sources of $CP$ violation in the Higgs sector or in the neutrino sector. This is among the most important motivations for upcoming and future experiments.

Observables such as total and differential cross sections are $CP$-invariant, and so the observation of $CP$ violation requires additional observables beyond those discussed in the previous section.
In physics channels with sufficiently complex final states,  $CP$-sensitive triple products can be constructed, or $\tau$ polarization can be exploited. In other cases, however, initial state
polarization, and here especially transverse polarization is the only possibility. While it is not part of the
default ILC operating scenario, the spin rotators could provide transverse polarization. The ILC polarimeters can measure this transverse polarisation when equipped with an appropriate detector for the Compton-scattered electrons~\cite{Mordechai:2013zwm}. For measurements sensitive to longitudinal polarization only, data-sets  with transverse polarizations are equivalent to data setz with unpolarized beams, just as it was the case at LEP~\cite{Arnaudon:1992rn}, or at HERA~\cite{Kramer:1994cv} before the installation of spin rotators.

The importance of positron polarization in searches for $CP$ violation in di-boson production depends on the number of parameters used to describe cross sections.   For the process $e^+e^- \to W^+W^-$, an Effective Field Theory description with dimension-6 operators only gives 5 parameters to describe the new physics effects. Of these, 3 are $CP$-conserving and 2 are $CP$-violating.  All of these parameters can be extracted using only longitudinal polarization.  Positron polarization is not required, though it gives a quantitative advantage in the statistical precision~\cite{Fujii:2017vwa,Marchesini:2011aka,Rosca:2016hcq}.  Similarly, in $e^+e^-\to \gamma Z$, the EFT description gives only one $CP$-violating parameter, which can be measured even without any beam polarization.

However, if we would like to test the most general parameter set for these reactions, the situation is different.  Assuming only Lorentz invariance, the triple gauge boson vertices in  $e^+e^-\to W^+W^-$ allows 14 complex (or 28 real) parameters~\cite{Diehl:2003qz}. Several of these parameters require positron polarization for their determination.  In particular, the $CP$ violating parameter $h^+ = \mbox{Im}(g^R_1 + \kappa^R_1)/\sqrt{2}$ appears in the cross section formula in a term that contains the product of \ensuremath{\P_{e^+}}\ and \ensuremath{\P_{e^-}}\ and so explicitly requires both beams to be (transversely) polarized. Similar arguments apply to the determination of the most general parameters for the triple gauge boson vertices contributing to $e^+e^-\to \gamma Z$~\cite{MoortgatPick:2005cw,POWER_SB2009}. Measurements based on these most general parameter sets are only possible with positron polarization.

\section{Precision Characterization of the 125-GeV Higgs Boson}
\label{sec:higgs}
The most obvious effect of positron polarization on Higgs physics is an increase of the number of produced Higgs bosons
by about $20\%$: Assuming SM cross sections, about 420 000 Higgs bosons will be produced in the \ensuremath{\P_{e^+}}=0 scenario,
while this number increases to about 500 000 in the \ensuremath{\P_{e^+}}=30\% case. These numbers are based on the luminosity sharing between helicity configurations as defined at the end of Section~\ref{sec:polbasics}.  Thus without positron polarization, the running time (and thus the running costs) would need to be increased by 19\% in order to reach the same number of produced Higgs bosons.

The report~\cite{Barklow:2017suo} presented projections of uncertainties in Higgs boson couplings using an analysis based on Effective Field Theory.  That analysis was based on the \ensuremath{\P_{e^+}}=30\% scenario. The results of the same analysis redone for the \ensuremath{\P_{e^+}}=0 case and also with both polarizations zero are shown in Table~\ref{tab:EFThiggs}. In this
interpretation, the loss of positron polarization has only a relatively small effect, degrading the Higgs coupling ratios $g(h\tau\tau)/g(hWW)$ and $g(hbb)/g(hWW)$ by $6\%$, with smaller effects on the absolute coupling determinations. This is mainly a statistical effect consistent with the drop in $\ensuremath{\L_{\eff}}$ explained in Section~\ref{sec:polbasics}, since coupling precisions scale inversely with the square root of the luminosity.
\begin{table}[h]
\begin{center}
\begin{tabular} {lcccccc}
 & no pol.   & 80\%/0\%    & 80\%/30\% \\
\hline
$g(hbb)$                           &    1.33        &  1.13        & 1.09  \\
$g(hcc)$                            &    2.09        &  1.97        & 1.88  \\
$g(hgg)$                           &    1.90        &  1.77        & 1.68  \\
$g(hWW)$                        &    0.978      &  0.683      & 0.672 \\
$g(h\tau\tau)$                   &    1.45        &  1.27        & 1.22   \\
$g(hZZ)$                          &    0.971       &  0.693     & 0.682  \\
$g(h\gamma\gamma)$     &    1.38        &  1.23        & 1.22   \\
$g(h\mu\mu)$                   &    5.67        &  5.64        & 5.59   \\
$g(h\gamma Z)$               &    14.0        &  6.71        & 6.63   \\
\hline
$g(hbb)/g(hWW)$             &    0.911      &  0.909       & 0.861 \\
$g(h\tau\tau)/g(hWW)$     &    1.08        &  1.08         & 1.02   \\
$g(hWW)/g(hZZ)$            &     0.070      &  0.067       & 0.067 \\
\hline
$\Gamma_h$                   &    2.93         &  2.60         & 2.49   \\
\hline
$BR(h\to inv)$                  &    0.365       &  0.327       & 0.315  \\
$BR(h\to other)$              &    1.68          &  1.67         & 1.58   \\
\end{tabular}
\caption{Projected relative errors for Higgs boson couplings and other Higgs observables at 250 GeV, in \%,
comparing three cases of beam polarization: $2$\,ab$^{-1}$ with \ensuremath{\P_{e^-}} = \ensuremath{\P_{e^+}} = 0\%, as well as the \ensuremath{\P_{e^+}}=0 and \ensuremath{\P_{e^+}}=30\% scenarios defined in the Introduction.}
\label{tab:EFThiggs}
\end{center}
\end{table}

However it should be noted that these results were obtained by scaling only the statistical uncertainties and assuming that all the systematic uncertainties stay the same, independently of the polarization. In particular
it was assumed that the uncertainties on the measured cross sections ($\times BR$) due to finite knowledge of the luminosity and the polarization are $0.1\%$ each, whereas for $H\to b\bar{b}$-channels an additional uncertainty of $0.1\%$ on the $b$-tagging efficiency has been considered. No uncertainties on e.g.\ the residual background contributions etc.\ were taken into account. While it is clearly better than not including any systematic uncertainties at all, this scheme does not reflect the increase in systematic uncertainties expected for the \ensuremath{\P_{e^+}}=0 case:
\begin{itemize}
\item {\bf luminosity uncertainty}: As we discussed for the example of SM cross section and asymmetry measurements in Section~\ref{sec:sm:xs}, the precision with which a global scaling uncertainty can be pinned down by treating it as a nuisance parameter in a global fit of many observables can depend quite significantly on the presence or absence of positron polarization.
\item {\bf polarization uncertainty}: Likewise we showed in Section~\ref{sec:sm:xs} that the precision to which the positron polarization can be determined in-situ depends significantly on its absolute value. For \ensuremath{\P_{e^+}}=0, it will be given de facto by the positron polarimeter measurement alone, and any bias in the polarimeter measurement
will propagate into $e^+e^-$ cross section measurements.
\item {\bf $b$-tagging uncertainty}: It still needs to be investigated in the future in how far the $b$-tagging efficiency could be constrained better
from data when using all four datasets which are available in the \ensuremath{\P_{e^+}} = 30\% scenario, similar to the EW cross section example given below.
\item {\bf background uncertainty}: While such types of uncertainty are not included at all in the current Higgs coupling fit, it has been shown in other contexts, e.g.\ the $WW$-threshold scan~\cite{Wilson:2016hne}, that the datasets which have beam helicity configurations unfavourable for the signal can give a very important handle on the residual background. For many Higgs channels, $W$-pair production is an important background. In principle $WW$ and $ZH$ production exhibit already very different polarization dependence with electron polarization only.
It should be subject of future studies whether this gives enough redundancy to control systematic uncertainties, since actually the left-right asymmetry of the $ZH$ cross section plays an important role as input observable in EFT-based fits~\cite{Barklow:2017suo,Durieux:2017rsg}. Therefore, positron polarization is needed here as an extra handle in order to simultaneously constrain the left-right asymmetry of the signal and background contributions with different polarization.
\end{itemize}

Finally, though the EFT-based Higgs coupling fit has strong theoretical justification, it would also be desirable to test the validity of this framework from observations.  Some parameters in this fit are constrained by precision electroweak measurements and constraints on anomalous triple gauge couplings. In~\cite{Barklow:2017suo}, the triple gauge couplings are constrained to depend on only 3 parameters (corresponding to shifts in $g_Z$, $\kappa_{\gamma}$ and $\lambda_{\gamma}$), as predicted by the EFT formalism in leading order. As we have discussed in Section~\ref{sec:sm:cp}, it is possible at the ILC to constrain the most general set of triple gauge coupling deviations allowed by Lorentz invariance, but only if both polarized electron and positron beams are available. The presence of positron beams would then allow new tests that could confirm the constraints predicted by the EFT formalism or, alternatively, might indicate new light particles that would require further corrections outside this formalism.

\section{Physics beyond the Standard Model}
\label{sec:bsm}
As opposed to the Higgs and SM precision program, which is guaranteed at the ILC, statements about new physics models could well remain speculative until we have ILC data in hand.
While a discovery of course would be the ultimate triumph, also exclusion bounds can change the paradigms of our thinking. A recent example are the null-results in searches for plain vanilla SUSY as predicted by the constrained MSSM in the first run of the LHC. We have explained in~\cite{Fujii:2017vwa, Fujii:2017ekh} that the ILC, even at 250\,GeV, has an interesting window for new particle discovery consistent with the exclusions of new particles reported by the LHC experiments. However, still, these particles might not appear.

The role of positron polarization in BSM physics is sometimes discussed separately for discovery or exclusion, on one hand, and for characterization of the signal once it has been found, on the other hand, implying that e.g.\ in case of the ILC a polarized source could be built once a discovery has been made.

There are examples where this approach is valid, e.g.\ rather plain-vanilla SUSY, where e.g.\ pair production of (sufficiently light) scalar electrons could be disovered with a large significance after a few weeks or months of data taking~\cite{Berggren:2015qua}, even without positron polarization. Then, indeed a polarized source would be essential and could be added in order to perform a full study of the chiral structure of the new states~\cite{MoortgatPick:2005cw,Moortgat-Pick:redreport}.

In general, though, and especially for the highest accessible masses and the lowest detectable couplings, one could very well end up in a situation where a deviation from the SM becomes visible only with the full  dataset, possibly even with a medium significance between $3$ and $5\,\sigma$. In such cases, the ability to test the ``characterization'' part of the BSM program by exploiting the extra observables provided by positron polarization would give important and in some cases even crucial hints for convincing ourselves that the observed effect is indeed new physics, and would narrow down the possible interpretations. In other words, only in special cases is there a strict separation between ``discover first'' and ``characterize later'', while in real life, both are often intertwined.

Thus, there are examples where a discovery of new particle might be missed without the assistance of positron polarization.  The most important of these are the following:

\begin{itemize}
\item {\bf Invisible particles:} A very important search in which $e^+e^-$ colliders offer a distinct advantage is the search for pair production of invisible particles in association with initial-state radiation. This search can uncover WIMP dark matter particles and other very weakly coupled states. At an $e^+e^-$ collider, this is the mono-photon (photon + missing energy) signature~\cite{Habermehl:2017dxh,Bartels:2012ex}. The SM predicts a low rate of mono-photon events, and that rate is calculable at the part per mille level.  The pair-production reaction has a definite chirality, so that it occurs either in the LR/RL or LL/RR chirality configurations.  Thus, positron polarization allows us to collect event samples in which the new physics effect is reduced and the SM backgrounds can be measured.

For the model of invisible particle production through an effective operator, the exclusion reach in the plane of the particle mass $M_\chi$ and the mediator scale $\Lambda$ has been evaluated for \ensuremath{\P_{e^+}}=30\% and \ensuremath{\P_{e^+}}=0 based on~\cite{Habermehl:2017dxh}. Since the cross section depends on the mediator scale as $\Lambda^{-4}$, the extension with positron polarization is significant, the equivalent of a 25\% increase in running time\footnote{With \ensuremath{\P_{e^+}}=0, an additional 940\,fb$^{-1}$ would be needed to reach the same sensitivity. Assuming the luminosity upgrade with an annual luminosity of 384\,fb$^{-1}$, this corresponds to 2.5 years of additional operation, which is about 25\% of the running time foreseen for the 250\,GeV stage.}. This does not account the ability to test the control of systematics, as described in the previous paragraph, or the fact that, in the event of a discovery, positron polarization can assist in determining the operator structure of the production process~\cite{Bartels:2012ex,Dreiner:2012xm,Chae:2012bq,Choi:2015zka}.

\item {\bf Heavy Leptons:} The study of $W$ pair production can reveal $t$-channel exchange of new heavy leptons with masses beyond the ILC collider energy. Exchange of such heavy leptons changes the chirality structure of the production process, so it is in principle distinguishable from modification of the triple gauge couplings, which leaves the production chirality-conserving. To measure this effect, however, double polarization asymmetries are needed~\cite{Moortgat-Pick:2013jra}.  Thus, for this search, positron polarization is required.

\item {\bf $R$-Parity Violating SUSY:} While there are strong bounds from the LHC on strongly interacting SUSY particles decaying into final states with large missing transverse energy (MET), electroweak new particles are much less constrained, especially in $R$-parity violating models, which give small MET. A search that is particularly powerful for $e^+e^-$ colliders is that for $s$-channel exchange of a scalar neutrino, which would then decay to $\mu^+\mu^-$ or $\tau^+\tau^-$.  This is another example in which the new process has a chirality structure different from that of the SM. In this case, longitudinal positron polarization increases the signal-to-background ratio by more than a factor 2~\cite{Moortgat-Pick:redreport}. This leads to an increased discovery reach, not only to higher masses, but also to smaller R-parity violating couplings.

\item {\bf Contact Interactions:} Finally, a model-independent search for contact interactions in Bhabha scattering profits significantly from the ability to measure the cross sections for all four helicity combinations which are available with both beam polarized~\cite{Moortgat-Pick:redreport}. In this example, positron polarization increases the reach in probed energy scales of new physics by a factor~1.3.

\end{itemize}

\section{Conclusions}
\label{sec:conclusions}
In this document we discussed the impact of positron polarization on the ILC physics program, with special emphasis on a first energy stage at 250\,GeV. While the  {\em statistical} effects on many standard
Higgs and electroweak observables are small, and could be compensated by longer operation time, positron polarization plays an important role in controlling {\em systematic} uncertainties, and thus enables us to fully exploit the potential of the ILC. Without positron polarization, many important measurements will be limited by systematic uncertainties.

In many cases positron polarization opens the door towards more model-independent interpretations of the data, which are important both in presence and in absence of discoveries. In case of discoveries, positron polarization plays a large role in identifying the underlying new physics model. But it also can be the decisive handle to identify an observation with (otherwise) medium significance as incompatible with the SM, and thus as a real discovery.

In conclusion, we find that positron polarization has an important role to play in the ILC program.

\bigskip  \bigskip \begin{center} \begin{large
The authors thank the ILD and SiD detector concepts groups for providing material for this document. We thank Peter Rowson and Michael Woods for insight into the SLD experience discussed in Section~\ref{sec:polarimetry}.
T.~Barklow, M.~Berggren, C.~Grojean, M.~Habermehl, R.~Karl, J.~List, G.~Moortgat-Pick and J.~Reuter thankfully acknowledge the support by the Deutsche Forschungsgemeinschaft (DFG) through the Collaborative Research Centre SFB 676 Particles, Strings and the Early Universe, projects B1 and B11. C.~Grojean is also supported by the European Commission through the Marie Curie Career Integration Grant 631962 and by the Helmholtz Association. F.~Simon acknowledges the support of the DFG cluster of excellence `Origin and Structure of the Universe'. R.~P\"oschl is supported by the Quarks \& Leptons programme of the French IN2P3. G.~W.~Wilson is supported by the US National Science Foundation under Award PHY-1607262. The work of T.~Barklow and M.~Peskin is supported by the U.S. Department of Energy, contract
DE-AC02-76SF00515.
Some material is based upon work supported by the U.S. Department of Energy, Office of Science,
Office of High Energy Physics under Award Number DE-SC-0009956.
This work is also supported by the Grants-in-Aid for Science Research No. 16H02173 and 16H02176 of the Japan Society for Promotion of Science (JSPS).

\input{bibliography}

\end{document}

High Energy Physics - Phenomenology (hep-ph); High Energy Physics - Experiment (hep-ex)


Interactive Trajectory Adaptation through Force-guided Bayesian Optimization

Abstract: Flexible manufacturing processes demand robots to easily adapt to changes in the environment and interact with humans. In such dynamic scenarios, robotic tasks may be programmed through learning-from-demonstration approaches, where a nominal plan of the task is learned by the robot. However, the learned plan may need to be adapted in order to fulfill additional requirements or overcome unexpected environment changes. When the required adaptation occurs at the end-effector trajectory level, a human operator may want to intuitively show the robot the desired changes by physically interacting with it. In this scenario, the robot needs to understand the human intended changes from noisy haptic data, quickly adapt accordingly and execute the nominal task plan when no further adaptation is needed. This paper addresses the aforementioned challenges by leveraging LfD and Bayesian optimization to endow the robot with data-efficient adaptation capabilities. Our approach exploits the sensed interaction forces to guide the robot adaptation, and speeds up the optimization process by defining local search spaces extracted from the learned task model. We show how our framework quickly adapts the learned spatial-temporal patterns of the task, leading to deformed trajectory distributions that are consistent with the nominal plan and the changes introduced by the human.

\section{INTRODUCTION}
The next generation of small and medium-sized enterprises (SMEs) is envisioned as places where humans and robots will fluently collaborate in a plethora of manufacturing processes. In current SMEs, the requirements of such processes may rapidly vary or a whole process could even change, requiring fast and efficient adaptation capabilities of workers. These conditions are clearly not a problem for human operators, as we exhibit extraordinary adaptation skills when faced with uncertain and highly-dynamic environments. However, when it comes to introduce robotic coworkers in SMEs, the required adaptation features for SMEs processes are still under development.

Humans working in SMEs daily collaborate with each other in a large diversity of scenarios, showing complex adaptation capabilities that involve role specialization, motion re-planning, and role switching~\cite{ReedPeshkin:PhysicalHRI08}. Concerning human-robot collaboration (HRC), adaptation has been addressed either on one of the agents (i.e. leader-follower approaches), or on both of them (i.e. mutual adaptation)~\cite{Nikolaidis17:Models4HRC}. In both cases, robot adaptation is imperative to account for environment uncertainties, to respond to unseen task conditions, and to handle varying human preferences. In the latter case, it is difficult to build an accurate model of the human actions and internal state as it demands an understanding of human behavior when interacting with robotic systems. However, by exploiting the information obtained from several human-robot interactions and the human knowledge about the task, the aforementioned modeling and adaptation problems may be partly alleviated.

When collaboration involves physical interaction among humans, haptic cues are often exploited to communicate intentions or negotiate roles~\cite{ReedPeshkin:PhysicalHRI08}. HRC settings may similarly exploit this information to, for example, guide the robot adaptation (i.e. exploration) when new actions are required for a new task situation. Argall~\MakeLowercase{\textit{et al.}}~\cite{Argall10:TactileCorrection} used tactile feedback to convey human corrections for a policy that was initially learned from demonstrations. These tactile corrections were translated into incremental shifts for the robot pose, which were then used to either improve a current model or learn a different solution for the task. Human-guided exploration using force data was also employed to learn haptic affordances in robotic manipulation in~\cite{Chu16:HapticAffordance,Chu17:GuidedExploration}. The robot action search space was initialized from demonstrations, while the exploration was determined by the human guidance. The foregoing works provided the human with full control to govern the robot exploration, meaning that the robot was nearly behaving as a pure follower when human guidance took place.

\begin{figure}[!tbp]
	\includegraphics[width=.22\textwidth]{RobotTraj}
	\includegraphics[width=.22\textwidth]{RobotTrajPushed}
	\caption{Illustration of the nominal plan execution learned from human demonstrations (\textit{left}), and the interactive trajectory adaptation phase (\textit{right}). When no adaptation is needed, the robot executes the task by following a reference trajectory distribution (green solid line depicting the mean and shared area representing the variance) extracted from the nominal plan (light orange ellipse). A human operator can introduce local trajectory deformations by physically interacting with the robot, which adapts the nominal task plan (dark orange ellipse) so that disagreement forces are minimized.}
	\label{Fig:Illustration}
\end{figure}

Robots can adapt more autonomously if human guidance provides only partial information about the possible exploration directions. Schroecker~\MakeLowercase{\textit{et al.}}~\cite{Schroecker16:DirectingPolicySearch} proposed to use human demonstrations to provide the robot with salient (i.e. relevant) points of a trajectory while the robot autonomously learned the shape of the in-between trajectory. These points also served as online corrections to guide a reward-dependent robot exploration. However, handcrafting reward functions is tedious, error-prone, and becomes significantly more complex when a robot needs to collaborate with a human partner and adapt on-the-fly to her preferences.

A potential alternative solution to endow robots with adaptation capabilities without explicitly defining an objective function is Bayesian optimization (BayesOpt)~\cite{Shahriari16:BayesOpt}. BayesOpt is a gradient-free global optimization of black-box objective functions that are expensive to evaluate and often multimodal. These characteristics properly fit the basic requirements of robots that need to adapt to new task conditions (i.e. expensive trials, unknown or hard-to-model objective function and data efficiency). BayesOpt has recently gained interest in different robotic applications, such as behavior adaptation for damaged robots~\cite{Cully15:RobotsAsAnimal}, automatic controller tuning for balancing~\cite{Marco16:LQRbayesOpt}, locomotion~\cite{Antonova17:DeepKernelsBO}, and interaction tasks~\cite{Drieb17:ConstBayesOptForceTask}, and in physical HRC~\cite{Ghadirzadeh16:RL4pHRI,Kupcsik15:HandoverBayes}. BayesOpt was used in~\cite{Ghadirzadeh16:RL4pHRI} to select the collaborative actions that minimize a Q-function in a model-based reinforcement learning approach. Interaction forces and positional data, representing the robot state, were exploited to determine the exploration actions. In~\cite{Kupcsik15:HandoverBayes}, the authors employed BayesOpt to approximate an unknown reward function for a contextual policy search aimed at learning a handover skill by interacting with a human. However, both~\cite{Ghadirzadeh16:RL4pHRI,Kupcsik15:HandoverBayes} did not exploit the fact that the robot may learn an initial policy from demonstrations, which may speed up the learning process and reduce explorations.

Inspired by the insights on haptic communication~\cite{ReedPeshkin:PhysicalHRI08} and robot adaptation requirements in HRC~\cite{Nikolaidis17:Models4HRC}, we propose to exploit BayesOpt to adapt the robot nominal task plan through physical human guidance. Specifically, our framework learns a nominal plan of the task from human demonstrations using a hidden semi-Markov model~\cite{Yu:HSMM10,Rozo16Front:ADHSMM}. This model is later used to compute a smooth reference trajectory distribution representing the sensorimotor patterns of the task, as explained in Section~\ref{sec:Learning}. The nominal plan allows the robot to take a leading role during the task execution under normal conditions. A human operator can physically interact with the robot to indicate, through force-based cues, a desired adaptation for a new task situation, as illustrated in Fig.~\ref{Fig:Illustration}. BayesOpt is used to carry out a local adaptation of the nominal model so that the interaction forces (measuring the human-robot disagreement) are minimized (see Section~\ref{sec:BayesOpt}).

A similar work on physically interactive trajectory deformations used an analytical smooth family of trajectories to find the local spatial deformations as a function of the applied force~\cite{Losey18:interactiveTrajDeformation}. The analytical formulation allowed to use gradient-based optimization to find the parameters of the deformed trajectory. In contrast, our approach allows for both spatial and temporal adaptation of the nominal task plan, and provides data-efficient adaptation by confining BayesOpt to carry out local searches at the level of the model states distribution, which significantly reduces the parameter space dimensionality. Moreover, the search space is automatically defined from the learning model, sharing some similarities with safe BayesOpt approaches~\cite{Berkenkamp16:safeBO}. The proposed framework is evaluated in Section~\ref{sec:Results} for different instances of a simulated 2D pick-and-place task, where data-efficient trajectory adaptations exploiting force-based guidance are successfully reported.

\section{LEARNING A NOMINAL TASK PLAN}
\label{sec:Learning}
Robots can take advantage of learning-from-demonstration (LfD) approaches to learn to execute a task or collaborate with a human partner. Human demonstrations can be encoded by a probabilistic model that represents the nominal plan of the task for the robot. This model can subsequently be used to generate the desired robot movements as a function of the state of the human partner and the environment. To do so, we use a hidden semi-Markov model (HSMM)~\cite{Yu:HSMM10} to represent the nominal task plan of the robot. This model and its classical formulation (HMM) have been successfully exploited to learn manipulation skills with force sensing~\cite{RozoEtAl11:HMMGMRa,RozoEtAl13:ForceLfD}, in semi-autonomous teleoperation~\cite{Tanwani19:nonParamRobotLearn}, and in HRC~\cite{MedinaEtAl:CollForce11}.  HSMM allows us to encapsulate not only the observed sensorimotor patterns but also the temporal structure of the task. This model is then combined with a trajectory generation approach that exploits task dynamic features to retrieve a smooth reference distribution of sensorimotor trajectories. This is later used by the robot to both execute the desired task and monitor deviations that indicate an adaptation process. Both the learning model and retrieval of sensorimotor trajectories are described next.

\subsection{Hidden semi-Markov model}
\label{subsec:HSMM}
A $K$-states hidden Markov model (HMM) is characterized by an initial state distribution $\{\pi_i\}_{i=1}^K$, a transition probability matrix $\{a_{ij}\}_{i,j=1}^K$, and an observation distribution for each state $i$ in the model, commonly represented by a Gaussian distribution $\mathcal{N}(\bm{\mu}_i, \bm{\Sigma}_i)$, with mean $\bm{\mu}_i$ and covariance matrix $\bm{\Sigma}_i$. In HMM, the self-transition probabilities $a_{i,i}$ only allow a crude implicit modeling of the state duration which follows a geometric distribution $\mathcal{P}_i(d) = a_{i,i}^{d-1} (1-a_{i,i})$, decreasing exponentially with time~\cite{Rabiner89:TutorialHMM}. Thus, HMM is not suitable to encode tasks where temporal patterns are relevant.

Variable duration modeling techniques such as \emph{hidden semi-Markov model} (HSMM) extend standard HMMs by embedding temporal information of the underlying stochastic process. That is, while in HMM the underlying hidden process is assumed to be Markov, i.e., the probability of transitioning to the next state depends only on the current state, in HSMM the state process is assumed semi-Markov.
This means that a transition to the next state depends on the current state as well as on the elapsed time since the state was entered~\cite{YuKobayashi:efficientHSMM06}. Since the state duration is always positive, its distribution should preferably be modeled by a function preserving this property. Thus, we here follow the approach proposed in~\cite{Rozo16Front:ADHSMM} and use a univariate lognormal distribution $\mathcal{N}(\mu^\ty{D}_i,\sigma^\ty{D}_i)$ with mean $\mu^\ty{D}_i$ and associated variance $\sigma^\ty{D}_i$ to model the logarithm of the duration, which is equivalent to the use of a lognormal distribution to fit the duration data. Therefore, an HSMM is characterized by the parameters set $ {\bm{\Theta}} = \left\{ \{a_{ij}\}_{j=1}^K, \mu_i^\ty{D}, \sigma_i^\ty{D}, \pi_i, \bm{\mu}_i, \bm{\Sigma}_i \right\}_{i=1}^K$, which can be trained by an expectation-maximization procedure.

Once trained, an HSMM can be used to derive a nominal task plan in the form of a desired sequence of states ${\bm{s}_{1:T}=\{s_1,s_2,\ldots,s_T\}}$ for a given time horizon of length $T$ and a set of discrete states ${s_t\in\{1,\ldots,K\}}$. To do so, we exploit the definition of the \emph{forward variable} in HSMM to compute the probability to be in state $i$ at time step $t$ and observe the partial observation $\bm{\zeta}_{1:t}~=~ \{\bm{\zeta}_1,\bm{\zeta}_2,\ldots,\bm{\zeta}_t\}$, namely $\alpha_{t,i} \triangleq \mathcal{P}(s_t\!=\!i \,,\, \bm{\zeta}_{1:t} )$, which is recursively computed with (see~\cite{Rabiner89:TutorialHMM} for details)\footnote{Equation~\eqref{eq:alphahsmm} can be efficiently computed as in~\cite{Yu:HSMM10, YuKobayashi:efficientHSMM06}.}
\begin{align}
\begin{split}
\alpha_{t,i}
&= \sum\limits_{d=1}^{d^{\max}} \sum\limits_{j=1}^K
\alpha_{t-d,j} \; a_{j,i} \; \mathcal{N}^\ty{D}_{d,i}
\hspace{-.2cm}\prod\limits_{s=t-d+1}^{t}\hspace{-.2cm} \mathcal{N}_{s,i}
\label{eq:alphahsmm}
\end{split}\\
\text{where}\quad \mathcal{N}^\ty{D}_{d,i}
&= \mathcal{N}\big(\log(d) |\; \mu^\ty{D}_i,\sigma^\ty{D}_i\big)
\;\quad\text{and}\;\\
\mathcal{N}_{s,i} &= \mathcal{N}\big(\bm{\zeta}_s |\; \bm{\mu}_i,\bm{\Sigma}_i\big) .\nonumber
\end{align}
For $t\!<\!d^{\max}$, the initialization is given by
\begin{gather*}
\alpha_{1,i} = \pi_i \; \mathcal{N}^\ty{D}_{1,i} \; \mathcal{N}_{1,i} ,
\\
\alpha_{2,i} = \pi_i \; \mathcal{N}^\ty{D}_{2,i} \prod_{s=1}^2 \! \mathcal{N}_{s,i}
+\!
\sum_{j=1}^K
\alpha_{1,j} \; a_{j,i} \; \mathcal{N}^\ty{D}_{1,i} \mathcal{N}_{2,i} ,
\\
\alpha_{3,i} = \pi_i \; \mathcal{N}^\ty{D}_{3,i} \prod_{s=1}^3 \!\ \mathcal{N}_{s,i}
+\!
\sum\limits_{d=1}^{2} \sum_{j=1}^K
\alpha_{3-d,j} \; a_{j,i} \; \mathcal{N}^\ty{D}_{d,i}
\hspace{-.2cm}\prod\limits_{s=4-d}^{3}\hspace{-.2cm} \mathcal{N}_{s,i} ,
\end{gather*}
etc., which corresponds to the update rule
\begin{equation}
\alpha_{t,i} = \pi_i \; \mathcal{N}^\ty{D}_{t,i} \prod_{s=1}^t \! \mathcal{N}_{s,i}
+\!
\sum\limits_{d=1}^{t\!-\!1}	\sum_{j=1}^K
\alpha_{t-d,j} \; a_{j,i} \; \mathcal{N}^\ty{D}_{d,i}
\hspace{-.3cm}\prod\limits_{s=t-d+1}^{t}\hspace{-.3cm} \mathcal{N}_{s,i} .
\end{equation}

Then, given a time horizon $\{1\,:\,T\}$ and the definition of the forward variable~\eqref{eq:alphahsmm}, we can obtain the most likely sequence of states from
\begin{equation}
	s_t = \argmax_{i} \alpha_{t,i} .
\end{equation}
HSMM can be viewed as a model representing a high-level abstraction of the task, which encapsulates the observed sensorimotor and temporal patterns through the set observation, duration and transition probabilities. This model is here exploited to \emph{(1)} retrieve a smooth reference trajectory distribution to drive the robot motion (Section~\ref*{subsec:trajGMM}) and \emph{(2)} localize the force-guided adaptation (Section~\ref{subsec:BayesOpt}).

\subsection{Trajectory generation using dynamic features}
\label{subsec:trajGMM}
In order to retrieve the reference trajectory distribution from HSMM, we resort to an approach that exploits both static and dynamic features of the observed data, encapsulated in the observation and duration probability distributions. In robotics, this provides a simple approach to synthesize smooth trajectories, which is achieved by coordinating the distributions of both static and dynamic features in the considered time series. This approach has rarely been exploited in robotics, at the exception of the works from~\cite{Rozo16Front:ADHSMM,SugiuraEtal:MotionLearningTrajHMM11} employing it to represent object manipulation movements or collaborative behaviors. We here take advantage of this approach for retrieving a smooth reference trajectory distribution that will drive the robot motion according to the nominal task plan encoded by HSMM.

Formally, let us define the state of the robot as ${\bm{\xi} \in \mathbb{R}^D}$, which can represent the robot end-effector pose, its joint configuration, or be composed of additional sensory information such as sensed Cartesian forces or joint torques. For sake of simplicity, we here present the retrieval of a reference distribution of trajectories of the robot end-effector position ${\bm{x} \in \mathbb{R}^D}$, with $D=3$. However, the approach can be straightforwardly applied to alternative robot state representations as explained at the end of this section.

For encoding robot movements, Cartesian velocities $\dot{\bm{x}}$ and accelerations $\ddot{\bm{x}}$ can be used as dynamic features of the robot motion. By considering an Euler approximation, they are computed as
\begin{equation}
\dot{\bm{x}}_t = \frac{\bm{x}_{t+1}-\bm{x}_t}{\Delta t} ,
\ddot{\bm{x}}_t = \frac{\dot{\bm{x}}_{t+1}-\dot{\bm{x}}_t}{\Delta t} =
\frac{\bm{x}_{t+2}-2\bm{x}_{t+1}+\bm{x}_t}{\Delta t^2} .
\label{eq:trajGMM_dx_ddx}
\end{equation}
By using \eqref{eq:trajGMM_dx_ddx}, the observation vector $\bm{\zeta}_t$ introduced in Section \ref{subsec:HSMM} will be used to represent the concatenated position, velocity and acceleration vectors at time step $t$, as follows
\begin{equation}
\bm{\zeta}_t=\left[
\begin{matrix}\bm{x}_t \\ \dot{\bm{x}}_t \\ \ddot{\bm{x}}_t\end{matrix}\right]
= \left[\begin{matrix}\bm{I} & \bm{0} & \bm{0} & \\
-\frac{1}{\Delta t}\bm{I} & \frac{1}{\Delta t}\bm{I} & \bm{0} \\
\frac{1}{\Delta t^2}\bm{I} & -\frac{2}{\Delta t^2}\bm{I} & \frac{1}{\Delta t^2}\bm{I}
\end{matrix}\right]
\left[
\begin{matrix} \bm{x}_t \\ \bm{x}_{t+1} \\ \bm{x}_{t+2}
\end{matrix}\right] ,
\label{eq:Phi1D}
\end{equation}
where $\bm{I} \in \mathbb{R}^{D\times D}$ is the identity matrix and ${\Delta t}$ the sampling time. Note that the number of derivatives is set up to acceleration, but the results can be generalized to a higher or lower number of derivatives. Then, new variables $\bm{\zeta}$ and $\bm{x}$ are defined as large vectors by concatenating $\bm{\zeta}_t$ and $\bm{x}_t$ for all time steps, namely $\bm{\zeta}=\left[\begin{matrix}\bm{\zeta}^\mathsf{T}_1 \> \bm{\zeta}^\mathsf{T}_2 \> \hdots \> \bm{\zeta}^\mathsf{T}_T \end{matrix}\right]^\mathsf{T}$ and ${\bm{x}=\left[\begin{matrix}\bm{x}^\mathsf{T}_1 \> \bm{x}^\mathsf{T}_2 \> \hdots \> \bm{x}^\mathsf{T}_T \end{matrix}\right]^\mathsf{T}}$. Similarly to the matrix operator \eqref{eq:Phi1D} defined for a single time step, a large sparse matrix $\bm{\Phi}$ can be defined so that $\bm{\zeta}\!=\!\bm{\Phi}\bm{x}$, namely\footnote{Note that a similar operator is defined to handle border conditions, and that $\bm{\Phi}$ can be constructed through the use of Kronecker products.}
\begin{equation}
\overbrace{
	\left[\begin{matrix}
	\vdots \\
	\color{rr}\bm{x}_t \\ \color{rr}\dot{\bm{x}}_t \\ \color{rr}\ddot{\bm{x}}_t \\
	\color{bb}\bm{x}_{t+1} \\ \color{bb}\dot{\bm{x}}_{t+1} \\ \color{bb}\ddot{\bm{x}}_{t+1} \\
	\vdots
	\end{matrix}\right]}^{\bm{\zeta}}\!\!=\!\!
\overbrace{
	\left[\arraycolsep=2.5pt
	\begin{matrix}
	\ddots & \vdots & \vdots & \vdots & \iddots & \\
	\cdots & \color{rr}\bm{I} & \color{rr}\bm{0} & \color{rr}\bm{0} & \cdots & \\
	\cdots & \color{rr}-\frac{1}{\Delta t}\bm{I} & \color{rr}\frac{1}{\Delta t}\bm{I} & \color{rr}\bm{0} & \cdots &\\
	\cdots & \color{rr}\frac{1}{\Delta t^2}\bm{I} & \color{rr}-\frac{2}{\Delta t^2}\bm{I} &
	\color{rr}\frac{1}{\Delta t^2}\bm{I} & \cdots & \\
	& \cdots & \color{bb}\bm{I} & \color{bb}\bm{0} & \color{bb}\bm{0} & \cdots\\
	& \cdots & \color{bb}-\frac{1}{\Delta t}\bm{I} & \color{bb}\frac{1}{\Delta t}\bm{I} & \color{bb}\bm{0} & \cdots\\
	& \cdots & \color{bb}\frac{1}{\Delta t^2}\bm{I} & \color{bb}-\frac{2}{\Delta t^2}\bm{I} &
	\color{bb}\frac{1}{\Delta t^2}\bm{I} & \cdots \\
	& \iddots & \vdots & \vdots & \vdots & \ddots\\
	\end{matrix}\right]}^{\bm{\Phi}}\!\!
\overbrace{
	\left[
	\begin{matrix}
	\vdots \\
	\color{rr}\bm{x}_t \\ \color{rb}\bm{x}_{t+1} \\ \color{rb}\bm{x}_{t+2} \\ \color{bb}\bm{x}_{t+3} \\
	\vdots
	\end{matrix}\right]}^{\bm{x}}\!.
\label{eq:zeta}
\end{equation}

The state sequence ${\bm{s}_{1:T}=\{s_1,s_2,\ldots,s_T\}}$ representing the nominal task plan can be exploited here to retrieve a reference trajectory distribution used to drive the robot end-effector movements. To do so, we define the likelihood of a movement $\bm{\zeta}$ for a given sequence $\bm{s}$ as
\begin{equation}
\mathcal{P}(\bm{\zeta}|\bm{s}) = \prod_{t=1}^T \mathcal{N}(\bm{\zeta}_t|\bm{\mu}_{s_t},\bm{\Sigma}_{s_t}) ,
\end{equation}
where $\bm{\mu}_{s_t}$ and $\bm{\Sigma}_{s_t}$ are the mean and covariance matrix of state $s_t$ at time step $t$. This state $s_t$, as defined in~\ref{subsec:HSMM}, is obtained from the most likely sequence of states sampled from the HSMM for a given horizon $\{1\,:\;T\}$ using the forward variable~\eqref{eq:alphahsmm} (note that we drop the subscript on $\bm{s}$ from now on). This product
can be rewritten a
\begin{gather}
\mathcal{P}(\bm{\zeta}|\bm{s}) = \mathcal{N}(\bm{\zeta}|\bm{\mu}_{\bm{s}},\bm{\Sigma}_{\bm{s}}) ,
\label{eq:Pzeta}
\\
\mathrm{with}\quad
\bm{\mu}_{\bm{s}} \!=\! \left[\begin{matrix}\bm{\mu}_{s_1} \\ \bm{\mu}_{s_2} \\ \vdots \\ \bm{\mu}_{s_T} \end{matrix}\right]
\:\text{and}\quad
\bm{\Sigma}_{\bm{s}} \!=\! \left[\begin{matrix}\bm{\Sigma}_{s_1} & \bm{0} & \cdots & \bm{0} \\
\bm{0} & \bm{\Sigma}_{s_2} & \cdots & \bm{0} \\
\vdots & \vdots & \ddots & \vdots \\
\bm{0} & \bm{0} & \cdots & \bm{\Sigma}_{s_T} \end{matrix}\right] .\nonumber
\end{gather}

By using the relation $\bm{\zeta}\!=\!\bm{\Phi}\bm{x}$, we then seek for a trajectory $\bm{x}$ maximizing the logarithm of \eqref{eq:Pzeta}, namely
\begin{equation}
\hat{\bm{x}} = \arg\max_{\bm{x}} \; \log\mathcal{P}(\bm{\Phi}\bm{x} \;|\;\bm{s}) .
\label{eq:MaxTrajGMM}
\end{equation}

The part of $\log\mathcal{P}(\bm{\Phi}\bm{x} \, | \, \bm{s})$ dependent on $\bm{x}$ takes the well-known quadratic error form ${c(\bm{x}) =
{(\bm{\mu}_{\bm{s}}-\bm{\Phi}\bm{x})^\mathsf{T} \bm{\Sigma}_{\bm{s}}^{-1} (\bm{\mu}_{\bm{s}}-\bm{\Phi}\bm{x})}}$.
Then, a solution can be found by differentiating $c(\bm{x})$ and equating to $0$, providing the trajectory (in vector form)
\begin{equation}
\hat{\bm{x}} = \left(\bm{\Phi}^\mathsf{T} {\bm{\Sigma}_{\bm{s}}}^{-1} \bm{\Phi}\right)^{-1}
\bm{\Phi}^\mathsf{T} {\bm{\Sigma}_{\bm{s}}}^{-1} \bm{\mu}_{\bm{s}} ,
\label{eq:trajHMM}
\end{equation}
with the covariance error of the weighted least-squares estimate given by
\begin{equation}
\hat{\bm{\Sigma}}^{\bm{x}} = \sigma \left(\bm{\Phi}^\mathsf{T} {\bm{\Sigma}_{\bm{s}}}^{-1} \bm{\Phi}\right)^{-1} ,
\label{eq:trajHMM_cov}
\end{equation}
where $\sigma$ is a scale factor. Both \eqref{eq:trajHMM} and \eqref{eq:trajHMM_cov} describe a reference trajectory distribution $\mathcal{N}(\hat{\bm{x}}, \hat{\bm{\Sigma}}^{\bm{x}})$, which represents the nominal task plan learned from demonstrations. Note that the aforementioned equations can be computed efficiently with Cholesky and/or QR decompositions by exploiting the positive definite symmetric band structure of the matrices.

As we are interested in exploiting haptic cues to indicate desired adaptations to the learned nominal plan, we propose to augment the robot state as $\bm{\xi}\! = \! [\bm{x}^\mathsf{T} \bm{f}^\mathsf{T}]^\mathsf{T}$, where $\bm{f} \in \mathbb{R}^D$ represents the sensed Cartesian forces at the robot end-effector, leading to ${\bm{\xi} \in \mathbb{R}^{2D}}$ and $\bm{I} \in \mathbb{R}^{2D\times 2D}$. In this case, the dynamic features of the sensed forces are also considered ~\eqref{eq:zeta}. As a consequence, on the basis of~\eqref{eq:MaxTrajGMM}, the trajectory retrieval for this augmented state is formulated as
\begin{equation}
\hat{\bm{\xi}} = \arg\max_{\bm{\xi}} \; \log\mathcal{P}(\bm{\Phi}\bm{\xi} \;|\;\bm{s}) ,
\label{eq:MaxTrajGMMaugmented}
\end{equation}
whose solution has the same form as~\eqref{eq:trajHMM} and~\eqref{eq:trajHMM_cov}. This augmented state $\bm{\xi}$ allows us not only to retrieve a desired end-effector trajectory distribution $\mathcal{N}(\hat{\bm{x}}, \hat{\bm{\Sigma}}^{\bm{x}})$, but also a reference force distribution $\mathcal{N}(\hat{\bm{f}}, \hat{\bm{\Sigma}}^{\bm{f}})$. The latter can be employed to set a reference force profile to track during the execution of the task as well as to identify external forces indicating a task adaptation phase. In this work, the reference force distribution $\mathcal{N}(\hat{\bm{f}}, \hat{\bm{\Sigma}}^{\bm{f}})$ will be mainly used to detect adaptation phases from physical interactions.

\section{TRAJECTORY ADAPTATION VIA FORCE-GUIDED BAYESIAN OPTIMIZATION}
\label{sec:BayesOpt}

Once a nominal plan has been learned, the robot is ready to carry out the task. However, during reproduction, new task requirements may arise (e.g., alternative motion trajectories, new locations of objects of interest), thus requiring the robot to adapt its nominal plan to the new situation. A way to indicate desired task adaptations is haptic communication. Here we allow a human operator to physically interact with the robot in order to show the required adaptation through force-based cues. Notice that in the case where no physical interaction is possible, artificial guidance forces could be computed from a virtual environment where a human commands a virtual proxy to indicate necessary adaptations.

Let us assume that interaction forces convey information about the task adaptation required by a human. In other words, force-based cues provide information about an unobservable reward/objective function that the human is trying to optimize through the trajectory adaptation. As crafting reward/objective functions is significantly cumbersome and data-efficient adaptation is imperative when a robot interacts with a human, we propose to exploit BayesOpt to adapt the parameters of the nominal plan. We here provide a short introduction to BayesOpt and later explain how this is exploited into the force-guided robot adaptation.

\subsection{Bayesian optimization}
\label{subsec:BayesOpt}
In general terms, we are considering the problem of finding a global maximizer (or minimizer) of an unknown objective function $f$
\begin{equation}
	\bm{\theta}^* = \argmax_{\bm{\theta} \in \mathcal{X}} f(\bm{\theta}) ,
\end{equation}
where $\mathcal{X} \subseteq \mathbb{R}^{D_{\mathcal{X}}}$ is some design space of interest, with $D_{\mathcal{X}}$ being the dimensionality of the parameter space. Furthermore, we assume that the black-box function $f$ has no simple closed form, but can be evaluated at any arbitrary query point $\bm{\theta}$ in the domain. This evaluation produces noise-corrupted outputs $y \in \mathbb{R}$ such that ${\mathbb{E}[y | f(\bm{\theta})] = f(\bm{\theta})}$. In other words, we can only observe the function $f$ through unbiased noisy point-wise observations $y$. In this setting, we consider a sequential search algorithm which, at iteration $n$, selects a location $\bm{\theta}_{n+1}$ at which to query $f$ and observe $y_{n+1}$. After $N$ queries, the algorithm makes a final recommendation $\bm{\theta}_N$, which represents the algorithm's best estimate.

BayesOpt prescribes a prior belief over the possible objective functions and then sequentially refines this model as data are observed via Bayesian posterior updating. Equipped with this probabilistic model, BayesOpt can sequentially induce acquisition functions $\gamma_n : \mathcal{X} \mapsto \mathbb{R}$ that leverage the uncertainty in the posterior to guide the exploration. Intuitively, the acquisition function evaluates the utility of candidate points for the next evaluation of $f$; therefore, $\bm{\theta}_{n+1}$ is selected by maximizing $\gamma_n$, where the index $n$ indicates the implicit dependence on the currently available data.

A common way to model the prior and posterior for $f$ is by using a Gaussian Process $f(\bm{\theta}) \sim \mathcal{GP}(\mu(\bm{\theta}), k(\bm{\theta}_i, \bm{\theta}_j))$ with mean function $\mu :  \mathcal{X} \mapsto \mathbb{R}$ and positive-definite kernel (or covariance function) $k : \mathcal{X} \times \mathcal{X} \mapsto \mathbb{R}$. Let $\mathcal{D}_n = \{(\bm{\theta}_i, y_i)\}_{i=1}^n$ denote the set of observations and $\tilde{\bm{\theta}}$ represent an arbitrary test point. The random variable $f(\tilde{\bm{\theta}})$ conditioned on observations $\mathcal{D}_n$ is also normally distributed with the following posterior mean and variance functions:
\begin{align}
	\mu_n(\bm{\theta}) &= \mu(\tilde{\bm{\theta}}) + \bm{k}(\tilde{\bm{\theta}})^\mathsf{T} (\bm{K} + \sigma^2 \bm{I})^{-1}(\bm{y} - \bm{\mu}(\bm{\theta})), \\
	\sigma_n^2(\bm{\theta}) &= k(\tilde{\bm{\theta}},\tilde{\bm{\theta}}) - \bm{k}(\tilde{\bm{\theta}})^\mathsf{T} (\bm{K} + \sigma^2 \bm{I})^{-1} \bm{k}(\tilde{\bm{\theta}}),
\end{align}
where $\bm{y}$ is the observed outputs vector, $\bm{k}(\tilde{\bm{\theta}})$ is a vector of covariance terms between $\tilde{\bm{\theta}}$ and $\bm{\theta}_{1:n}$, and $\bm{K}$ is the covariance matrix for all the pairs $\bm{\theta}_i$ and $\bm{\theta}_j$. The posterior mean and variance evaluated at any point $\tilde{\bm{\theta}}$ represent the model prediction and uncertainty, respectively, in the objective function at the point $\tilde{\bm{\theta}}$. These posterior functions are exploited to select the next query point $\bm{\theta}_{n+1}$ by means of an acquisition function.

An acquisition function performs a trade-off between exploitation (e.g. selecting the point with the highest posterior mean) and exploration (e.g. selecting the point with the highest posterior variance) using the information given by the posterior functions. Here we use expected improvement (EI), which incorporates the amount of improvement upon $\tau$, and can be analytically computed as follows
\begin{align}
	\gamma_{EI}(\bm{\theta}; \mathcal{D}_n) & = (\mu_n(\bm{\theta}) - \tau) \Phi\left( \frac{\mu_n(\bm{\theta}) - \tau}{\sigma_n(\bm{\theta})} \right) \nonumber \\
	&+ \sigma_n(\bm{\theta}) \phi\left( \frac{\mu_n(\bm{\theta}) - \tau}{\sigma_n(\bm{\theta})} \right) ,
	\label{eq:EI}
\end{align}
where $\Phi$ is the normal cumulative distribution function, $\phi$ represents the corresponding probability density function, and $\tau$ is the threshold improvement. Intuitively, EI selects the next parameter point where the expected improvement over $\tau$ is maximal (more details are given in~\cite{Shahriari16:BayesOpt,Mockus75:EI}).

\subsection{Force-guided robot adaptation}
\label{subsec:RobotAdapt}
As mentioned previously, the human partner can physically interact with the robot to indicate, through force-based cues, a desired adaptation of the learned nominal plan. In order to let the robot negotiate its adaptation according to the human intention (which is noisily observed), we here exploit BayesOpt to carry out a local adaptation of the nominal model so that the interaction forces (measuring the human-robot disagreement) are minimized. As our learning model encapsulates both sensorimotor and duration patterns, spatiotemporal adaptations are possible.

Formally, sensorimotor patterns and duration information are locally encoded as Gaussian distributions $\mathcal{N}(\bm{\mu}_i, \bm{\Sigma}_i)$ and $\mathcal{N}(\mu_i^{\mathcal{D}}, \sigma_i^{\mathcal{D}})$, as described in Section \ref{sec:Learning}. These distributions directly influence the robot task execution through~\eqref{eq:alphahsmm} and~\eqref{eq:MaxTrajGMM}. So, in order to locally adapt the robot trajectory, we define the vector of local model parameters as
\begin{equation}
\bm{\theta}_i = \left[
\begin{matrix}\bm{\mu}_i^{\mathcal{O}} \\ \mu_i^{\mathcal{D}} \end{matrix}\right] ,
\label{eq:thetaParams}
\end{equation}
where $\bm{\mu}_i^{\mathcal{O}}$ and $\mu_i^{\mathcal{D}}$ respectively represent the mean vector of motor commands and duration for state $i$.\footnote{Note that we omit covariance parameters for simplicity.} The vector of parameters $\bm{\theta}_i$ is thus optimized using BayesOpt to find the optimal $\bm{\theta}_{i}^*$ that minimizes the human-robot disagreement forces when a desired adaptation is triggered. Note that when no desired force profile is required for the task, the disagreement forces directly correspond to the noisy readings $\bm{f}^s$ of the force sensor mounted at the robot wrist. On the contrary, if a reference force distribution $\mathcal{N}(\hat{\bm{f}}, \hat{\bm{\Sigma}}^{\bm{f}})$ is given, the disagreement forces can be easily computed as the L-2 norm of the difference between the reference $\hat{\bm{f}}$ and the sensed forces $\bm{f}^s$. Therefore, in our case, the set of observations used to compute the posterior mean and variance functions is $\mathcal{D}_n = \{(\bm{\theta}_{i,j}, ||\hat{\bm{f}}-\bm{f}^s||_j)\}_{j=1}^n$.

The fact that the learning model encodes the nominal plan using a set of states (represented by Gaussian distributions) allows us to carry out a local search of the optimal parameters by identifying the state $i$ in which the robot is when a desired adaptation is triggered. To do so, we exploit the definition of the forward variable~\eqref{eq:alphahsmm} and choose the local adaptation parameters $\bm{\theta}_i$ as those corresponding to the state
\begin{equation}
	i = \argmax_j \alpha_{t,j} .
	\label{eq:argmax_i}
\end{equation}
This reduces the dimensionality of the parameter space in contrast to a high-dimensional vector $\bm{\theta}$ composed of all the mean vectors of motor commands and durations of the learning model. Moreover, the domain $\mathcal{X}$ can be automatically extracted from human demonstrations, where both $\bm{\Sigma}_i^{\mathcal{O}}$ and $\sigma_i^{\mathcal{D}}$ specify local bounds in which BayesOpt is allowed to look for the optimal $\bm{\theta}_{i,N}$. For example, lower and upper bounds for the sensorimotor component of $\bm{\theta}_i$ may be defined as ${(\bm{\mu}_i^{\mathcal{O}}-2\bm{\sigma}_i^{\mathcal{O}}, \bm{\mu}_i^{\mathcal{O}}+2\bm{\sigma}_i^{\mathcal{O}})}$, where $\bm{\sigma}_i^{\mathcal{O}}$ is the variance vector in $\bm{\Sigma}_i^{\mathcal{O}}$. Interestingly, this automatic domain extraction shares similarities with the modified BayesOpt approach proposed in~\cite{Berkenkamp16:safeBO} for safe controllers optimization.

\begin{figure*}[!tbp]
	\includegraphics[width=.245\textwidth]{DemonsGMM_mod}
	\includegraphics[width=.245\textwidth]{NominalAdaptedPlan}
	\includegraphics[width=.245\textwidth]{DemonsGMM_L_mod}
	\includegraphics[width=.245\textwidth]{NominalAdaptedPlan_L}
	\caption{2D pick-and-place task (C- and L-shape trajectories). \emph{Left:} Demonstration data of 2D trajectories are displayed as gray points, while the nominal task plan encoded by an HSMM is shown as light-color ellipses. The adapted model is displayed using the same color format but with darker tones. The direction of the applied force is represented by the black arrows. \emph{Right:} the trajectory distributions obtained from nominal plan and force-guided BayesOpt adaptation are respectively displayed in red and blue. The solid line depicts the mean of the trajectory while the covariance is displayed as ellipses centered in the mean.}
	\label{Fig:NomAdapPlan}
\end{figure*}

\subsection{Online adapted trajectory generation}
When human intervention is detected, meaning that trajectory adaptation should take place, it is necessary to update the reference trajectory over the course of the task. To do so, every time our force-based local BayesOpt finds a set of optimal parameters $\bm{\theta}_{i,N}$, a new reference distribution of trajectories is generated by computing a new sequence of states $\bm{s}_{t:T_w}$ via~\eqref{eq:alphahsmm} for a time window of length $T_w$. This is later used to generate the new trajectory distribution through~\eqref{eq:trajHMM} and~\eqref{eq:trajHMM_cov}. Note that the specification of a time window assumes that the interactive trajectory adaptation occurs for relatively short time periods, meaning that the robot is expected to resume the execution of the nominal plan once the human operator does not trigger any adaptation. Moreover, the time window favors the computational cost of the whole adaptation process.

\begin{figure}[!tbp]
	\includegraphics[width=.2\textwidth]{TransDur_NominalPlan}
	\hspace{0.3cm}
	\includegraphics[width=.24\textwidth]{DurProbs_5state}
	\caption{2D pick-and-place task (C-shape trajectories). \emph{Left:} HSMM transition graph and state duration probabilities as lognormal distributions, with the six states depicted with the same colors as the Gaussian distributions in Fig. \ref{Fig:NomAdapPlan}. \emph{Right:} Original and adapted duration probability distributions of the fifth HSMM state in encoding of C-shape trajectories shown in Fig. \ref{Fig:NomAdapPlan}.}
	\label{Fig:TransDurNominalPlan}
\end{figure}

\begin{figure}[!tbp]
	\centering
	\includegraphics[width=.4\textwidth]{InteractionForce}
	\caption{Force profile applied to the robot end-effector in order to introduce force-guided local trajectory adaptations.}
	\label{Fig:InteractionForce}
\end{figure}

\section{EXPERIMENTS}
\label{sec:Results}

\subsection{Description}
The pick-and-place task is a standard setup where a robotic manipulator is required to reach for an object, grasp it, and consequently move it to a target location into its workspace, where the object is released. This scenario is highly relevant in SMEs as its main characteristics require range of common robot skills that are found in other types of tasks. For the experiments, we collected six synthetic demonstrations where both 2D Cartesian trajectories and sensed force profiles were generated while the robot followed C- and L-shape trajectories, as shown in Fig.~\ref{Fig:NomAdapPlan}. Note that in this particular task the robot does not require to apply a specific force while moving the object, which means that the recorded sensed forces are zero-mean.
For both datasets, we trained a couple of 6-states HSMMs to learn the nominal plan of the task, which is mainly encapsulated by the set of normal distributions $\{ \mathcal{N}_{s,i}, \mathcal{N}^{\mathcal{D}}_{i}\}_{i=1}^K$ encoding local sensorimotor patterns and duration information.

During reproduction of the task, the robot end-effector movement is mainly driven by the reference trajectory distribution computed by~\eqref{eq:trajHMM} and~\eqref{eq:trajHMM_cov}. The detection of human intervention is carried out by monitoring the difference between the reference force distribution $\mathcal{N}(\hat{\bm{f}}_t, \hat{\bm{\Sigma}}^{\bm{f}}_t)$ and the sensed forces $\bm{f}^s_t$. A time window of five time steps was used to compute these disagreement forces. If human intervention is detected, a local adaptation process is triggered by running the forge-guided local search described in Section~\ref{subsec:RobotAdapt}. This local search is implemented by proposing new local parameters $\bm{\theta}_{i,t+1}$ at each time step $t$ according to the acquisition function~\eqref{eq:EI}. Once the optimal set of local parameters $\bm{\theta}_i^*$ has been found, the reference trajectory distribution is recomputed using the updated observation and duration probabilities of state $i$ with new means $\bm{\mu}_i^{\mathcal{O}^*}$ and $\mu_i^{\mathcal{D}^*}$. Note that this simple example for testing our trajectory adaptation approach resembles the robot negotiation for human intervention in the context of a mail delivery task~\cite{Yin18:HumanInRobotMotionLearning}.

\subsection{Learning and reproduction of nominal plan}
Figure~\ref{Fig:NomAdapPlan} shows the encoding of the nominal plan of the task extracted from the synthetic demonstrations for both datasets (i.e, C- and L-shape trajectories). The ellipses represent the HSMM observation probability distributions $\mathcal{N}_{s,i}$, where light tones correspond to the original nominal task plan. The reference trajectory distribution driving the end-effector motion during the execution of the nominal plan (i.e. no adaptation is triggered) is depicted by the red solid line (mean) and shared area (covariance). The transition graph and state duration probabilities $\mathcal{N}^{\mathcal{D}}_{i}$ for the HSMM encoding the C-shape trajectories are displayed in Fig.~\ref{Fig:TransDurNominalPlan}. As expected, the reference trajectory distribution retrieved by using the HSMM encoding properly encodes the sensorimotor patterns observed in the demonstrations.

\subsection{BayesOpt-based adaptation}
It is more interesting though to notice the effects of using force-guided BayesOpt. To do so, a simulated external force was applied to the robot end-effector during the execution of the nominal plan. The direction of the applied force is represented by the black arrows in Fig.~\ref{Fig:NomAdapPlan} while its profile is shown in Fig.~\ref{Fig:InteractionForce}. The first and third plot in Fig.~\ref{Fig:NomAdapPlan} show the adapted model with darker-tone ellipses, where some of the them fully overlap the ellipses depicting the original plan as the force-guided adaptation was triggered only for a specific time horizon (approximately from $t=55$ to $t=90$). As it can be observed, the purple and green ellipses -- in the C-shape trajectories model-- were vertically translated as an agreement with the external force. A similar effect was produced by the force-guided BayesOpt in the L-shape trajectories model where the orange Gaussian was horizontally moved. Additionally, our adaptation process also modified the duration probability distributions when necessary. Figure~\ref{Fig:TransDurNominalPlan}-\emph{right} shows how the duration probability distribution of the fifth HSMM state was adapted so that the robot stays longer in it.

\begin{figure}[!tbp]
	\centering
	\includegraphics[width=.45\textwidth]{GP_BO_C_state4_mod}
	\includegraphics[width=.45\textwidth]{GP_BO_C_state5_mod}
	\caption{2D pick-and-place task (C-shape trajectories). The plots show the mean and confidence intervals estimated with a Gaussian process (GP) as surrogate model of BayesOpt after convergence (shaded area delimits $95\%$ credible intervals). The GPs correspond to the force-guided BayesOpt adaptation of HSMM states four (top) and five (bottom), depicted as purple and green ellipses in Fig. \ref{Fig:NomAdapPlan}. }
	\label{Fig:GPposterior}
\end{figure}

Note that the local adaptation of the HSMM states directly modifies the reference trajectory distribution that the robot tracks. The second and fourth plots in Fig.~\ref{Fig:NomAdapPlan} display the resulting reference distributions as blue solid lines (means) and shared areas (covariances) for both C- and L-shape models. As a result of the local adaptation, the retrieved trajectories match the nominal plan when no external forces were applied, and they deform according to the local changes imposed by the force-guided BayesOpt. More specifically, the new means $\bm{\mu}_i^{\mathcal{O}^*}$ and $\mu_i^{\mathcal{D}^*}$ computed by BayesOpt directly affect the adapted trajectory distribution computed via~\eqref{eq:trajHMM}. This local adaptation allows the user to introduce small changes in the trajectory without affecting relevant patterns of the task, such as the start and end of the reference distributions in Fig.~\ref{Fig:NomAdapPlan}, which are crucial for reaching and releasing the manipulated object.

Figure~\ref{Fig:GPposterior} shows the Gaussian process posterior for the force-guided adaptation of the fourth (purple Gaussian) and fifth (green Gaussian) states of the C-shape model. Note that BayesOpt efficiently converged after only 6 and 5 iterations, respectively. The local adaptations leaded to disagreement forces close to zero, meaning that the new trajectory distribution was in accordance with the desired modifications imposed by the external force. We also ran our force-guided BayesOpt for the case in which the full set of means $\{\bm{\mu}_{s,i}, \mu^{\mathcal{D}}_{i}\}_{i=1}^K$ is considered, in contrast to confining the exploration through~\eqref{eq:argmax_i}. In this case, BayesOpt did not converge. We attribute this to the fact that the vector $\bm{\theta} \in \mathbb{R}^{18}$, leading to the curse of dimensionality, a well-known limitation of BayesOpt. Although dimensionality reduction techniques may be applied, we instead exploited the task model to locally restrict the optimization process so that the search space dimensionality was low.

\section{DISCUSSION}
Note that the proposed trajectory adaptation has several aspects in common with a classical impedance control approach. Although we here assume that we can exploit the impedance control capabilities of the robot to physically interact with it, our approach differs from this classical control method in that the human provides force cues to modify the reference trajectory that the robot tracks, as in~\cite{Losey18:interactiveTrajDeformation}. This approach is advantageous as the human operator can experience the rendered compliance of the impedance controller, and deform a segment of the desired trajectory. Because of our local adaptation strategy, the deformed trajectory returns to the nominal plan when no interaction is detected, and so the robot can also contribute toward completing the shared task. Another advantage of the force-guided BayesOpt is the possibility to include constraints into the optimization problem which may aim at minimizing jerky motions, enforcing safe physical interaction, or restricting high deviations from the nominal plan.

In our force-guided robot adaptation we omitted the covariance parameters for simplicity. Extending the vector or local model parameters~\ref{eq:thetaParams} to include the covariance terms is not trivial as $\bm{\Sigma}_i$ imposes positive-definiteness constraints for BayesOpt. In this line, two potential solutions are: \emph{(i)} to consider the Cholesky decomposition of $\bm{\Sigma}_i$ as proposed in~\cite{Abu-Dakka18:ForceImpedanceLearning} to learn variable stiffness matrices, or \emph{(ii)} to exploit the geometry of symmetric positive definite matrices as proposed in~\cite{Rozo17:Manipulability} to learn robot manipulability ellipsoids.

\section{CONCLUSIONS AND FUTURE WORK}
\label{sec:Conclusions}
This paper introduced a data-efficient optimization framework aimed at adapting trajectory distributions as a function of interaction forces generated by a human operator. The proposed approach assumed that the force data produced by the human convey information regarding the human intended adaptation. These data were used to compute disagreement forces that the robot sought to minimize by locally adapting its task model, previously learned from demonstrations. Our approach leveraged the benefits of BayesOpt and the probabilistic encoding provided by HSMM to efficiently adapt spatio-temporal patterns in a handful of iterations. Smooth adapted trajectories were retrieved by taking advantage of the static and dynamic features encapsulated in the learned model in the form of a mixture of Gaussian distributions.
The reported experiments did not tackle the adaptation of the model covariance matrices as this entails a high-dimensional problem for BayesOpt. We plan to study how latent space representations of the HSMM parameters may be exploited to alleviate this issue. Moreover, we will evaluate the proposed framework in different real scenarios, considering tasks where time-varying force profiles are relevant for the successful performance.

\addtolength{\textheight}{-12cm}

\bibliographystyle{IEEEtran}

Robotics (cs.RO); Machine Learning (cs.LG); Optimization and Control (math.OC)