The RoboPol Program: Optical Polarimetric Monitoring of Blazars

Abstract: After three years of polarimetric monitoring of blazars, the RoboPol project has uncovered several key characteristics of polarimetric rotations in the optical for these most variable sources. The most important of these is that polarization properties of the synchrotron emission in the optical appear to be directly linked with gamma-ray activity. In this paper, we discuss the evidence for this connection, as well as the broader features of polarimetric behavior in blazars that are key in making progress with theoretical modeling of blazar emission.

\section{Introduction} \label{sec:intr}
The first measurements of optical polarization of quasars followed soon after the identification of their
optical counterparts. These measurements have revealed that many of these sources are linearly
polarized to some extent, and variable in both the degree of polarization and the electric
vector position angle (EVPA) \citep{Kinman1966}. Fifteen years later, optical polarization was
measured for $\sim$60 compact extragalactic sources, and the first statistical conclusions could be
derived, together with some preliminary tests of consistency with models \citep{Angel1980}. It was found that,
typically, the behavior of polarization degrees and angles are erratic \citep{Moore1982}. This can be
explained by the presence of a persistent component and one or multiple variable components
\citep{Brindle1985,Blinov2009,Uemura2010,Villforth2010}. In fact, such a model was first used for the
interpretation of irregular radio polarization behavior as early as 1967 \citep{Goldstein1967}.
However, it has been known since the end of the 1970s that some sources exhibit long monotonic
rotations of the EVPA in the radio band emission during flares \citep{Ledden1979,Jones1985}. The
first optical polarization rotation, simultaneous with a similar rotation in the radio band, was
reported by Kikuchi et al. (1988) \citep{Kikuchi1988}. During the first years of operation of the
Fermi $\gamma$-ray observatory, optical EVPA rotations have attracted a lot of interest, because it
was noticed that some of the strong $\gamma$-ray flares were accompanied by these events
\citep{Marscher2008,Abdo2010,Marscher2010}. If EVPA rotations were physically related to the
high-energy activity, they could help localize and clarify the nature of the latter one, which was
actively discussed at that time.

Therefore, a number of optical polarization monitoring programs have since focused on observations
of blazars that are bright and active in $\gamma$-rays. Such major monitoring programs have been carried
out by the Boston University group \citep{Jorstad2016}, at St. Petersburg University
\citep{Hagen2013}, at the Hiroshima University
\textcolor{black}{(Kanata~program)}~\citep{Itoh2018}, at the Steward Observatory \cite{Smith2009},
at the Calar Alto observatory \citep{Agudo2012}, and at the Tuorla observatory \citep{Takalo2008}.

From a theoretical standpoint, a~wide variety of mechanisms have been proposed to explain the
origin of EVPA rotations. They include the following scenarios:
\begin{enumerate}[leftmargin=10mm,labelsep=5.8mm]
\item A two-component model, where changes in the polarization are due only to changes in the
relative flux of the two components \citep{Bjornsson1982}.
\item Propagation of an emission component along the jet with a non-axisymmetric magnetic field
\citep{Konigl1985}.
\item \textcolor{black}{Polarized synchrotron flares rotating in the accretion disc}
\citep{Sillanpaa1993}.
\item Motion of an emission component along a helical streamline in the helical magnetic field
\citep{Marscher2008,Marscher2010}.
\item Jet bending \citep{Abdo2010}.
\item Changes of polarization parameters and total flux due to the Doppler factor variability
\citep{Larionov2013}.
\textls[-15]{\item Random variation of the polarization vector in the emission produced by a turbulent~zone~\citep{Jones1985,Marscher2014,Kiehlmann2016}.}
\item \textcolor{black}{Propagation of a relativistic shock in the jet that causes compression of the
magnetic field and alters its direction and the degree of ordering} \citep{Zhang2014}.
\item Precession of the jet \citep{Lyutikov2017}.
\item Kink instability \citep{Nalewajko2017}.
\item Magnetic reconnections \citep{Zhang2018}.
\item Relativistic EVPA rotation in the observer frame combined with the conical jet geometry and
particle cooling \citep{Peirson2018}.
\end{enumerate}

Inspired by the perplexing and unclear nature of EVPA rotations, an intensive polarization
monitoring program, RoboPol, was designed and executed.

\section{The RoboPol Program}

In spite of the immense significance of EVPA rotations for an understanding of the magnetic field
structure in the jets and their importance for constraining theoretical models, only a handful of
these events were reported in the 25 years since their discovery. \textcolor{black}{Additionally,
most of these observing programs} were monitoring small numbers of the most violently variable blazars;
these were biased samples, and not appropriate for statistical studies. Therefore, such questions as
"Do all blazars rotate the EVPA? How often these rotations  occur on average? Does this depend on
the blazar class?" would be impossible to address accurately using data of monitoring programs
already existing at the time when RoboPol~was~designed.

In order to overcome these shortcomings, an international collaboration between the University of
Crete and Foundation of Research and Technology Hellas (FORTH, Greece), the~Max-Planck Institute for
Radioastronomy (Germany), the~California Institute of Technology (USA), the~Inter-University Centre
for Astronomy and Astrophysics (India), and the Nicolaus Copernicus University (Poland) developed
a new blazar monitoring program called RoboPol. This program specifically aimed at maximizing the
number of detected EVPA rotations and characterizing the optical polarization behavior of blazars.

\subsection{Instrumentation}

The key element of the the RoboPol program is a novel-design optical polarimeter that was developed
specifically for the project. This polarimeter has a fixed set of half wave plates and Wollaston
prisms (see Figure \ref{fig:optics}) that split each incident ray into four rays with the polarization
plane rotated by 45$^{\circ}$\, with respect to each other. Therefore, one can measure the Stokes parameters
$q = Q/I$ and $u = U/I$ of the linearly polarized light in one exposure. Given the large field of
view (13'~$\times$~13'), the total flux density $I$ can also be measured at the same time if a
comparison star with a known magnitude is present in the field. Since the polarimeter has no moving
parts except the filter, it avoids systematic uncertainties related to an imperfect alignment of
rotating optical elements, and simplifies the~instrument~calibration.

The polarimeter was hosted on a 1.3 m telescope at the Skinakas Observatory, which is a joint
facility of the FORTH and the University of Crete. It is located at an altitude of 1750~m in Crete
(35.2120$^{\circ}$\, N, 24.8982$^{\circ}$\, E).

\begin{figure}[H]
 \centering
 \includegraphics[width=0.45\textwidth]{img/robopol_drawing.eps}
 \caption{Schematic of the polarizing optics of RoboPol. The~array of two half-wave plates
(HWP) and two Wollaston prisms (WP) is shown. The~fast optical axes of the elements and the
polarization planes of the rays are shown with arrows. \textcolor{black}{The HWP with the fast axis
oriented at 67.5$^{\circ}$\, rotates the polarization plane by $2\cdot67.5 = 135$$^{\circ}$\, (see, e.g.,
\cite{clarke2009}).}}
 \label{fig:optics}
\end{figure}

The site offers seeing conditions which are among the best in Europe, comparable to several of the largest
facilities in the world (median seeing of 0.7 arcsec \citep{Boumis2001}). The~Skinakas Observatory
granted the RoboPol collaboration four nights per week, on average, over the entire Skinakas
observing season (May~to~November~each~year), for three years (2013--2015).

In addition to the advanced instrument, we developed an automated telescope control system, as well as a
processing pipeline. The~former one was able to point automatically and estimate the brightness of
targets using the pointing exposures. It could also estimate polarization parameters on the fly,
reaching a particular signal-to-noise ratio in EVPA if required.

A more detailed description of the polarimeter can be found in Ramaprakash et al. (2019)
\cite{Ram2019}, while the pipeline and the control system are presented in detail in King et
al. (2014) \citep{King2014}.

\subsection{Unbiased Observing Sample}

\textls[-5]{Another unique feature of the program is the observing sample, constructed in a way which allows for
systematic statistical studies. We built the sample relying on unbiased and statistically rigorous
criteria. Firstly, we constructed a flux-limited sample of $\gamma$--loud blazars from the second
Fermi-LAT (2FGL) source catalog \citep{Nolan2012}, selecting sources classified as BL Lac (bzb),
FSRQ (bzq), or an active galaxy of uncertain type (agu) that have the integrated photon flux
$F(100 \mbox{ MeV} \le E \le 100 \mbox{ GeV}) \ge 2\times10^{-8}~{\rm ph\,~cm^{-2}~s^{-1}}$. Then, we
excluded sources with $|b|\le10^{\circ}$ in order to avoid any dependence of the sensitivity on the photon
index and minimize polarization induced by the Galactic dust. Finally, applying visibility
constraints and introducing the optical magnitude cut $R\le17.5$, we obtained the unbiased $\gamma$--loud
monitoring sample of 62 sources. The~control sample of $\gamma$--quiet sources was selected from
Candidate Gamma-Ray Blazar Survey (CGRaBS) sources \citep{Healey2008} that had not been detected by Fermi
at that time (not~included~in~2FGL). We introduced constraints on the 15 GHz mean flux density $\ge
0.06$ Jy and intrinsic modulation index \textcolor{black}{as defined by Richards et al.
\citep{Richards2011}} $m \ge 0.05$. Then, we applied similar visibility cuts and the $R\le17.5$
condition as for the $\gamma$--loud sources. Finally, we randomly selected 15~sources that passed all the
cuts. Two out of 15 sources in our $\gamma$--quiet sample appeared in the four-year Fermi-LAT (3FGL)
catalog as $\gamma$-ray sources. Therefore, we added two more blazars that satisfied the previous criteria,
but were not reported as $\gamma$-ray--loud in either 2FGL or 3FGL. Additionally, to the unbiased samples
of $\gamma$--loud and --quiet sources, we observed 24 additional "high~interest" sources, which were
not taken into account in population studies where they could bias results \textcolor{black}{due to
their activity being higher than average}. More details on the sample selection are discussed in~Pavlidou~et~al.~(2014)~\citep{Pavlidou2014}.}

\section{Results}

After running the program for three observing seasons, we identified 40 EVPA rotations
in 24~individual blazars \textcolor{black}{(see Section~\ref{sec:caveats} for discussion on the EVPA
rotation definition). As an example, in Figure ~\ref{fig:rot_examp} we show one of the rotation events
observed in the $\gamma$-ray--loud sample blazar J1751+0939.} We were able to study statistical
properties of the detected events, their characteristics, and connection to the high-energy flares
in blazars. Here, we briefly discuss our major results related to the EVPA rotations, as well as the
behavior of polarization in blazars in general.

\begin{figure}[H]
 \centering
 \includegraphics[width=0.447\textwidth]{img/RBPLJ1751+0939.eps}
 \includegraphics[width=0.43\textwidth]{img/RBPLJ1751+0939_qu.eps}
 \caption{An example of an EVPA rotation (marked by the red circles). Based on data from
\cite{Blinov2016a}.}
 \label{fig:rot_examp}
\end{figure}

\subsection{Polarization Properties of Different Types of Blazars} \label{sec:polbeh}

\textls[-8]{\textcolor{black}{Using data obtained} during the first month of RoboPol operation, we have found that the
average fractional polarizations of the $\gamma$--loud and $\gamma$--quiet samples are
$<p>$ = $6.4^{+0.9}_{-0.8}\times10^{-2}$ and $3.2^{+2.0}_{-1.1}\times10^{-2}$, respectively. The
hypothesis that the polarization fractions of blazars in these samples are drawn from the same
distribution is rejected at the $3\sigma$ level of significance \cite{Pavlidou2014}. Later, this
result was confirmed using averaged monitoring data for a larger sample of sources
\cite{Angelakis2016}. There, the difference of average intrinsic polarization for the two groups
was found to be even larger, $9.2\pm0.8\times10^{-2}$ and $3.1~\pm0.8~\times~10^{-2}$. A~similar
conclusion that $\gamma$-ray luminosity and optical polarization are correlated was made, also based on
Kanata monitoring data \citep{Itoh2016}. These results immediately suggest that the average
``polarizing~efficiency'' in jets of $\gamma$--loud and $\gamma$--quiet blazars is systematically different.
This efficiency depends on the ordering of the magnetic field in the jet and the electron energy
distribution index. However, the~latter is a rather weak dependence, and cannot solely be responsible
for the wide range of optical polarization observed in blazars (see discussion in
\citep{Itoh2016}).}

As per Hovatta et al. \citep{Hovatta2016}, we tested whether the \textcolor{black}{BL Lac objects} that
were detected with the current-generation TeV instruments had different optical polarization
characteristics compared to similar sources that have not yet been detected. It was found that after
correction for depolarization produced by the host galaxies, TeV-detected and non-detected BL Lacs
had the same mean fractional polarization. The~rate of EVPA variations is also statistically
indistinguishable between the two groups. This was considered as an indication that there are no
intrinsic differences in the polarization properties of the TeV-detected and non-detected
high-energy BL Lac objects. Therefore, the~magnetic field ordering and the electrons distribution
within their jets are not expected to differ systematically.

Besides the relation of optical polarization with $\gamma$-ray emission of blazars, it has also been
found that polarization parameters depend on the position of the synchrotron peak maximum. For
instance, low-synchrotron peaked (LSP, $\log(\nu_p) < 14$) sources have higher fractional
polarization and a broader range of values compared to intermediate- (ISP, $14 \le \log(\nu_p)
\le 15$) and high-synchrotron peaked (HSP, $15 < \log(\nu_p)$) sources \cite{Angelakis2016}.
The dependence of the maximum polarization on the synchrotron peak position was also found in
other monitoring programs \citep{Jermak2016,Itoh2016}. Moreover, HSP
 sources tend to demonstrate
\textcolor{black}{significant peaks in EVPA distributions (i.e., a preferred direction of polarization
plane), while LSPs typically have uniform distributions of EVPA.} Hints of this behavior were
noted earlier, but for very small samples of sources so that no
statistically robust conclusion could be made \citep{Villforth2010}.

Considering the aforementioned dependencies, Angelakis et al. \cite{Angelakis2016} proposed a
qualitative model sketched in Figure ~\ref{fig:jet_sketch}. In this scenario, a mildly relativistic
shock in the jet causes efficient particle acceleration (e.g., due to diffusive shock acceleration
or magnetic reconnection) in a small volume directly downstream of the shock. Propagating further
downstream, these accelerated particles progressively lose their energy due to the emission of
inverse Compton and synchrotron radiation. Therefore, the~emission at the synchrotron peak
frequency or higher is produced by the most energetic particles congregated in a small volume
adjacent to the shock, where the magnetic field is compressed and expected to be highly ordered.
Lower-frequency synchrotron emission is received from a region with greater volume downstream the
jet. This region contains a larger range of magnetic field line orientations. Additionally,
the ordered component of the magnetic field can be less dominant in this zone, when compared to the
turbulent component. Thus, at the fixed frequency of the R-band, we~observe highly polarized
emission near the synchrotron peak maximum and above in LSP sources, and significantly less
polarized emission at the low-energy side of the peak in HSP blazars (see~Figure~\ref{fig:jet_sketch}). Itoh et al. \citep{Itoh2016} also explain the lower polarization of HSP
sources by larger emitting volume and superposition of multiple emitting regions with independent
orientations of magnetic field lines. However, their scenario includes a spine-sheath model, where
the spine has a highly ordered magnetic field and produces highly polarized and variable emission,
while the sheath, in contrast, produces steady and low-polarized light. Then, the systematic difference
in the polarization of HSP and LSP sources can be caused by the increasing jet volume fraction occupied
by the spine. Potentially, an observational test can discriminate between these two models. The
first model predicts that most of the sources \textcolor{black}{must demonstrate decreasing
fractional} polarization from higher (Ultraviolet) to lower frequencies (Near~Infrared), while for
the spine-sheath model, no systematic behavior is expected.

\begin{figure}[H]
 \centering
 \includegraphics[width=0.5\textwidth]{img/shock_in_jet_sketch.png}
 \caption{Sketch of the shock-in-jet scenario proposed by Angelakis et al. \cite{Angelakis2016}.}
 \label{fig:jet_sketch}
\end{figure}

\subsection{Connection of EVPA Rotations and $\gamma$-Ray Flares} \label{sec:rot_vs_gamma}

\textls[-5]{One of the main goals of the project was to verify the previously assumed connection of EVPA
rotations with strong $\gamma$-ray flares. We addressed this issue in~\cite{Blinov2015} based on the
first-season data and~in~\citep{Blinov2018} using the entire set of 40 events detected during the
three-year program. Our main test included a direct estimation of the probability that all EVPA rotations
occur at random times with respect to $\gamma$-ray flares in the light curves of corresponding
blazars.}

For this purpose, we identified all flares in $\gamma$-ray light curves that occurred during intervals
covered by RoboPol. Then, we fitted these flares with an analytic function and derived their basic
parameters, such as the peak time, flux at the peak, rise, and decay times. After this, we found the
time lags between EVPA rotations and flares and constructed the cumulative distribution function
(CDF) of their values (see Figure~\ref{fig:CDF_TL}). Dropping random points at the $\gamma$-ray light
curves, we ran a Monte-Carlo simulation that simulated EVPA rotations distributed at random moments
in time. Thus, we~obtained $10^6$ simulated CDFs ($10^4$ of them are shown in
Figure~\ref{fig:CDF_TL}), and found that only 70 were entirely located closer to zero than the observed
one. Therefore, we concluded that the probability of all time lags in the sample being accidentally
so close to zero, as observed, is $\sim 7 \times 10^{-5}$. This implies that at least some of the
time lags between EVPA rotations and $\gamma$-ray flare we observed are physically linked.

\begin{figure}[H]
 \centering
 \includegraphics[width=0.35\textwidth]{img/CDF_TL.png}
 \caption{CDFs
 of the time lags $\tau_{obs}$ between the EVPA
 rotations and peak moments of the
closest $\gamma$-ray flares for the main sample rotators. The black line represents observed time lags, and the thin grey
lines represent $10^4$ simulated values for the whole sample of rotations. Based on data from
\cite{Blinov2018}.}
 \label{fig:CDF_TL}
\end{figure}

\textcolor{black}{This interpretation received further support} when we compared amplitudes and
timescales of EVPA rotations and $\gamma$-ray flares (see Figure~\ref{fig:gamma_vs_rot}).
\textcolor{black}{In spite of large uncertainties in determination of the rotations amplitudes
$\Delta \theta_{max}$ (because of the difficulty in identifying the onset and the end of a
particular event), it} has been found that the peak luminosities of flares $L_p$ closest to
rotations are anticorrelated with $\Delta \theta_{max}$. Such a relation could be caused by a
correlation of jet parameters that influence luminosity, i.e., the Lorentz and Doppler factors and
the jet viewing angle, with the rotations amplitudes. The~correlations between these parameters were
indeed found to be significant as well. Moreover, we demonstrated that the characteristic timescales
of rotations and $\gamma$-ray flares were marginally correlated \cite{Blinov2018}. Thus, one of the most
significant outcomes of the RoboPol program is that it has proven, for the first time in a
statistically robust fashion, that at least some EVPA rotations must be physically related to $\gamma$-ray
flares.

\begin{figure}[H]
 \centering
 \includegraphics[width=0.36\textwidth]{img/rot_ampl_vs_gamma_lumin.eps}
 \includegraphics[width=0.36\textwidth]{img/durat_vs_Tr+Td.eps}
 \caption{{\bf Left panel}: $\gamma$-ray flare luminosity, $L_p$, vs. EVPA rotation amplitude, $\Delta
\theta_{max}$. Empty circles correspond to blazars with uncertain z. {\bf Right panel}: characteristic
timescale of flares, $T_{flare}$, vs. duration of rotations, $T_{rotation}$, in the observer frame
Lines show linear fits to the data. Based on data from \cite{Blinov2018}.}
 \label{fig:gamma_vs_rot}
\end{figure}

\subsection{Properties of EVPA Rotations}\label{s3.3}

RoboPol provided enough data for a first attempt to characterize the statistical properties of
EVPA rotation events, as well as the behavior of the total flux and fractional polarization during
these events. We detected EVPA rotations with durations, $\Delta T$ from 19 to 518 days in the jet
reference frame and amplitudes, $\Delta \theta$ from 74$^{\circ}$\, to 427$^{\circ}$\,
\textcolor{black}{(see Figure \ref{fig:gamma_vs_rot})}. The~histogram of rotation amplitudes was found
to be consistent with both normal and uniform distributions. However, it has an apparent peak near
the mean 186$^{\circ}$\, value, which is consistent with ``the magic number'' of
180$^{\circ}$\, appearing in several models explaining these events. \textcolor{black}{When EVPA rotations are
approximated by straight lines, their} rotation rates $\Delta \theta/\Delta T$ in the jet reference
frame span from 0.2$^{\circ}/{\rm d}$ to 16$^{\circ}/{\rm d}$. They are strongly non-uniform and,
presumably, can be approximated by a power law distribution. This, however, could just be a natural outcome of $\Delta \theta$ and $\Delta T$ being uniformly distributed. The~values
of $\Delta \theta$ and $\Delta T$ are not correlated. We have presented statistical evidence that
$\Delta \theta$ and $\Delta T$ of the rotations are likely to be limited—that is, arbitrary lasting and
high amplitude events are not present in nature. The~null hypotheses that rotation amplitudes and
durations can exceed 460$^{\circ}$\, and 500 days are rejected at the significance level of 0.01. These limits
are presumably caused by restraints of the physical parameters presented in the jets, such as
topology of the magnetic field, emission region size, and finite bulk speed of the moving emission
features, or other characteristics related to the EVPA rotations.

We have demonstrated that most of the rotations are accompanied by a decrease in the fractional
polarization. At the same time, its modulation indices, on average, remain constant. Moreover, we~have found a marginal ($2.7\sigma$ significance level) anticorrelation of the depolarization
factor—that is, average polarization during rotation normalized by its value in a non-rotating state,
with the rotation rate in the jet reference frame. This dependence, if confirmed with a larger
sample of events, could place significant constraints on models explaining EVPA rotations.
Finally, it has been demonstrated that during most of the EVPA rotations, there is no clear
systematic increase in the total flux density. These and other properties have been reported in
\cite{Blinov2015,Blinov2016a,Blinov2016b,Blinov2018}.

\subsection{Rotations in Different Types of Blazars} \label{sec:rot_typ}

Thanks to the unbiased nature of the RoboPol monitoring sample, we were able to
investigate the occurrence of EVPA rotations in blazars of different classes \citep{Blinov2016b}. We
started by demonstrating that the polarization plane rotations frequency (at least for events
with rates $<20^{\circ}/d$) is not the same for different sources. For instance, the~null
hypothesis \textcolor{black}{that EVPA rotations occur equally frequently in all sources} was found
to be very unlikely ($p=10^{-7}$). At the same time, this difference in the EVPA rotations
occurrence could not be clearly related to the $\gamma$--quiet -- $\gamma$--loud dichotomy of blazars. We~could
not reject the null hypothesis that the $\gamma$--quiet sample blazars show rotations with the
same frequency as sources in the $\gamma$--loud sample. Therefore, according to our data, it is not
impossible that the frequency of EVPA rotations is the same for the blazars in the main and the
control samples. Nevertheless, we found that the frequency of rotations very significantly varies,
even between sources of the main $\gamma$--loud sample itself. For example, the~upper limit on the
average frequency of rotation in the $\gamma$--loud sample sources, where we detected zero rotations, is
more than two orders of magnitude smaller than frequency of rotations in three sources which exhibited
four rotations during the three years of monitoring. Furthermore, we verified that no external
reason can entirely explain the distinction in rotation frequencies. A~difference in accuracy of
EVPA measurements that is caused by various fractional polarization levels cannot be solely
responsible for this effect. Similarly, we have shown that there is no significant distinction in
the time-delay factors $\delta/(1+z)$ between the samples with detected rotations and without.
Therefore, we concluded that there exists a sub-class of objects that exhibit EVPA rotations
significantly more frequently than others. This difference is not simply related to whether or not a
blazar is detected by Fermi-LAT. It also cannot be explained by the non-uniformity of observations
or by observational biases due to differences in the average fractional polarization
\citep{Blinov2016b}.

We also looked for other intrinsic reasons that may cause the difference in rotation rates.
{We~compared numbers of EVPA rotations and numbers of sources where these events
occurred (rotators) for the three classes of blazars: low-synchrotron peaked (LSP: $log(\nu_p)<14$),
intermediate- (ISP: $14\le log(\nu_p) < 15$), and
high-synchrotron peaked (HSP: $15 \le log(\nu_p)$).} It was found that rotations tend to occur
in sources with a lower synchrotron peak (see Figure~\ref{fig:rot_hist}). For instance, the~probability
that sources with rotations
were randomly drawn from the parent sample is 1.4\% if both $\gamma$--quiet and $\gamma$--loud samples
are considered as the parent sample. In the case when we consider only the $\gamma$--loud sample, this
probability is reduced to 0.5\%. This tendency of EVPA rotations occurring in LSP blazars is in
agreement with findings of Hovatta et al. \citep{Hovatta2014}, who has shown that LSP blazars are
more variable than HSP in the total optical flux density \textcolor{black}{(Stokes parameter I).}
Their work explains this finding with a scenario similar to the one sketched in
Figure ~\ref{fig:jet_sketch} and discussed before—that is, the LSP sources are observed near and above the
electron energy peak, which causes stronger variations of the emission compared to HSP sources,
where the lower energy electrons cool down slowly and produce mild variability. This scenario is
expected to work for the polarized flux and EVPA variability as discussed in Section~\ref{sec:polbeh}.

\begin{figure}[H]
 \centering
 \includegraphics[width=0.43\textwidth]{img/sync_peak.eps}
 \caption{Occurrence of rotations and rotators (sources exhibiting rotations) among blazars of
different synchrotron peak position class. \textcolor{black}{Numbers correspond to the number of
blazars (the number of events in the case of rotations) in each class. Based on data
from \cite{Blinov2016b}.}}
 \label{fig:rot_hist}
\end{figure}

We can briefly summarize the major results of the RoboPol project as follows: only about a quarter
of blazars ($\sim$28\% of sources in both $\gamma$--loud and $\gamma$--quiet samples) exhibit EVPA rotations
with rates $<$$20^{\circ}/{\rm d}$ in the optical band. The~average frequency of appearance of these
events is 1/232 ${\rm d}^{-1}$ (in the observer frame). The~remaining part of sources $\sim$72\% did
not show any rotations registered in our data. Even if they do produce EVPA rotations, this should
happen with average frequency less than $\sim$1/3230 ${\rm d}^{-1}$ in that sample.

\subsection{EVPA Rotations as Random Walks}

The EVPA behaviour in blazars most of the time is consistent with random walk variability
\citep{Moore1982}. This leads to the natural suspicion that long periods of monotonic change of the
EVPA in one direction is nothing but a rare accidental outcome of the same random walk process. We
tested the hypothesis that all EVPA rotations detected during the first observing season of RoboPol
could be explained by stochastic variability of EVPA and found it to be unlikely \citep{Blinov2015}.
In Kiehlmann et al. \citep{Kiehlmann2017} a similarly simple random walk model was tested against
EVPA variability in the three years of monitoring data. They found that the model is able to reproduce
many parameters including durations, amplitudes and rates of EVPA rotations. However, it failed in
simulating realistic distributions of the mean and standard deviation of the fractional
polarization. This potentially could be corrected by more realistic models, such as by introduction of
several populations of sources with different intrinsic variability properties. More comprehensive
models that take into consideration both turbulent and ordered magnetic components as well as
realistic electron energy and density distributions are successful in reproducing the evolution of
polarization parameters and spectral energy distribution in high detail~\citep{Marscher2014}.
RoboPol data do not reject the hypothesis that individual EVPA rotations could be produced by a
random walk of the polarization vector. Nevertheless, the~correlations between parameters of EVPA
rotations and $\gamma$-ray flares parameters described in Section~\ref{sec:rot_vs_gamma} would be totally
smeared out if all rotations were produced by random walks. Therefore, we deduced that at least some
rotations are influenced by deterministic processes that define their properties.

It is not impossible that both types (deterministic and stochastic) of EVPA rotation events coexist.
Moreover, they may be present in the emission of a single blazar \citep{Kiehlmann2016}. It could be
that deterministic ones are produced during periods of high activity in the total flux and high
polarization state. While random walks are more likely to detect during low polarization states,
because the uncertainty of EVPA measurements is larger at these periods \citep{Larionov2016}. This
would explain why, on average, we do not see an increase of optical total flux during rotations and
observe depolarization during most of the events.

\subsection{Possible EVPA Rotations in NLSy1 Galaxies}

Radio loud narrow line Seyfert 1 galaxies are an enigmatic type of AGN that host relativistic jets
and have been detected in $\gamma$-rays \citep{Foschini2015}. They, presumably, represent the low
luminosity and the low black hole mass tail of the flat spectrum radio-quasars population
\citep{Paliya2018}. Therefore, it is expected that their polarization properties may be similar to
properties of LSP blazars, and they may exhibit high polarization variability together with EVPA
rotations. Indeed, despite a lack of optical polarization measurements, at least some of NLSy1
have demonstrated high fractional polarization variability down to minute scales
\citep{Itoh2013,Eggen2013,Maune2014}.

Angelakis et al. \citep{Angelakis2018} searched for optical polarization plane rotations in emission
of 10 radio--loud NLSy1 galaxies using a combined monitoring data set obtained by RoboPol and other
monitoring programs. They concluded that all sources with \textcolor{black}{multiple measurements
show significant} variability, both in EVPA and fractional polarization. However, due to either
sparse sampling of EVPA curves or high level of noise, they could not confidently report the
detection of a long \textcolor{black}{($>$$90^{\circ}$)} continuous EVPA rotation in NLSy1, despite
having several candidates for such events. \textcolor{black}{Long-term monitoring campaigns} with
more accurate observations are needed to provide clear evidence of similarity of this type of AGN
to blazars in their optical polarization variability.

\section{Caveats with EVPA Ambiguity, Rotation Definition, and Hidden Rotations} \label{sec:caveats}

The EVPA defines orientation of the preferred polarization plane with respect to the celestial
meridian. For this reason, the EVPA value is a factor of 180$^{\circ}$\, ambiguous, i.e., values EVPA $\pm
n\times180^{\circ}$ are mathematically equivalent. This means that every measurement of EVPA is, in
fact, an infinite countable set of values separated by 180$^{\circ}$\,, as is the difference between two
measurements (see the left panel of Figure \ref{fig:sim_curve}). The~most natural way to solve this
180$^{\circ}$\, ambiguity is to assume that the variability time scale is much longer than the interval
between measurements \textcolor{black}{(see discussion in Section~\ref{s3.3}~of~\citep{Blinov2015})}
and minimize the difference of EVPA between each of the two consecutive measurements. Nevertheless,
even the procedure of this minimization can be performed in several different ways. For instance,
many authors minimize the following value $\Delta \theta_n = |\theta_n - \theta_{n-1}| - SL
\sqrt{\sigma(\theta_n)^2 + \sigma(\theta_{n-1})^2}$, where $\theta_n$ and $\sigma(\theta_n)$ are
$n$-th EVPA measurement and its uncertainty, $SL$ is the significance level typically set to unity.
If $\Delta \theta_n > 90^{\circ}$, the~angle $\theta_n$ is shifted by $\pm n \times 180^{\circ}$,
where the integer $\pm n$ is chosen in such a way that it minimizes $\Delta \theta_n$. If $\Delta
\theta_n \le 90^{\circ}$, then $\theta_n$ is left unchanged.

\begin{figure}[H]
 \centering
 \includegraphics[width=0.64\textwidth]{img/evpa_amb.eps}
 \includegraphics[width=0.28\textwidth]{img/evpa_opt.eps}
 \caption{Left panel: an example of a family of EVPA curves \textcolor{black}{demonstrating the
180$^{\circ}$\, ambiguity}. All~paths connecting consecutive points in the graph are mathematically
equivalent. Only the three most probable escapes for each step are shown. Right panel: an example of a
spurious jump in the EVPA curve introduced by the 180$^{\circ}$\, ambiguity solution method.}
 \label{fig:sim_curve}
\end{figure}

However, Kiehlmann et al. \citep{Kiehlmann2016} assert that this method itself may lead to
inconsistent results, since it depends on the choice of the initial interval. They suggest to exclude
the term with uncertainties and use $\Delta \theta_n = |\theta_n - \theta_{n-1}|$ for minimization.
Although, in general, these arguments are correct, we caution that this method of the 180$^{\circ}$\, ambiguity
solution can lead to spurious $\sim$$180^{\circ}$ jumps in the EVPA curve caused by measurements with
high uncertainties. An example of such a case is shown in the right panel of
Figure~\ref{fig:sim_curve}, where the black and blue EVPA curves are constructed with and without
EVPA uncertainties taken into account.

Besides the high sensitivity of EVPA curves to the particular procedure of the 180$^{\circ}$\, ambiguity solution
and sampling, there is \textcolor{black}{ambiguity} in the EVPA rotation definition. Potentially, any
significant change of EVPA between two measurements can be considered as an EVPA rotation. Usually,
long amplitude, and smooth and monotonic changes of EVPA are of interest, and corresponding requirements
are introduced into the definition of an EVPA rotation. \textcolor{black}{For instance, our
definition of an EVPA rotation stated that it must be comprised of at least four measurements and
have a total amplitude $>$$90^{\circ}$, while the start and end points of a rotation event are defined by
a change of the EVPA curve slope by a~factor~of~$\ge$5 or a change of its sign. In the cases with
large gaps $\Delta t > 30$~d between consecutive measurements, we considered rotations being
terminated at these gaps. A~particular} choice of the definition may affect parameters of derived
EVPA rotations. A~different approach for the identification of EVPA rotations was proposed by
Larionov et al. \citep{Larionov2016}, who suggest using the Discrete Correlation Function (DCF)
between the Q and U Stokes parameters. This method, however, requires very
well-sampled light curves that may be unavailable for monitoring programs with large samples and
single observing facilities. \textcolor{black}{For instance, the~EVPA rotations detected by RoboPol
are traced by~4~to~22~measurements, while the DCF method requires an order of magnitude with more dense
sampling.}

Studies of Angelakis et al. \citep{Angelakis2016} and Hovatta et al. \citep{Hovatta2016} demonstrate
that HSP and ISP sources tend to have preferred directions of EVPA, which is presumably caused by an
emission of a steady polarized source superimposed on the erratic component
\citep{Hagen2002,Uemura2010,Villforth2010}. This component leads to an apparent shift of the
measurements distribution centroid $Q_c$,$U_c$ from the origin of coordinates on the Q-U Stokes
parameters plane. In such cases, rotations of the polarization plane may manifest themselves on the
Q-U plane as consecutive loops around ($Q_c$,$U_c$) instead of (0,0). These events show a
low-amplitude variability in EVPA curves and may be missed, since many studies identify rotations
directly in the EVPA curves. Therefore, a~significant fraction of EVPA rotations can be ``hidden'' and
require analysis of the variability in the Q-U plane \citep{Ikejiri2011,Larionov2016b}. A~handy tool
for such analysis was proposed by Uemura et al. \citep{Uemura2016}, who also demonstrated its
efficiency using a combined set of Kanata and RoboPol~observations~\citep{Uemura2017}.

\section{Future Prospects}

Despite the recent progress and advances provided by optical polarimetry monitoring programs, there
are a number of questions related to polarization of blazars that remain unclear. We still do not
know what process or processes produce polarization plane rotations in blazar emission and how they
are related to $\gamma$-ray flares. The~short-term polarization variability of AGN remains mostly
unexplored. It is known that both fractional polarization and EVPA can significantly vary on the
minutes–hours time scale \citep{Itoh2013,Itoh2013b,Larionov2016}, but the origin of this variability
is obscure. The~solution of these and other problems requires continuation of systematic optical
polarimetric monitoring programs accompanied by a multiwavelength total-flux follow-up. Moreover,
the necessity of such programs is boosted in the context of future X-ray polarimetry missions
\citep{Fabiani2018,Weisskopf2018}. The~near-future X-ray polarimeters will require rather long
integration times. Therefore, ongoing continuous monitoring of optical polarization in blazars will
be extremely useful for picking highly polarized and active targets for observations in the X-rays.
RoboPol results (see Section~\ref{sec:rot_typ}), as well as theoretical studies \citep{Zhang2013}
suggest that at least HBL sources are expected to be highly polarized and variable in the X-rays
band. Therefore, we expect a bright future for polarimetry of AGN in the optical in synergy with the
X-rays \citep{Peirson2018}.

\vspace{12pt}

\authorcontributions{All authors contributed equally to the manuscript.}

\funding{D.B. acknowledges support by the European Research Council (ERC) under the
European Union’s Horizon 2020 research and innovation program under the grant agreement No 771282.}

\acknowledgments{The {\em RoboPol} project is a collaboration between Caltech in the USA, MPIfR in
Germany, Toru\'{n}~Centre~for~Astronomy~in~Poland, the~University of Crete/FORTH in Greece, and
IUCAA in India.}

\conflictsofinterest{The authors declare no conflict of interest.}

\reftitle{References}

High Energy Astrophysical Phenomena (astro-ph.HE)


Self-supervised Speaker Diarization

Abstract: Over the last few years, deep learning has grown in popularity for speaker verification, identification, and diarization. Inarguably, a significant part of this success is due to the demonstrated effectiveness of their speaker representations. These, however, are heavily dependent on large amounts of annotated data and can be sensitive to new domains. This study proposes an entirely unsupervised deep-learning model for speaker diarization. Specifically, the study focuses on generating high-quality neural speaker representations without any annotated data, as well as on estimating secondary hyperparameters of the model without annotations.The speaker embeddings are represented by an encoder trained in a self-supervised fashion using pairs of adjacent segments assumed to be of the same speaker. The trained encoder model is then used to self-generate pseudo-labels to subsequently train a similarity score between different segments of the same call using probabilistic linear discriminant analysis (PLDA) and further to learn a clustering stopping threshold. We compared our model to state-of-the-art unsupervised as well as supervised baselines on the CallHome benchmarks. According to empirical results, our approach outperforms unsupervised methods when only two speakers are present in the call, and is only slightly worse than recent supervised models.

\section{Introduction}

\IEEEPARstart{T}{his} document is a template for \LaTeX. If you are reading a paper or PDF version of this document, please download the electronic file, trans\_jour.tex, from the IEEE Web site at \url{http://www.ieee.org/authortools/trans_jour.tex} so you can use it to prepare your manuscript. If you would prefer to use LaTeX, download IEEE's LaTeX style and sample files from the same Web page. You can also explore using the Overleaf editor at {https://www.overleaf.com/blog/278-how-to-use-overleaf-with-ieee-collabratec-your-quick-guide-to-getting-started\#.xsVp6tpPkrKM9}

If your paper is intended for a conference, please contact your conference editor concerning acceptable word processor formats for your particular conference.

\section{Guidelines For Manuscript Preparation}

The IEEEtran\_HOWTO.pdf is the complete guide of \LaTeX\ for manuscript preparation included with this stuff.

\subsection{Information for Authors}

{\em IEEE Signal Processing Letters} allows only four-page articles. A fifth page is allowed for ``References'' only, though ``References'' may begin before the fifth page. Author biographies or photographs are not allowed in Signal Processing Letters. Please review the Information for Authors at for {\em IEEE Signal Processing Letters:} https://signalprocessingsociety.org/publications-resources/ieee-signal-processing-letters/information-authors-spl

\section{Guidelines for Graphics Preparation and Submission}
\label{sec:guidelines}

\subsection{Types of Graphics}
The following list outlines the different types of graphics published in
{\it IEEE Signal Processing Letters}. They are categorized based on their construction, and use of
color/shades of gray:

\subsubsection{Color/Grayscale figures}
{Figures that are meant to appear in color, or shades of black/gray. Such
figures may include photographs, illustrations, multicolor graphs, and
flowcharts.}

\subsubsection{Line Art figures}
{Figures that are composed of only black lines and shapes. These figures
should have no shades or half-tones of gray, only black and white.}

\subsubsection{Tables}
{Data charts which are typically black and white, but sometimes include
color.}

\subsection{Multipart figures}
Figures compiled of more than one sub-figure presented side-by-side, or
stacked. If a multipart figure is made up of multiple figure
types (one part is lineart, and another is grayscale or color) the figure
should meet the stricter guidelines.

\subsection{File Formats For Graphics}\label{formats}
Format and save your graphics using a suitable graphics processing program
that will allow you to create the images as PostScript (PS), Encapsulated
PostScript (.EPS), Tagged Image File Format (.TIFF), Portable Document
Format (.PDF), Portable Network Graphics (.PNG), or Metapost (.MPS), sizes them, and adjusts
the resolution settings. When
submitting your final paper, your graphics should all be submitted
individually in one of these formats along with the manuscript.

\subsection{Sizing of Graphics}
Most charts, graphs, and tables are one column wide (3.5 inches/88
millimeters/21 picas) or page wide (7.16 inches/181 millimeters/43
picas). The maximum depth a graphic can be is 8.5 inches (216 millimeters/54
picas). When choosing the depth of a graphic, please allow space for a
caption. Figures can be sized between column and page widths if the author
chooses, however it is recommended that figures are not sized less than
column width unless when necessary.

\begin{figure}
\centerline{\includegraphics[width=\columnwidth]{fig1.png}}
\caption{Magnetization as a function of applied field. Note that ``Fig.'' is abbreviated. There is a period after the figure number, followed by two spaces. It is good practice to explain the significance of the figure in the caption.}
\end{figure}

\begin{table}
\caption{Units for Magnetic Properties}
\label{table}
\small
\setlength{\tabcolsep}{3pt}
\begin{tabular}{|p{25pt}|p{75pt}|p{110pt}|}
\hline
Symbol&
Quantity&
Conversion from Gaussian and \par CGS EMU to SI$^{\mathrm{a}}$ \\
\hline
$\Phi $&
Magnetic flux&
1 Mx $\to  10^{-8}$ Wb $= 10^{-8}$ V $\cdot$ s \\
$B$&
Magnetic flux density, \par magnetic induction&
1 G $\to  10^{-4}$ T $= 10^{-4}$ Wb/m$^{2}$ \\
$H$&
Magnetic field strength&
1 Oe $\to  10^{-3}/(4\pi )$ A/m \\
$m$&
Magnetic moment&
1 erg/G $=$ 1 emu \par $\to 10^{-3}$ A $\cdot$ m$^{2} = 10^{-3}$ J/T \\
$M$&
Magnetization&
1 erg/(G $\cdot$ cm$^{3}) =$ 1 emu/cm$^{3}$ \par $\to 10^{-3}$ A/m \\
4$\pi M$&
Magnetization&
1 G $\to  10^{-3}/(4\pi )$ A/m \\
$\sigma $&
Specific magnetization&
1 erg/(G $\cdot$ g) $=$ 1 emu/g $\to $ 1 A $\cdot$ m$^{2}$/kg \\
$j$&
Magnetic dipole \par moment&
1 erg/G $=$ 1 emu \par $\to 4\pi \times  10^{-10}$ Wb $\cdot$ m \\
$J$&
Magnetic polarization&
1 erg/(G $\cdot$ cm$^{3}) =$ 1 emu/cm$^{3}$ \par $\to 4\pi \times  10^{-4}$ T \\
$\chi , \kappa $&
Susceptibility&
1 $\to  4\pi $ \\
$\chi_{\rho }$&
Mass susceptibility&
1 cm$^{3}$/g $\to  4\pi \times  10^{-3}$ m$^{3}$/kg \\
$\mu $&
Permeability&
1 $\to  4\pi \times  10^{-7}$ H/m \par $= 4\pi \times  10^{-7}$ Wb/(A $\cdot$ m) \\
$\mu_{r}$&
Relative permeability&
$\mu \to \mu_{r}$ \\
$w, W$&
Energy density&
1 erg/cm$^{3} \to  10^{-1}$ J/m$^{3}$ \\
$N, D$&
Demagnetizing factor&
1 $\to  1/(4\pi )$ \\
\hline
\multicolumn{3}{p{251pt}}{Vertical lines are optional in tables. Statements that serve as captions for
the entire table do not need footnote letters. }\\
\multicolumn{3}{p{251pt}}{$^{\mathrm{a}}$Gaussian units are the same as cg emu for magnetostatics; Mx
$=$ maxwell, G $=$ gauss, Oe $=$ oersted; Wb $=$ weber, V $=$ volt, s $=$
second, T $=$ tesla, m $=$ meter, A $=$ ampere, J $=$ joule, kg $=$
kilogram, H $=$ henry.}
\end{tabular}
\label{tab1}
\end{table}

\subsection{Resolution }
The proper resolution of your figures will depend on the type of figure it
is as defined in the ``Types of Figures'' section. Author photographs,
color, and grayscale figures should be at least 300dpi. Line art, including
tables should be a minimum of 600dpi.

\subsection{Vector Art}
In order to preserve the figures' integrity across multiple computer
platforms, we accept files in the following formats: .EPS/.PDF/.PS. All
fonts must be embedded or text converted to outlines in order to achieve the
best-quality results.

\subsection{Accepted Fonts Within Figures}
When preparing your graphics IEEE suggests that you use of one of the
following Open Type fonts: Times New Roman, Helvetica, Arial, Cambria, and
Symbol. If you are supplying EPS, PS, or PDF files all fonts must be
embedded. Some fonts may only be native to your operating system; without
the fonts embedded, parts of the graphic may be distorted or missing.

A safe option when finalizing your figures is to strip out the fonts before
you save the files, creating ``outline'' type. This converts fonts to
artwork what will appear uniformly on any screen.

\subsection{Using Labels Within Figures}

\subsubsection{Figure Axis labels }
Figure axis labels are often a source of confusion. Use words rather than
symbols. As an example, write the quantity ``Magnetization,'' or
``Magnetization M,'' not just ``M.'' Put units in parentheses. Do not label
axes only with units. As in Fig. 1, for example, write ``Magnetization
(A/m)'' or ``Magnetization (A$\cdot$m$^{-1}$),'' not just ``A/m.'' Do not label axes with a ratio of quantities and
units. For example, write ``Temperature (K),'' not ``Temperature/K.''

Multipliers can be especially confusing. Write ``Magnetization (kA/m)'' or
``Magnetization (10$^{3}$ A/m).'' Do not write ``Magnetization
(A/m)$\,\times\,$1000'' because the reader would not know whether the top
axis label in Fig. 1 meant 16000 A/m or 0.016 A/m. Figure labels should be
legible, approximately 8 to 10 point type.

\subsubsection{Subfigure Labels in Multipart Figures and Tables}
Multipart figures should be combined and labeled before final submission.
Labels should appear centered below each subfigure in 8 point Times New
Roman font in the format of (a) (b) (c).

\subsection{File Naming}
Figures (line artwork or photographs) should be named starting with the
first 5 letters of the author's last name. The next characters in the
filename should be the number that represents the sequential
location of this image in your article. For example, in author
``Anderson's'' paper, the first three figures would be named ander1.tif,
ander2.tif, and ander3.ps.

Tables should contain only the body of the table (not the caption) and
should be named similarly to figures, except that `.t' is inserted
in-between the author's name and the table number. For example, author
Anderson's first three tables would be named ander.t1.tif, ander.t2.ps,
ander.t3.eps.

\subsection{Referencing a Figure or Table Within Your Paper}
When referencing your figures and tables within your paper, use the
abbreviation ``Fig.'' even at the beginning of a sentence. Do not abbreviate
``Table.'' Tables should be numbered with Roman Numerals.

\subsection{Checking Your Figures: The IEEE Graphics Analyzer}
The IEEE Graphics Analyzer enables authors to pre-screen their graphics for
compliance with IEEE Transactions and Journals standards before submission.
The online tool, located at
\underline{http://graphicsqc.ieee.org/}, allows authors to
upload their graphics in order to check that each file is the correct file
format, resolution, size and colorspace; that no fonts are missing or
corrupt; that figures are not compiled in layers or have transparency, and
that they are named according to the IEEE Transactions and Journals naming
convention. At the end of this automated process, authors are provided with
a detailed report on each graphic within the web applet, as well as by
email.

For more information on using the Graphics Analyzer or any other graphics
related topic, contact the IEEE Graphics Help Desk by e-mail at
graphics@ieee.org.

\subsection{Submitting Your Graphics}
Because IEEE will do the final formatting of your paper,
you do not need to position figures and tables at the top and bottom of each
column. In fact, all figures, figure captions, and tables can be placed at
the end of your paper. In addition to, or even in lieu of submitting figures
within your final manuscript, figures should be submitted individually,
separate from the manuscript in one of the file formats listed above in
Section \ref{formats}. Place figure captions below the figures; place table titles
above the tables. Please do not include captions as part of the figures, or
put them in ``text boxes'' linked to the figures. Also, do not place borders
around the outside of your figures.

\subsection{Color Processing/Printing in IEEE Journals}
All IEEE Transactions, Journals, and Letters allow an author to publish
color figures on IEEE Xplore\textregistered\ at no charge, and automatically
convert them to grayscale for print versions. In most journals, figures and
tables may alternatively be printed in color if an author chooses to do so.
Please note that this service comes at an extra expense to the author. If
you intend to have print color graphics, include a note with your final
paper indicating which figures or tables you would like to be handled that
way, and stating that you are willing to pay the additional fee.

\section{Conclusion}

A conclusion section is not required. Although a conclusion may review the main points of the paper, do not replicate the abstract as the conclusion. A conclusion might elaborate on the importance of the work or suggest applications and extensions.

\section*{Acknowledgment}

The preferred spelling of the word ``acknowledgment'' in American English is without an ``e'' after the ``g.'' Use the singular heading even if you have many acknowledgments. Avoid expressions such as ``One of us (S.B.A.) would like to thank . . . .'' Instead, write “F. A. Author thanks ... .” In most cases, sponsor and financial support acknowledgments are placed in the unnumbered footnote on the first page, not here.

\section*{References and Footnotes}

\subsection{References}

References need not be cited in text. When they are, they appear on the line, in square brackets, inside the punctuation.  Multiple references are each numbered with separate brackets. When citing a section in a book, please give the relevant page numbers. In text, refer simply to the reference number. Do not use ``Ref.'' or ``reference'' except at the beginning of a sentence: ``Reference [3] shows . . . .'' Please do not use automatic endnotes in {\em Word}, rather, type the reference list at the end of the paper using the ``References'' style.

Reference numbers are set flush left and form a column of their own, hanging out beyond the body of the reference. The reference numbers are on the line, enclosed in square brackets. In all references, the given name of the author or editor is abbreviated to the initial only and precedes the last name. Use them all; use {\em et al.} only if names are not given. Use commas around Jr., Sr., and III in names. Abbreviate conference titles.  When citing IEEE transactions, provide the issue number, page range, volume number, year, and/or month if available. When referencing a patent, provide the day and the month of issue, or application. References may not include all information; please obtain and include relevant information. Do not combine references. There must be only one reference with each number. If there is a URL included with the print reference, it can be included at the end of the reference.

Other than books, capitalize only the first word in a paper title, except for proper nouns and element symbols. For papers published in translation journals, please give the English citation first, followed by the original foreign-language citation. See the end of this document for formats and examples of common references. For a complete discussion of references and their formats, see the IEEE style manual at www.ieee.org/authortools.

\subsection{Footnotes}

Number footnotes separately in superscripts (Insert $\mid$ Footnote).\footnote{It is recommended that footnotes be avoided (except for the unnumbered footnote with the receipt date on the first page). Instead, try to integrate the footnote information into the text.}  Place the actual footnote at the bottom of the column in which it is cited; do not put footnotes in the reference list (endnotes). Use letters for table footnotes (see Table I).

\section*{References}

\subsection*{Basic format for books:}

J. K. Author, ``Title of chapter in the book,'' in {\em Title of His Published Book}, xth ed. City of Publisher, (only U.S. State), Country: Abbrev. of Publisher, year, ch. x, sec. x, pp. xxx--xxx.

\subsection*{Examples:}
\def{}

\section{Discussion and future work}
\label{sec:conclusions}

This paper introduces a new approach for completely unsupervised speaker diarization. We showed that this method with a relatively small model, performs close to state-of-the-art fully supervised methods on the CallHome test set, further closing the gap between supervised and unsupervised methods.

For future work, we will explore ways of dealing with overlapping speech, which is not referenced in this work. Additionally, we will examine ways to incorporate negative examples in a way that can further improve results. Finally, we are interested in studying how leveraging massive amounts of unlabeled data affects this model.

\section{Empirical Evaluation}
\label{sec:experimental_setup}

In this section we describe a set of experiments to demonstrate the efficiency of the proposed method.

\subsection{Datasets}
For training, we used Fisher English Training Speech Part 1 (LDC2004S13) and Part 2 (LDC2005S13), CallFriend, Switchboard-1 Release 2 \cite{godfrey1992switchboard}, RuSTeN (LDC2006S34), Gulf (LDC2006S43) and Iraq (LDC2006S45) Arabic Conversational Telephone Speech. Our development and test sets were different parts of CallHome. Supervised methods \cite{horiguchi2020end, zeghidour2021dive, fujita2019end, fujita2020end, horiguchi2021end, fujita2019end_} commonly reported Diarization error rate (DER) results on a subset of the CallHome with a development set consisting of 155 two-speaker calls, and a test set of 148 two-speaker calls prepared in \cite{fujita2019end_}. For a fair comparison we use the same subset in Table~\ref{tab:all_new} and Table~\ref{tab:ablations}. Unsupervised methods \cite{castaldo2008stream, shum2013unsupervised} reported DER on the full multi-speaker test set of CallHome (the NIST SRE 2000 set), which we also use in Table \ref{tab:shum}. Since the two-speaker development set is a subset of the full Callhome test, in this setting we used a different subset of the NIST SRE set, namely disks six and seven, where there are more than one speaker per call, as the development set.

\subsection{Experimental setting}

We start with the speaker-discriminative embeddings training. The aforementioned training datasets were all converted into single-channel recordings. We applied VAD\footnote{We used Google's WebRTC with sensitivity 1 and frame size of 500 msec. It can be accessed at \url{https://github.com/wiseman/py-webrtcvad}.} to remove silences and non-speech. We randomly selected the inputs $\mathbf{x}_t$ and $\mathbf{x}_{\tau}$, where our segment size was 500 msec, and the segment pairs were 500 msec apart (using two adjacent segments resulted in an overfitting which did not generalize to good speaker separation). The encoder $f_\theta$ consists of 7 layers of 1-D convolutions (kernel sizes: 10,10,10,8,4,4,4; strides: 5,5,5,4,2,2,2; 128 channels per layer). Following this, there are 3 linear layers, each with 512 output units. The first 2 layers are followed by batch normalization and ReLU. We used the LARS optimizer \cite{you2017large}. We set a learning rate of 0.2 for the weights and 0.0048 for the biases and batch normalization parameters. We used a learning rate warm-up period of 10 epochs, after which we reduced the learning rate using a cosine decay schedule \cite{loshchilov2016sgdr}. The model was trained using a batch size of 2048. We trained the encoder $f_{\theta}$ until convergence (on our data, at around 30 epochs). This convergence also correlates well with an approximated DER we computed by clustering the produced embeddings. The resulting embeddings represent a 500 msec segment of speech.

To generate a self-supervised development set with pseudo labels, we took the development subset of CallHome and extracted embeddings using our encoder. Next, we clustered the embeddings using AHC. We cutoff the resulting dendograms arbitrarily at 10 clusters, selected the largest cluster and assumed cluster purity in the sense of it consisting of same speaker segments.
We assumed each resulting cluster from different files corresponded to different speakers.
To test our assumptions regarding the clustering purity, we checked DER by labeling only the segments of the largest cluster as a single speaker in each file and marked the rest of the file as silence. We got only a 0.3\% speaker error, along with an extremely high percentage of missed speech which is to be expected. This validates our assumption that each of the selected clusters consists almost entirely of a single speaker, creating accurate pseudo-labels for a development dataset.

The PLDA model was then trained on this pseudo labeled development set in the same fashion as in the supervised setting, using Kaldi's setup \cite{povey2011kaldi}. We also found a stopping threshold for the hierarchical clustering using this data by randomly generating 20,000 utterances.

\subsection{Results}\label{sec:results}

\begingroup
\setlength{\tabcolsep}{3.3pt}
\begin{table}[t]
\caption{Speaker error rate on the multi-speaker test set of CallHome compared with unsupervised methods.}
\label{tab:shum}
\centering
    \begin{tabular}{lcccccccc}
    \hline
    \hline
    \#speakers & $2$ & $3$ & $4$ & $5$ & $6$ & $7$ & $avg$ \\
    \hline
    Castaldo \cite{castaldo2008stream} & $8.7$ & $15.7$ & $15.1$ & $20.2$ & $25.5$ & $29.8$ & $13.7$ \\
    Schum \cite{shum2013unsupervised} & $14.5$ & ${\bf 14.0}$ & ${\bf 13.0}$ & ${\bf 17.0}$ & ${\bf 19.5}$ & ${\bf 24.0}$ & $14.5$ \\

    Ours & ${\bf 6.1}$ & $14.9$ & $18.3$ & $23.0$ & $30.7$ & $30.5$ & ${\bf 13.4}$ \\
\hline
    \#calls & $303$ & $136$ & $43$ & $10$ & $6$ & $2$ & $500$\\
\hline
\hline
\end{tabular}
\end{table}
\endgroup

\begin{table}[t]
  \caption{Comparison of our unsupervised model with recent supervised methods. DER is reported on the two speaker test set of CallHome prepared in \cite{fujita2019end_}.
}
\label{tab:all_new}
\centering
\small
\setlength{\tabcolsep}{3pt}
\begin{tabular}{l S}
\hline
\hline
Model & {DER} \\
\hline
UIS-RNN V1  \cite{zhang2019fully} & 10.6 \\
UIS-RNN V2  \cite{zhang2019fully} & 9.6  \\
UIS-RNN V3  \cite{zhang2019fully} & 7.6  \\
x-vector + LSTM (oracle VAD) \cite{lin2019lstm} & 6.6 \\
DIVE  \cite{zeghidour2021dive} &  5.9 \\
\hline
Ours (unsupervised, oracle VAD)   &  { 6.6} \\
Ours (unsupervised) &   { 9.1} \\
\hline
\hline
\end{tabular}
\end{table}

We turn to evaluate our method against state-of-the-art unsupervised and supervised models. Then, we demonstrate the effect of the amount of (unsupervised) data on the system's performance. Finally, we conclude with a detailed ablation study to gauge the impact of each component of our system. All measurements were obtained using the NIST DER tool with a collar of 250 msec and ignore overlapping sections.

We start by presenting our work compared with other unsupervised works mentioned in Section \ref{sec:intro}.
In these papers, the full multi-speaker test set of CallHome was used. This amounts to 500 recordings, each 2-5 minutes in length, containing between 2 and 7 participants.

Table \ref{tab:shum} reports the confusion error per speaker count and the average speaker error rate using oracle VAD of our work compared with two unsupervised methods: Castaldo \emph{et al.} \cite{castaldo2008stream} and Shum \emph{et al.} \cite{shum2013unsupervised}.
Since an oracle VAD is employed in these measurements, we
exclude false-alarms and miss-detection from our evaluations, and report speaker confusion only.
Our system is better for the two speakers setting and our overall (weighted) results are better than both methods.

Next, we would like to compare the performance of our unsupervised system to the performance of state-of-the-art supervised systems from recent years. Note that supervised systems are often evaluated on the two-speakers subset of the dataset that used in the evaluation of unsupervised methods. Table \ref{tab:all_new} reports DER on the two speakers test set of CallHome compared to recent state-of-the-art \emph{supervised} methods. UIS-RNN \cite{zhang2019fully} is a hybrid system training an RNN on top of pre-trained speaker embeddings, with the three version: V1 was trained on 36M utterances from 18K US English speakers (all mobile phone data); V2 was trained on additional non-public containing 34M utterances from 138K speakers, LibriSpeech, VoxCeleb, and VoxCeleb2, and V3 was based on variable-length processing windows, drawn uniform at random from [240 msec, 1600 msec]. DIVE \cite{zeghidour2021dive} combines three modules which are trained jointly: projection of the waveform to an embedding space, iterative selection of long-term speaker representations, and per-speaker per-timestep VAD and is the current state of the art reaching 5.9\% without overlap. Similar to UIS-RNN, \cite{lin2019lstm} suggested training an LSTM to model the similarity between pretrained speaker embeddings and then to perform the diarization. Unlike the other models in Table \ref{tab:all_new}, it uses oracle speech activity labels (removing silences).

The last rows are the result of our model, with learned embeddings, PLDA, and clustering tuned on pseudo labels, with and without overlap. The error breakdown without overlap is as follows: 1.5\% missed-speech, 2.2\% false-alarm, and 5.2\% speaker error. For a fair comparison, we added the performance of our system against using the oracle VAD labels. In this setting, we achieve a 6.58\% DER comparable to their 6.63\%. As seen, our unsupervised system is only slightly worse than state-of-the-art supervised systems from recent years.

\begin{figure}[t]
  \centering
  \includegraphics[width=0.95\linewidth]{figures/der_vs_data_line.pdf}
  \caption{Drop in DER as a function of the amount of hours of training data used for training.}
  \label{fig:data}
\end{figure}

In Figure \ref{fig:data} we show that the more unlabeled data that is used, the better the embedding model. Here we present different combinations of datasets we used and the total hours of training data used in each mode. The total hours are calculated by summing all the VAD segments longer than 3 seconds, as this is what was used for training and the DER is the calculated by clusering the embeddings using the number of speakers as a stopping criteria. As seen the DER ranges from 28\% for 50 hours of data to 11.5\% for 2400 hours. This bodes well for the system as most diarzation users will have access to large amounts of in domain unlabeled data.

\iffalse
\begin{table}
  \caption{An ablation study of the different system components. Results are reported in terms of DER [\%] on the two-speaker test set of CallHome prepared in \cite{fujita2019end_}.}
  \label{tab:ablations}
    \centering
    \small
    \setlength{\tabcolsep}{3pt}
  \begin{tabular}{l l l}
    \hline
    \hline
   Model & {Overlap} & {No overlap} \\
    \hline
    Embeddings + clustering + \#spk            &  16.94  & 12.03 \\
    Embeddings + PLDA + clustering          &  14.47  & 9.15 \\
    Embeddings + PLDA + \#spk &  14.21      &  8.89 \\
    Embeddings + PLDA + clust. + (oracle VAD) &    12.31   &  6.59 \\
    VicReg embeddings + clustering + \#spk   &  17.9     &  13.1 \\
    \hline
    \hline
  \end{tabular}
\end{table}
\fi

\begin{table}
  \caption{An ablation study of the different system components. Results are reported on the two-speaker test set of CallHome.}
  \label{tab:ablations}
    \centering
  \begin{tabular}{l S}
    \hline
    \hline
   Model & {DER} \\
    \hline
    Embeddings + clustering + \#spk & 11.5 \\
    Embeddings + PLDA + clustering & 9.1 \\
    Embeddings + PLDA + \#spk & 8.7 \\
    Embeddings + PLDA + clust. + (oracle VAD) & 6.6 \\
    VicReg embeddings + clustering + \#spk & 13.1 \\
    \hline
    \hline
  \end{tabular}
  \vspace{-.3cm}
\end{table}

Last, we evaluated our system by methodically omitting the different components. The results in terms of DER [\%] on the test set of CallHome are reported in Table \ref{tab:ablations}.

In the table, \emph{Embeddings + clustering + \#spk} denotes the results when using only the learned speaker-embeddings without incorporating the learned PLDA similarity matrix, but where the speaker count is known and is used as a stopping criterion for the clustering procedure. This is essentially a measure of the goodness of the learned speaker embeddings. \emph{Embeddings + PLDA + clustering} refers to our final choice of architecture, as described in Section~\ref{sec:model}.

\emph{Embeddings + PLDA + \#spk} is our final model when the speaker count in each file is known and used as the clustering stopping criteria. As expected, knowing the speaker count in each file yields slightly better results. These results suggest that incorporating our proposed self-supervised PLDA similarity matrix improved results by 24\% in relative terms over pure embeddings. \emph{Embeddings + PLDA + oracle VAD} denotes the results when using the oracle VAD. This shows that improving the VAD can significantly lower the DER.

For completeness, we also trained the speaker-embedding system using the \emph{VicReg} objective proposed in \cite{bardes2021vicreg}. This objective similar to \emph{Barlow Twins}, exchanges the variance term with a hinge loss function and adds a term minimizing the MSE between embedding vectors.  We denote this setting by \emph{VecReg embeddings + clustering + \#spk}. However results suggest that this system was inferior by approximately 1.5\% absolute DER.

\section{Introduction}
\label{sec:intro}

Speaker diarization is the task of determining ``who spoke when'' in an audio recording, usually with an unknown number of speakers and variable speech duration. Traditionally, speaker diarization systems consist of four submodules. The first module extracts speech segments using a voice activity detector (VAD). The second module transforms the spoken segments into fixed-length vectors representing speakers' identity. This transformation is performed using a parametric probabilistic model, such as i-vectors based on GMM-UBM \cite{dehak2010front}, or using deep neural network (DNN) embeddings, such as x-vectors \cite{snyder2018x} or d-vectors \cite{wan2018generalized}. The third module finds a pairwise similarity measurement between segments using cosine distance, probabilistic linear discriminant analysis (PLDA) \cite{ioffe2006probabilistic}, or DNN models \cite{lin2019lstm}. Finally, the resulting similarity is clustered to determine the number of speakers and assign a speaker label to each segment. This is often done using a clustering model such as agglomerative hierarchical clustering (AHC) \cite{day1984efficient}, with a stopping threshold tuned on some held-out data \cite{fuchs2017spoken}.

One common theme amongst all the methods is the need for large amounts of labeled data, especially when considering DNN-based speaker embeddings. This would not be an issue if not for the domain sensitivity of the diarization systems. That is, training embeddings on one dataset might not translate into good representations on a different domain, and annotating data in the new domain can be prohibitively expensive.

Our study aims to build the traditional diarization pipeline, but to replace each component with components that are trained completely unsupervised, thus removing the need for any annotated data. This opens new opportunities in applying speaker diarization especially for domains for which annotated data is not available. Hence, the work is focused on the main submodules of speaker diarization, namely, speaker embedding, segment similarity, and segment clustering, where the VAD is left out of the scope of this study.

In the self-supervised learning (SSL) setting, unlabeled inputs define an auxiliary task that can generate pseudo-labeled training data. Then, a model can be trained using supervised learning techniques, and typically with the contrastive loss function, which aims to learn a metric indicating that a pair of positive and negative examples are dissimilar. For example, the SSL model in \cite{kreuk2020self} is designed to distinguish between an adjacent frame, which is considered a positive example, and a randomly sampled future frame in the sequence, which is considered a negative example.

In the supervised diarization setting, a positive and negative pair can correspond to the same speaker and different speaker segments, respectively. In the unsupervised setting, on the other hand, we have no supervision of who spoke when, hence we cannot easily generate positive and negative pairs. We cannot confidently assume that randomly sampled frames in an audio sequence represent a different speaker and hence considered as the negative frames as proposed in \cite{kreuk2020self}. Additionally, we cannot sample negative segments from different audio recordings as this can introduce other artifacts, and the negative examples might represent different acoustic environments rather than different speakers.

Due to these constraints, we take inspiration from recent developments in SSL for image representation \cite{zbontar2021barlow, bardes2021vicreg} that do not require negative examples. In those works, pairs of positive examples are generated by creating two distortions of the same image. The objective of the learning algorithm is to create a representation embedding invariant to distortions. In our setting, the inputs to the proposed models are raw speech segments considered to be spoken by the same speaker. Our setting aims to create a speaker-embedding that is invariant to the segment spoken by the speaker, and it is achieved by maximizing the agreement between embedding vectors of two different segments of the same speaker (rather than two distortions of the same speech segment.)

The self-supervised speaker embeddings is then used to create a development set of pseudo-labels, which, in turn, are used to train a PLDA-based similarity matrix to the segments in a given audio stream. Additionally, the pseudo-labels are used to tune the speaker clustering stopping threshold.

The contribution of this paper is a full unsupervised model for training a speaker diarization system, including all its sub-components: a speaker embedding model, a PLDA-based scoring matrix, and the detection thresholds. The results show that our unsupervised method is better than other unsupervised methods when two speakers are concerned, and comparable to the state-of-the-art fully-supervised methods from just a couple years ago. Our implementation is available at {\footnotesize \url{https://github.com/MLSpeech/ssl_diarization}}.

\smallskip

\noindent {\bf Related work. }
There are few methods for unsupervised diarization that have been previously proposed. Castaldo \emph{et al.} \cite{castaldo2008stream} proposed an  expectation-maximization (EM) algorithm to the problem. First, a blind segmentation of the audio stream is performed. Then, a speaker model is estimated on the current segmentation to become the new global model or update the nearest speaker global model.
Shum \emph{et al.} \cite{shum2013unsupervised} also proposed an iterative algorithm for speaker clustering based on principal component analysis and variational inference. Finally, the work  \cite{le2016speaker} proposes to use Bayesian Information Criterion (BIC) for creating training data and training a PLDA similarity metric on top of it. All those methods are based on i-vectors of GMM-UBM. Furthermore, they all need annotated data for tuning thresholds or biases. In contrast, our approach relies on recent advances in DNN-based speaker representation and does not use any annotated data for tuning thresholds or biases.

\section{Model}
\label{sec:model}

\begin{figure}[t]
  \centering
  \includegraphics[width=0.85\linewidth]{figures/barlow.pdf}
  \caption{An illustration of our embeddings model and training scheme.}
  \label{fig:model}
\end{figure}

We denote a speech utterance of $T$ samples as $\mathbf{x}=\left(x_{1}, \ldots, x_{T}\right)$, where $\mathbf{x} \in \mathbb{R}^{T}$. We assume the existence of a label set $\mathcal{Y}=\left\{ \phi ,1,\ldots, K\right\}$, where $\phi$ is the symbol for silence or non-speech, and $K$ is the number of assumed speakers. In our setting, the value of $K$ is unknown at training time or during the inference, nor the labels associated with the inputs. We used the labels only for final evaluation purposes.

We denote by $\mathbf{x}_t=\mathbf{x}_{t-T'}^{t+T'}$ an audio segment of $2T'$ samples around the $t$-th sample, and we denote by $N$ the number of segments in the utterance. Our goal is to learn a function that predicts the speaker value  $y_t \in \mathcal{Y}$  for each segment, where it is assumed there is only one speaker within each segment. The proposed model is built from three components: an encoder $f_{\theta}$ with parameters $\theta$, a similarity scoring matrix $S\in\mathbb{R}^{N\times N}$ between the $N$ segments of the utterance, and a hierarchical clustering of the scoring matrix.

The encoder $f_{\theta}$ is depicted in Figure~\ref{fig:model}. It gets as input a speech segment $\mathbf{x}_t$ and output an embedding vector $\mathbf{z}_t=f_{\theta}(\mathbf{x}_t)$. Ideally, the encoder should be trained using a contrastive loss function. However, since we assume we do not have access to segments of different speakers (negative examples), the encoder is trained using a non-contrastive self-supervised loss function. Our loss function gets two representations assumed to be of the same speaker as input. We denote these representations by $\mathbf{z}_t$ and $\mathbf{z}_{\tau}$ ($t\ne\tau$,) which corresponds to input segments $\mathbf{x}_t$ and $\mathbf{x}_{\tau}$, respectively.

Denote the \emph{cross-correlation matrix} between the two embeddings as $\mathbf{R}_{\mathbf{Z}_t \mathbf{Z}_{\tau}}$:
\begin{equation}
\mathbf{R}_{\mathbf{Z}_t \mathbf{Z}_{\tau}} = \mathbb{E} \left[ \mathbf{z}_t^\top \mathbf{z}_\tau \right],
\end{equation}
where $\mathbf{Z}_t$ and $\mathbf{Z}_{\tau}$ denote the random variable associated with the vectors $\mathbf{z}_t$ and $\mathbf{z}_{\tau}$. Our self-supervised loss function is aimed at making the cross-correlation matrix $\mathbf{R}_{\mathbf{Z}_t \mathbf{Z}_{\tau}}$ as close as possible to the identity matrix, $\mathbf{I}$, under the Frobenius norm:
\begin{equation}\label{eq:loss}
L_{BT}(\mathbf{z}_t, \mathbf{z}_{\tau}) =  \|\mathbf{R}_{\mathbf{Z}_t \mathbf{Z}_{\tau}} - \mathbf{I} \|^2_\mathcal{F} ~.
\end{equation}
By using this loss, the diagonal elements of the matrix are steered towards 1 to make the embeddings invariant to different segments of the same speaker. In contrast, the off-diagonal elements are steered to 0, thus decorrelating the vector components of the embeddings. This loss function is known as the \emph{Barlow Twins} loss \cite{zbontar2021barlow}. Practically, the cross-correlation matrix is computed over the examples within the batch, and the parameters $\theta$ of the encoder function $f_\theta$ are found by minimizing this loss function on the unlabeled training set of utterances.

For subsequent processing, we generate a dataset with \emph{pseudo-speaker labels}. Our challenge is reliably labeling this dataset. First, we compute the embeddings of the utterances in our \emph{unlabeled development set} using the trained encoder. Then, we compute the Euclidean distances between these embeddings to generate the distance matrix $\mathbf{D}$ whose elements are:
\[
[\mathbf{D}]_{i,j} = \|\mathbf{z}_i - \mathbf{z}_{j}\|^2_2 ~.
\]
Next, we cluster the matrix $\mathbf{D}$ using agglomerative hierarchical clustering (AHC), resulting in a dendrogram, which is a diagram of the hierarchical relationship between segments. We decide on a cutoff point in the dendrogram that results in a number of clusters we assume to be higher than the speaker count of any file in our data, which consequently enables us to assume with high confidence cluster purity, i.e., the embeddings contained within each cluster -- all represent the same speaker. The higher the cutoff, the higher the confidence. Then, we take the embeddings from the largest-sized cluster from each utterance and label them as a pseudo-speaker. We repeat this process for each utterance in the development set, using a new pseudo-speaker label each time.

The second part of our system is a similarity scoring model $\mathbf{S}$, which outputs the similarity scores $\mathbf{S}(\mathbf{z})_{i,j}$ between $\mathbf{z}_i$ and $\mathbf{z}_j$ for all $i$ and $j$ segments of each utterance $\mathbf{z}$. The scoring model $\mathbf{S}$ is obtained by training a probabilistic linear discriminant analysis (PLDA) on the aforementioned development set in the same fashion as in the supervised setting, but with the pseudo generated labels \cite{prazak2011speaker}.

Finally, given an encoded utterance $\mathbf{z}$, we segment $\mathbf{z}$ to speakers by applying AHC on the similarity matrix $\mathbf{S}(\mathbf{z})$.
For this, we need to select the threshold for the stopping criteria of the hierarchical clustering in an \emph{unsupervised} manner. We use our self-labeled development set to randomly sample segments (10 frames) from each pseudo-speaker (with repetitions). Next, we randomly concatenate these segments to create utterances with multiple speakers. We compute the similarity matrix for each generated utterance and use the $i$-th row of the similarity matrix $\mathbf{S}(\mathbf{z})$, denoted $\mathbf{S}(\mathbf{z})_i$, to represent the $i$-th segment of an input utterance. We set the similarity metric between segment $i$ and segment $j$ as $\|\mathbf{S}(\mathbf{z})_i-\mathbf{S}(\mathbf{z})_j\|^2$. Traditionally, the PLDA scores themselves are used for this clustering step. However, during our experiments, we found that using the row representations results in a more robust threshold than when merely using the pre-computed scores. Using these scores, we tune the clustering threshold on the development dataset and select the threshold that yields the lowest Diarization Error Rate (DER) on the pseudo labels. Empirically, we found that the resulting threshold also generalizes well to the test data.

Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

Self-supervised Speaker Diarization