Small and young radio sources

Abstract: It is currently accepted that compact and bright radio sources characterized by a convex spectrum peaking at frequencies ranging from 100 MHz to a few GHz are young objects. In this scenario, high frequency peaker (HFP) radio sources, with a turnover frequency higher than 5 GHz are good candidates to be extremely young radio sources with ages of up to a few thousand years. The knowledge of the conditions in young radio source is fundamental in order to draw reliable evolution models able to describe the entire life-cycle of the radio emission. Given the high spatial resolution and the large frequency range spanned, VLBI observations provide a unique opportunity to constrain the physical conditions in young radio sources, and to investigate the role played by the environment on the source growth.

\section{Small and young radio sources}

An active galactic nucleus (AGN) is a compact and luminous
region hosted in the centre of a galaxy. Its luminosity, much higher
than that observed in ``non-active'' galaxies, is not due to stellar
emission, but it is related to accretion mechanisms on the central
black hole. Only a small fraction of the AGN population ($\sim$ 10\%)
is radio-loud, making the origin of the radio emission
a still open question.\\
The onset of radio emission is currently considered related to mergers
or accretion events taking place in the host galaxy. In the
early-stages of the radio source evolution, the radio emission has to
make its way through the dense interstellar medium (ISM) before plunging
into the smoother intracluster medium (ICM), and giving origin to extended (up
to a few Mpc) radio sources.
In this scenario, the linear size of the radio
source gives an indication of the evolution stage (i.e. the age) of the
radio emission.\\
A significant fraction (15-30\%) of sources in
flux-density limited radio catalogues is represented by powerful
(P$_{\rm 1.4 GHz} > 10^{25}$ W/Hz) and intrinsically
compact ($<$ 1$^{\prime\prime}$ ) radio
sources, with a convex synchrotron radio spectrum peaking at
frequencies between $\sim$100 MHz and a few GHz.
If the peak of the spectrum occurs either at a few hundreds MHz or around one
GHz, the radio sources are identified as ``compact steep spectrum
(CSS) or ``GHz-peaked spectrum'' (GPS) respectively. \\
When imaged with high
spatial resolution, they usually display a two-sided morphology
dominated by hot-spots and mini-lobes. For this reason, these radio
sources are known as ``Compact Symmetric Objects'' (CSOs).\\
Their small size, together with
their morphology, a scaled-down version (0.1 - 15 kpc) of the extended
classical radio galaxies, suggest
that these objects represent a young stage in the individual radio
source evolution (see e.g. \cite{fanti95, snellen00}). This interpretation is supported by the detection of
hot spot advance speed in a dozen of the most compact CSOs, leading to
a kinematic age of about $10^{3}$ years \cite{pc03}. The young age of
CSOs has a further confirmation by means of spectral studies
\cite{murgia99, murgia03} providing radiative ages of
$10^{3} - 10^{5}$ years.\\
The competing model, known as the {\it frustration scenario}, postulates
that these objects are small because their are trapped by an
extremely dense ambient medium able to frustrate the source
growth \cite{vbreugel84}.
However, no observational evidence of the presence of an
unusually dense environment has been found \cite{fanti00, aneta05},
giving an indirect support to the {\it youth scenario}.\\
The main property characterizing small and young radio sources is
the presence of a peak in their synchrotron spectrum.
This feature is commonly
explained by means of synchrotron-self absorption (SSA)
\cite{snellen00}. The empirical anti-correlation found between the linear
size and the peak frequency \cite{odea97} supports this view: as
the source grows, its radio emission becomes optically-thin at lower and lower
frequencies. This implies that the smallest, and thus youngest radio
sources, must be sought among those with a spectrum peaking at high
frequency, above a few GHz. With their spectral peak occurring well
above 4-5 GHz, high frequency peakers (HFP) are the best candidate to
be newly born radio sources \cite{dd00, dd03}.\\
Since the selection of young radio sources is based mainly on the
position of the spectral peak, it is possible that different kind of
objects, like blazars may contaminate the samples. In this objects, the
compact size is due to projection effects which
foreshorten their real linear size, and it is not related to the source
age. To discriminate between young and boosted objects, an accurate
analysis of the radio properties is unavoidable.\\

\section{Radio properties}

\subsection{Spectral variability}

The presence of a convex spectrum with a peak occurring at different
frequencies depending on the age of the source, is the main
characteristic of young radio sources. In the presence of SSA, the
peak is expected to move to lower frequency and the flux density in
the optically-thick part of the spectrum should increase
as the source grows in size,
without indication of significant variability in the spectrum on small
temporary scale. Multi-epoch observations showed that CSS, GPS
and HFP sources identified with galaxies, are the least variable class of
extragalactic objects \cite{odea98, stanghella98, orienti07},
while the majority of those
identified with quasars show flux-density and spectral-shape variability
\cite{torniainen05, tinti05}. In a
significant fraction of GPS/HFP quasars,
the radio spectrum
turns out to be flat in a subsequent observing epoch
\cite{torniainen05, orienti07},
indicating a different nature for this class of objects. \\

\subsection{Morphology}

Given the sub-arcsecond size of young radio sources, the study of
their structure became possible with the advent of VLBI. The first
pioneering VLBI works on CSS/GPS sources revealed simple symmetric
structures. Deeper observations allowed a more accurate description of
their morphology \cite{fanti90}.
Young radio sources, at least those identified with
galaxies, have a ``symmetric'' structure dominated by two well-separated
components with steep spectral index, interpreted as mini-lobes and/or
hot spots, occasionally with a weak
unresolved component in between, accounting for a small fraction
($<$1\%) of the flux density, usually detected at high frequency
only, which may contain the source core (Fig. \ref{symm}a).
In this context, the term ``symmetric'' only refers to a two-sided
morphology  and does not imply that the source components have similar
characteristics. Between the ``symmetric'' objects there are radio sources
with highly asymmetric structures (Fig. \ref{symm}b). In a significant
fraction ($\sim$65\%, \cite{saikia03}) of small radio galaxies, the
two lobes have very different luminosity. Furthermore, when the
source core is detected, it has been observed that also the separation
ratio of the outer components from the nucleus is asymmetric, being
the brightest also the closest to the core. Such
asymmetries are more commonly found in small radio sources, than
in large radio galaxies \cite{longair00}, suggesting an influence
played by the ambient medium on the source growth.\\
On the other hand, small radio
sources identified with quasars mainly display a one-sided Core-Jet
morphology with a straight or bent structure always with a well
visible core accounting for the majority of the flux
density, and located at one edge of the source. The Core-Jet
morphology is the structure usually displayed by beamed blazar
objects. Small radio sources with such a morphology also
show significant flux density and spectral-shape variability
\cite{orienti07}, suggesting, again, a different interpretation for the
radio emission.\\

\begin{figure}
\begin{center}
\special{psfile=1943_dd.eps voffset=-155 hoffset=0 vscale=35
  hscale=35}
\special{psfile=0147_evn.eps voffset=-165 hoffset=260 vscale=30 hscale=30}
\vspace{5cm}
\caption{Example of ``symmetric'' structures in CSS/GPS radio sources.}
\label{symm}
\end{center}
\end{figure}

\subsection{Polarization}

The study of the polarization enables us to derive the distribution of
the magnetic field in young radio sources. However,
young radio sources completely reside within the ISM
of the host galaxy. This gas can act as a Faraday Screen
causing substantial depolarization of the synchrotron radiation.\\
In large ($>$ 5 kpc) CSS objects, the radiation has high percentage of
polarization ($>$10\%, \cite{fanti04}) and the magnetic field is
generally parallel to the jet axis \cite{fanti01}, as found in
edge-brightened large radio galaxies. On the other hand,
radio galaxies smaller than 5 kpc are
unpolarized or possess very low percentage of polarized
emission \cite{fanti04, orienti08a}.
The high rotation measures (RM) found suggests that the
radiation is strongly depolarized by the magneto-ionic plasma of the
ambient medium. \\
As found for variability and morphological properties,
a different behaviour is shown by GPS and HFP quasars, where high
values of polarized emission are detected \cite{orienti08a,
  stanghella98}. \\

The different characteristics shown by small radio sources with
different optical identification suggest that
small radio objects hosted either by
galaxies or by quasars represent two different radio source populations:
genuinely young radio objects the former, beamed blazars the
latter.\\

\section{Physical conditions}

The radio emission
of extragalactic objects is due to synchrotron radiation produced by
relativistic electrons with a power-law energy distribution. A curvature
in the spectrum produced by
synchrotron self-absorption mechanism is thus present by default, and
the frequency at which it occurs is strongly related to the size of the
radio component. In the context of evolution models, this explain
the anti-correlation found between peak frequency and linear size:
as the source expands, the synchrotron radiation becomes optically
thin at lower energy, shifting the radio spectrum towards lower
frequency. \\
In order to draw a complete picture able to describe the various stages
of the radio source evolution, a deep knowledge of the physical
conditions in young radio sources is necessary.
Evolution models proposed so far, e.g.  \cite{fanti95, snellen00}, are
based on the assumption of minimum energy conditions, corresponding to
a near equipartition of energy between particles and magnetic field
\cite{pacho70}. Equipartition magnetic fields derived in young radio
sources ranges from a few mG, in the compact components of CSS
\cite{fanti95}, up to
tens or hundreds mG in HFPs \cite{orienti08b}.
However, there is no a-priori reason for believing that
magnetic fields are those of equipartition, and it is important to
derive the magnetic field in an independent way.\\
If the spectral peak is due to SSA as stated before, the magnetic
field $B$ of a homogeneous synchrotron component
can be computed by observable quantities only:

\begin{equation}
B = f( \alpha )^{-5} \theta^{4} \nu_{\rm max}^{5} S_{\rm max}^{-2}
(1+z)^{-1}
\label{ssa}
\end{equation}

\noindent where $\theta$ is the source angular size, $S_{\rm max}$ is
the peak flux density, $\nu_{\rm max}$ the peak frequency, $z$ is the
redshift and $f(\alpha$) is a function of the spectral index
\cite{kellerman81}. \\
The main difficulty in applying this method is the uncertainty in
determining the source parameters, in particular the peak frequency
and the angular
size. For an accurate determination of the peak frequency, the radio
spectrum must be well sampled in both its optically-thick and -thin
parts. Furthermore, the magnetic field should not be computed on the
whole source, but on individual sub-structures, which are a better
approximation of homogeneous components.\\
The VLBI has the ideal frequency coverage and spatial resolution to
perform such an investigation.\\
Magnetic fields derived from the spectral peak of CSS radio sources
were within a factor of 16 of the equipartition values
\cite{scott77}. The discrepancy between the values may arise from the
uncertainties in the determination of the spectral peak which occurs at
rather low frequencies where the sampling is more difficult and size
determination more critical.\\
In HFP objects, the spectral peak at a few GHz allows an accurate
determination of the peak parameters. Furthermore, given their compact
size due to the severe radiative losses which prevent the formation of
extended lobe-like features,
HFP sub-structures
can be considered a good approximation of uniform synchrotron components.
The magnetic fields inferred are
in good agreement with those computed assuming equipartition, with a
few exceptions \cite{orienti08b}.
In some HFP components, the magnetic fields derived are above a few
Gauss, that is one or two orders of magnitude higher than the equipartition
value. However, in these components it was noted that
the optically-thick part of the spectrum has a
spectral index $\alpha < -2.5$ ($S \propto \nu^{- \alpha}$), i.e. the
canonical value of SSA, implying the presence of an additional
contribution from free-free absorption (FFA). In this case, the
magnetic fields derived by means of the peak parameters are physically
meaningless. \\

\section{Ambient medium}

The origin of radio activity in AGN is thought to be related to the
availability of fuel to feed the central engine, likely provided by
merger events. For this reason, galaxies hosting a radio source in an early
evolutionary stage, are supposed to possess a rich and dense
interstellar medium. The presence of significant amount of gas in young
radio sources was proved by the high incidence of H{\small I} absorption
\cite{ylva03}, which is substantially higher than what typically found
in large (hundreds of kpc in size) radio galaxies \cite{morganti01}.\\
Statistical studies of the properties of the H{\small I} absorption in young
CSS/GPS objects pinpointed the existence of an anti-correlation
between the H{\small I} column density and the linear size \cite{ylva03,
  gupta06}: the smaller the source, the larger the H{\small I} column
density. This result can be explained assuming that the neutral
hydrogen is organized in a circumnuclear structure, such as a torus or
a disk. In this context, the H{\small I} absorption is detected against the
receading jet, when our line of sight passes through the circumnuclear
disk/torus along its way towards the radio emission
\cite{mundell03}. As the source grows, the background emission of
the counter-jet moves toward the outer and less dense part of the
atomic structure, causing a decrement of the H{\small I} column
density. \\
Surprisingly,
the anti-correlation found in CSS/GPS objects is not present in the
small HFP sources \cite{orienti06}.
This unexpected result can be explained again in a torus
scenario. Given the extremely small size of HFPs, the line of sight
to the counter-jet passes mainly through the ionized part of the ISM,
piercing the atomic structure in its innermost part, where it is
hotter and less dense. Support to this interpretation arises from the
detection of large amount gas in HFPs \cite{dallacasa09,
  orienti08b}, responsible for free-free absorption.\\
The anti-correlation between linear size and H{\small I}
column density was derived by observations with a spatial
resolution not adequate to determine the distribution of the atomic gas.
To test whether the neutral hydrogen is settled in a circumnuclear
structure, the high spatial resolution provided by VLBI observations
are needed.\\
In the GPS sources 3C\,31.04 \cite{conway96} and 1946+708 \cite{peck99} a
torus-like structure has been detected. On the other hand, in the
GPS sources 4C\,12.50 \cite{morganti04} and 0402+379
\cite{maness04}, and in the two very asymmetric
CSS 3C\,49 and 3C\,268.3 \cite{labiano06},
VLBI observations showed that H{\small I} absorption occurs far away
from the centre, indicating that the absorption is likely due to an
off-nuclear cloud. Furthermore, in 3C\,49 and 3C\,268.3, H{\small I}
absorption is detected only against the brightest lobe, which turned
out to be also the closest to the core. This suggests that
jet-cloud
interactions may take place, at least when the radio emission is still
growing within the ISM of the host galaxies. Such an impact
may slow down the propagation of the interacting jet and enhance its
luminosity, explaining in this way the strong luminosity an arm-length
asymmetries observed in many young radio sources.

\section{Conclusions}
Flux density limited catalogues are known to possess a significant
fraction of small radio sources showing a peak in their synchrotron
radio spectra. Their small size, together with their two-sided
morphology, a scaled-down version of the structures shown by large
radio galaxies, suggest that these radio sources represent an early
stage in the evolution of the radio emission.
The study of their radio properties, namely the spectral variability,
morphology and polarization, pointed out that those objects optically
identified with galaxies are genuinely young radio sources, while the
majority of those identified with quasars are likely boosted blazar
objects.\\
The presence of a spectral peak is the main characteristic of young
radio sources. This peak is due to SSA, although an additional
contribution from FFA has been found in the most compact objects.
Where the spectral peak is originated by SSA only, it is possible
to derive the magnetic field by means of the peak parameters. Magnetic
fields computed in this way are generally in good
agreement with those calculated assuming
equipartition of energy between particles and magnetic field,
providing a strong evidence that young radio
sources are in minimum energy conditions. \\
The presence of a dense and inhomogeneous ambient medium surrounding
young radio sources has been pinpointed by frequent detection of
neutral hydrogen in absorption. Although in many cases the atomic gas
is organized in settled, circumnuclear structures as disks and tori,
VLBI observations showed the presence of unsettled, off-nuclear atomic
clouds, responsible for H{\small I} absorption in the outer regions of
the radio sources. The interaction between one of the jet and an
off-nuclear cloud may influence the source growth, for example slowing
down the source expansion, and producing asymmetric structures. \\
Future telescopes with good sensitivity and resolution like SKA, will
provide us with new information on the source environment, that is a
fundamental ingredient for the definition of a complete picture of the
radio source evolution.\\

Cosmology and Nongalactic Astrophysics (astro-ph.CO)

Small and young radio sources


Drug-Target Interaction Prediction with Graph Attention networks

Abstract: Motivation: Predicting Drug-Target Interaction (DTI) is a well-studied topic in bioinformatics due to its relevance in the fields of proteomics and pharmaceutical research. Although many machine learning methods have been successfully applied in this task, few of them aim at leveraging the inherent heterogeneous graph structure in the DTI network to address the challenge. For better learning and interpreting the DTI topological structure and the similarity, it is desirable to have methods specifically for predicting interactions from the graph structure.Results: We present an end-to-end framework, DTI-GAT (Drug-Target Interaction prediction with Graph Attention networks) for DTI predictions. DTI-GAT incorporates a deep neural network architecture that operates on graph-structured data with the attention mechanism, which leverages both the interaction patterns and the features of drug and protein sequences. DTI-GAT facilitates the interpretation of the DTI topological structure by assigning different attention weights to each node with the self-attention mechanism. Experimental evaluations show that DTI-GAT outperforms various state-of-the-art systems on the binary DTI prediction problem. Moreover, the independent study results further demonstrate that our model can be generalized better than other conventional methods.Availability: The source code and all datasets are available atthis https URL

\section{Introduction}\label{sec:intro}

Detecting drug–target interactions (DTIs) potentially facilitates therapeutic target identification \citep{xia2010semi,petta2016modulation} and novel drug design \citep{skrabanek2008computational, ay2007drug, janga2009structure, kuhn2008large}.
Until quite recently, pharmacological effects were often discovered using primitive trial and error procedures, such as applying plant extracts on living systems and observing the outcomes \citep{singh2016biological}.
However, experiment-based methods remain expensive, labor-intensive and time-consuming \citep{dickson2004key, kola2004can, kapetanovic2008computer}.
Evidently, there is an immense need for reliable computational approaches to identify and characterize DTIs, hoping to accelerate the pace and reduce the cost of drug development.

With the rapid development of machine learning techniques, various computational prediction approaches have been proposed to predict drug-target interactions.
\cite{yamanishi2010drug} integrated the relationship among the pharmacological space, the chemical space, and the topology of drug–target interaction networks to predict the associations between drugs and targets, and their experimental results have demonstrated that drug–target interactions are more correlated with pharmacological effect similarity than with chemical structure similarity.
According to the similarity of chemical information, \cite{keiser2009predicting} proposed a method to explore the associations between drugs and targets. They selected 30 of predicted results for biological experiments and finally confirmed 23 with interrelationships.
\cite{wang2010computationally} used supervised machine learning methods to predict the relationship between drugs and targets. To solve the problem of sample imbalance, they are collecting the positive samples from the database, and the negative samples using the random selection method. The input features of the classifier consist of the chemical structure of the drug and the sequence information of the protein.
\cite{chen2012drug} developed a novel method of Network-based Random Walk with Restart on the Heterogeneous (NRWRH) network to predict potential drug–target interactions on a large scale. The excellent experimental results show that the proposed method can discover new potential drug–target interactions for drug development.
These approaches provide feasible solutions to the problem. However, the extracted features used in these approaches only have limited coverage on interaction information, since they are dedicated to specific facets of the protein and the drug profiles.

To alleviate the inadequacy of statistical learning methods, deep learning algorithms provide the powerful functionality to represent large-scale raw data for different tasks and thus facilitate the learning of latent patterns in the data \citep{lecun2015deep}.
Recently, deep learning architectures have produced powerful systems to address several estimation problems related to protein sequences,
such as protein-protein interaction \citep{chen2019multifaceted}, protein binding affinity upon mutation \citep{zhou2020mutation}, and protein structural changes \citep{senior2020improved} estimation.
These works typically use convolutional neural networks (CNNs) for automatically selecting local features, recurrent neural networks (RNN) that aim at preserving the contextualized and long-term ordering information or the combination of both CNNs and RNNs.
In contrast, fewer efforts have been made to capture the pairwise interactions of the protein drug interaction with deep learning,
which remains a non-trivial problem with the following challenges:
(i) Characterization of the proteins and drugs requires a model to effectively filter and aggregate their local features, while preserving significant contextualized and sequential information of the amino acids and drug fingerprints;
(ii) Constructing a deep neural architecture without biological insights often suffers from the interpretation issue;
(iii)
An effective mechanism is also needed to apprehend the mutual influence of protein-drug pairs in DTI prediction. Moreover,the framework needs to be scalable to large data.
Corresponding methods, including DeepDTI \citep{wen2017deep}, RFDT \citep{wang2018computational}, DeepDTA \citep{ozturk2018deepdta} and DeepConv-DTI \citep{lee2019deepconv}, employed deep neural networks on DTI prediction tasks.
Specifically, DeepDTI built by \cite{wen2017deep} used the deep belief network (DBN), with features such as the composition of amino acids, dipeptides, and tripeptides for proteins and fingerprints for drugs. RFDT by \cite{wang2018computational} employed stacked Auto-Encoder (AE) to abstract original features into a latent representation with a small dimension. With latent representation, they trained a Random Forest (RF), which performed better than previous methods.
DeepDTA \citep{ozturk2018deepdta} and DeepConv-DTI \citep{lee2019deepconv} both used CNN to extract local residue patterns to predict the binding affinity between drugs and targets. They performed convolution on various lengths of the subsequences' amino acids to capture local residue patterns of generalized protein classes.

The previous DTI prediction methods can be mainly separated into two categories: similarity-based methods and feature-based methods. Similarity-based methods assume that similar drugs or proteins may have similar interaction patterns. These methods use many different similarity measures based on fingerprint, chemical structure, sequence data and so on to identify drug-target interaction. Feature-based methods solve drug–target interaction prediction as a binary classification problem, such as  DeepConv-DTI \citep{lee2019deepconv}, use known drug-target pairs as positive sample. To combine both the similarity and feature vectors, the neural network models require a mechanism to represent both the interaction patterns and the features of drug and protein sequences.

In this paper, we introduce \model
(\textbf{D}rug-\textbf{T}arget \textbf{I}nteraction prediction with \textbf{G}raph \textbf{AT}tention networks), a deep neural network architecture that operates on graph-structured data with attention mechanism.
First, it converts the position-specific scoring matrix (PSSM) and the drug fingerprint as feature vectors for all the target proteins and drugs.
Secondly, a DTI graph is constructed based on the similarity of feature vectors. Each node represents a protein or a drug and the nodes are lined by edges representing interactions between protein-drug, protein-protein and drug-drug separately.
The graph attention network is then applied on the built graph to generate embeddings for each protein and drug, followed by a final decoder architecture to predict the interaction result.
Notably, based on the assumption that different neighbors' importance are different and the DTI graph is heterogeneous, we don't know which neighbor or domain knowledge (protein or drug) is more important for a given node in the DTI prediction task. The Graph attention (GAT) layer applies self-attention mechanism, which allows for assigning different attentional weight to neighbor nodes, enabling a leap in model capacity. Furthermore, analyzing the learned weights with attention can lead to benefits in interpreting the DTI topological structure and similarity.

Our contributions are 3-fold.
First, we provide an approach to transform the feature representations of proteins and drugs into a protein-drug interaction graph. We emphasize the need to extend the bipartite graph to a heterogeneous graph, by adding drug-drug and protein-protein similarities.
Second, we demonstrate that the attention mechanism can automatically extract the important high-level relationships among proteins and drugs by assigning different weights to each edge and drop all structural information.
Third, we provide an extensive analysis to show the better interpretability of learned attention weights for representing the topological structure and similarity.
Last, \model is highly efficient, as the operation of the self-attentional layer can be parallelized across all edges, and the computation of output features can be parallelized across all nodes.
\model significantly outperforms various state-of-the-art approaches on the DTI binary prediction task, which confirms the effectiveness of the graph attention strategy in identifying drug-target interactions.

\section{Methods}\label{sec:method}
\subsection{Preliminary}

\subsubsection{Protein feature representation}
In this work, protein sequences are represented as pseudo-position specific scoring matrix (PsePSSM) features to encode the evolution and sequential information for proteins with different lengths of sequences.
Note that this setting is consistent with previous studies~\citep{shi2019predicting}.

For a target protein sequence $p_m$ with $L$ amino acid residues, we use
the position-specific scoring matrix (PSSM) as its descriptor introduced by \cite{jones1999protein}.
The PSSM with a dimension of L $\times$ 20 can be expressed as:
\begin{equation*}
PSSM(p_m) =
\begin{pmatrix}
E_{1\rightarrow{1}} & E_{1\rightarrow{2}} & \cdots & E_{1\rightarrow{20}} \\
E_{2\rightarrow{1}} & E_{2\rightarrow{2}} & \cdots & E_{2\rightarrow{20}} \\
\vdots  & \vdots  & \ddots & \vdots  \\
E_{L\rightarrow{1}} & E_{L\rightarrow{2}} & \cdots & E_{L\rightarrow{20}}
\end{pmatrix},
\end{equation*}
where each element $E_{i\rightarrow{j}}$ in the PSSM matrix is then normalized to the interval (0, 1) as:
\begin{equation}
    \overline{E}_{i\rightarrow{j}} = \frac{1}{1+exp(E_{i\rightarrow{j}})}
\end{equation}

To make the PSSM descriptor a uniform representation despite proteins with different lengths correspond to different numbers of rows, we represent the uniformed PSSM of $p_m$ as:
\begin{equation}
    \overline{PSSM}(p_m) = [\overline{E}_{1}, \overline{E}_{2}, ..., \overline{E}_{20}]^T,
    \label{eq:uni_pssm}
\end{equation} where $T$ is the transpose operator.
Here, $\overline{E}_{j} = \frac{1}{L} \sum_{i = 1}^{L} \overline{E}_{i\rightarrow{j}}$ computes the average score of the residue in protein $p_m$ during the process of evolution, which is the mutation from amino acid type $i$ to $j$.
To retain the sequence information after Eq.~\ref{eq:uni_pssm}, the pseudo-position specific scoring matrix (PsePSSM) for protein $p_m$ is computed as:
\begin{equation}

    P_{\text{Pse}(p_m)}^{\lambda} = [\overline{E}_1, ..., \overline{E}_{20}, G_1^1, ..., G_{20}^1, ..., G_1^\lambda, ... G_{20}^\lambda]^T,
    \label{eq:psepssm}
\end{equation}
where
\begin{equation}
    G_{j}^{\lambda} = \frac{1}{L - \lambda} \sum_{i = 1}^{L - \lambda}[\overline{E}_{i \rightarrow{j}} - \overline{E}_{(i + \lambda)\rightarrow{j}} ]^2,
    (j=1,...,20;0 \leq \lambda \leq L).
\end{equation}
Here, $G_{j}^{\lambda}$ is the correlation factor of $j^{th}$ amino acid and $L$ is the continues distance along the protein sequence. Therefore, a protein sequence can be expressed as Eq.~\ref{eq:psepssm} using PsePSSM and generates a $20 + 20 \times \lambda$ dimensional feature vector.

\subsubsection{Drug feature representation}
It has been indicated in some studies that descriptors as molecular structure fingerprints can effectively represent the drug \citep{ding2017identification,yamanishi2011extracting}.
For drug molecules, we use the chemical structure of the molecular substructure fingerprints from the PubChem database \citep{kim2019pubchem}. For each drug molecule, it defines an 881- dimensional binary vector $Q$
to represent the molecular substructure, where the corresponding bits of the vector are encoded as 1s for existence of substructures and 0s for absence.
Therefore, given a drug $d_n$, its fingerprint feature is calculated as $Q(d_n) = [q1(d_n), q2(d_n), ... q881(d_n)]$
Fingerprints property is ``PUBCHEM\_CACTVS\_SUBSKEYS'' in PubChem SDF files and is Base64 encoded, which provides a textual description by the binary data.
The drug feature representations in this work are also consistent with the previous study~\citep{wang2018computational}.

\subsubsection{Problem statement}
Given a set of $N$ proteins $P = (p_1, p_2, ..., p_N)$ and a set of $M$ drugs $D = (d_1, d_2, ..., d_M)$, our goal is to predict the interaction ($I$) between $p_m$ and $d_n$ based on the the protein's PsePSSM: $P_{\text{Pse}(p_m)}^{\lambda}$ and the drug's fingerprint: $Q(d_n)$, where $p_m \in P$ and $d_n \in D$.
\begin{equation}
    I(p_m, d_n) =
\left\{
            \begin{array}{lr}
            1, \quad interaction &  \\
            0, \quad no\quad interaction &  \\
            \end{array}
\right.
\label{eq:interaction_define}
\end{equation}

\subsection{\model}
We introduce \model, a deep neural network architecture that operates on graph-structured data with attention mechanism for the feature-based DTI prediction task.
The overall learning architecture is illustrated in Figure~\ref{fig:dpigat}.

\begin{figure*}[!t]
    \centering
    \includegraphics[width=\linewidth]{images/model.pdf}

    \caption{Architecture of \model.}\label{fig:dpigat}

\end{figure*}

\model consists of the following main components:
First, given the pseudo-position specific scoring matrix (PsePSSM) of each protein and the fingerprint of each drug, a DTI graph is constructed based on the information about interaction target and similarity of local structure.
Then, the graph attention network is applied to the built graph to integrate higher-level information and generate embeddings for each protein and drug.
A final decoder architecture takes a pair of protein-drug embeddings and predicts the final interaction probability.

\subsubsection{Graph construction} \label{section:graph construction}
Given a protein set $P = (p_1, p_2, ..., p_N), p_i \in R^p$, drug set $D = (d_1, d_2, ..., d_M), d_i \in R^d$, and interaction targets set $T = (t_1, t_2, ..., t_k), t_i \in \{1,-1\}$, where $p$ is the PsePSSM feature dimension, $d$ is the fingerprint dimension, we construct a DTI-Graph  $G(V,E)$.

First, we transform the PsePSSM and the fingerprint to the same dimension, so that nodes can be easily aggregated. In our experiment, we use two weight matrices, $W_p \in R^{p \times F}$ and $W_d \in R^{d \times F}$ for transforming protein and drug separately,  where F is the feature dimension, and $ReLU$ nonlinearity to transform the feature dimension.
\begin{equation}
    p^{'}_i = ReLU(W_p \cdot p_i),  \ p^{'}_i \in R^F
    \label{eq:protein feature transform}
\end{equation}
\begin{equation}
    d^{'}_i = ReLU(W_d \cdot d_i),  \ d^{'}_i \in R^F
    \label{eq:drug feature transform}
\end{equation}
Once obtained, then we combine the new protein set $P^{'} = (p^{'}_1, p^{'}_2, ..., p^{'}_N), p^{'}_i \in R^F$ and the new drug set $D = (d^{'}_1, d^{'}_2, ..., d^{'}_M), d^{'}_i \in R^F$ to the graph node set $H = (h_1, h_2, ..., h_K), h_i \in R^F$, where each node is represented as a vector. These nodes are linked by undirected edges, $e \in \{1,-1\}$, which are generated from the drug-protein interaction, drug-drug and protein-protein similarity.

The edge set, $E$, contains two different components: \textbf{interaction edge} and \textbf{similarity edge}. The interaction edge, $e_i$, is either $1$ or $-1$, referring positive relation and negative relation of two nodes. Since the positive relation is from the ground truth interaction targets set, $T$, the initial graph of the DTI is very sparse and has an imbalanced issue. To solve the above issues, the negative samples are selected randomly from the unidentified drug–target pairs. We assume that the positive sample is a small percentage of all possible samples, so there is a low probability that real interaction will be selected as a negative sample. Actually, the proportions of positive samples detected in each dataset are $0.99\%$(Enzymes), $3.49\%$(Ion channels), $2.99\%$(GPCR) and $6.40\%$(Nuclear receptors). In the experiment, we choose a negative sample with the same number of positive samples. However, the initial interaction edges are only between the drug-protein pairs, which construct a bipartite graph that limit the information flow.
In order to aggregate more information, we transform the bipartite graph to a heterogeneous graph, by adding drug-drug and protein-protein similarities. Similarity edge, $e_s$, is based on the DTI bipartite graph and its common neighbor information. If the number of two nodes' common positive neighbor or negative neighbor is greater than a threshold, the two nodes will be connected by $1$, which means they are similar.
\begin{equation}
    e_s(i,j) =
\left\{
            \begin{array}{lr}
            1, \quad if \ com\_n(h_i, h_j) \textgreater \theta &  \\
            0, \quad if \ com\_n(h_i, h_j) \leq \theta &  \\
            \end{array}
\right.
\label{eq:similarity}
\end{equation}
where $com\_n$ is the function to compute the common neighbor of two nodes and $\theta$ is the threshold.
In our experiment, we use adjacent matrix $A \in R^{K \times K}$ to represent $E$, where $K$ is the number of node.

The interaction edge connects the protein-drug domain, similarity edge connects protein-protein domain and drug-drug domain separately, that allows information to flow across the entire DTI graph. What's more, positive edge (e.g. 1) means to pull two nodes closer and negative (e.g. -1) means to push them apart.

\subsubsection{Graph attention network}
Our approach uses Graph Attention Network (GAT)\citep{velikovi2017graph} to adaptively learn weights for each edge and represent each node by message passing.

The input of GAT is a set of node feature $H = (h_1, h_2, ..., h_K), h_i \in R^F$ and DTI adjacent matrix $A$. $H$ contains $H_p = (h_{p_1}, h_{p_2}, ..., h_{p_m})$ and $H_d = (h_{d_1}, h_{d_2}, ..., h_{d_n})$ and $A$ is generated by~\ref{section:graph construction}.

The output is a new set of node feature, $H^{'} = (h^{'}_1, h^{'}_2, ..., h^{'}_K), h^{'}_i \in R^{F^{'}}$.

In order to transform input features to higher level features, we apply a weight matrix, $W \in R^{F^{'}\times F}$, to each node.
\begin{equation}
    h^{'}_i = \sigma(W \cdot h_i),  \ h^{'}_i \in R^{F^{'}}
    \label{eq:node feature transform}
\end{equation}

Then we perform a self-attention mechanism on node pair, $a:R^{F^{'}} \times R^{F^{'}} \rightarrow R$, to compute attention weight, which indicates the importance of $n_j$ to $n_i$, $n_j \in  N_i$, where $N_i$ is the neighborhood of $n_i$ and  itself in DTI graph.
\begin{equation}
    w_{ij} = a(h^{'}_i, h^{'}_j), \ w_{ij} \in R^1
    \label{eq:attention1}
\end{equation}
In GAT layer, the attention mechanism $a$ is a single-layer neural network, parametrized by a weight matrix $\widetilde{a} \in R^{2F^{'}}$. Then the LeakyReLU nonlinearity is applied.
\begin{equation}
    w_{ij} = LeakyReLU(\widetilde{a}^{T}[h^{'}_i|| h^{'}_j])
    \label{eq:attention2}
\end{equation}
where $.^T$ is transposition and $||$ is the concatenation operation.

In the general formulation, attention mechanism allows every node to attend on every other node, dropping all structural information. To add graph structure information, we perform mask attention according to the DTI adjacent matrix $A$, which enables only the neighbor nodes to be attended. Then we normalize the attention weight across all choices of $j$ using the softmax function to make it comparable of different nodes.
\begin{equation}
    \alpha_{ij} = Softmax_j(w_{ij}) = \frac{exp(w_{ij})}{\sum_{k \in N_i}exp(w_{ik})}
    \label{eq:softmax}
\end{equation}
where $N_i$ is the set of $h_i$'s neighborhood and itself.

After obtaining the normalized attention score, we use message passing to compute a linear combination of the node features and output the final aggregated features of each node.
\begin{equation}
    h_i^{'} = \sigma (\sum_{j \in N_i}\alpha_{ij}Wh_j)
    \label{eq:message pass}
\end{equation}
where $\sigma$ is a nonlinearity, $H^{'}$ is final aggregated features.

\subsubsection{Final decoder}
The final decoder is a sample neural network, parametrized by a weight matrix $W \in R^{2F^{'}}$. It takes pairs of drug-protein embeddings, generated by GAT layer(e.g. $h_i$ and $h_j$), as input. Then the two node vector do an element-wise multiplication, $p \odot d \rightarrow v, \ v,p,d \in R^F$. Finally, through a layer of neural network $v^F \rightarrow R^1$ and a Sigmoid activate function, produce a probability score indicating whether they interact:
\begin{equation}
    s_{ij} = Sigmoid(ReLU(W(h^{'}_i \odot h^{'}_j)))
    \label{eq:decoder}
\end{equation}

\subsubsection{Loss function}
For training \model, we assign a binary class label ${0,1}$ (interact or not) to each identified drug-protein pair. We assign positive labels to interacted drug-protein pairs and negative labels to non-interacted drug-protein pairs. The output is the interaction probability.
The main learning objective is to minimize the following binary cross entropy loss (BCELoss) between the target $X = (x_1, x_2, ..., x_N), x_i \in \{0,1\}$ and the output $Y = (y_1, y_2, ..., y_N), y_i \in (0,1)$. The loss can be described as:
\begin{equation}
    l(x,y) =  \{l_1, l_2, ..., l_N\}^T
    \label{eq:BCELoss1}
\end{equation}
\begin{equation}
    l_n = -w_n[y_n\cdot log \ x_n + (1-y_n)\cdot log \ (1-x_n)]
    \label{eq:BCELoss2}
\end{equation}

\subsubsection{Implementation details}
In this experiment, we use PyTorch \citep{NEURIPS2019_9015} for efficient GPU-based implementation and PyTorch geometric \citep{Fey/Lenssen/2019} for a sparse Graph Attention Layer.

The \model contains the feature (PsePSSM, Fingerprint) encoder, the GAT embedding module and a final interaction decoder. Feature encoder is implemented as two MLP layers for protein and drug respectively. The input dimensions are 220 (PsePSSM) and 881 (fingerprint) and the output dimensions are both 256. The GAT embedding module consists of a Multi-head GAT layer, based on PyTorch geometric \citep{Fey/Lenssen/2019}. The hidden dimension and output dimension both are set to 256 for all datasets. In building the adjacency matrix, the setting of the common neighbor threshold is different in different datasets, it is set to 1 in Nuclear Receptor, 3 in GPCR, Ion Channel and Enzyme dataset. The final interaction decoder includes three MLP layers, which input dimension $2 \times 256 = 512$ and the hidden dimension is also 256. The output is a scalar, then be computed by a sigmoid function to get the final interaction probability.

In our paper, the \model are trained end-to-end using gradient descent based on Adam optimizer \citep{kingma2014adam}. We perform batch gradient descent using the full dataset for every training iteration, which is a viable option as long as datasets fit in memory. Base on the implementation of PyTorch Geometric \citep{Fey/Lenssen/2019}, we use a sparse representation for A. The memory requirement is $\mathcal{O} (|\varepsilon|)$, which is linear in the number of edges. The learning rate $\alpha$ is 0.0005 and the weight decay is $5e-4$. Stochasticity in the training process is by dropout \citep{JMLR:v15:srivastava14a}, which is set to 0.3. The model is trained until converging for each fold of the cross-validation,
for which we set the epoch number to be 4000, 8000, 10000, 15000 for Nuclear Receptor, GPCR, Ion Channel and Enzyme, respectively.

\section{Results}\label{sec:result}
\begin{table}[]
\caption{Number of drugs, target proteins, and their interactions for 5 datasets used for cross-validation.}
\centering
\begin{tabular}{|c|c|c|c|c|c|}
\hline
             & Enzyme & IC   & GPCR & NR & Drugbank\_approved      \\ \hline
Drugs        & 445    & 210  & 223  & 54 & 1555                    \\ \hline
Targets      & 663    & 204  & 95   & 26 & 1591                    \\ \hline
Interactions & 2925   & 1476 & 635  & 90 & 5831                    \\ \hline
\end{tabular}
\label{tab:data}
\end{table}

\begin{figure*}
        \centering
        \begin{subfigure}[b]{0.495\textwidth}
            \centering
            \includegraphics[width=\textwidth]{images/ez.pdf}
            \caption[Enzyme]%
            {{\small Enzyme}}
            \label{fig:mean and std of net14}
        \end{subfigure}
        \hfill
        \begin{subfigure}[b]{0.495\textwidth}
            \centering
            \includegraphics[width=\textwidth]{images/ic.pdf}
            \caption[]%
            {{\small Ion Channel}}
            \label{fig:mean and std of net24}
        \end{subfigure}
        \vskip\baselineskip
        \begin{subfigure}[b]{0.485\textwidth}
            \centering
            \includegraphics[width=\textwidth]{images/gpcr.pdf}
            \caption[]%
            {{\small GPCR}}
            \label{fig:mean and std of net34}
        \end{subfigure}
        \quad
        \begin{subfigure}[b]{0.485\textwidth}
            \centering
            \includegraphics[width=\textwidth]{images/nr.pdf}
            \caption[]%
            {{\small Nuclear Receptor}}
            \label{fig:mean and std of net44}
        \end{subfigure}
        \caption[]
        {Evaluation of DTI prediction on the four datasets based on 5-fold cross-validation under 6 evaluation metrics. We report the mean and standard deviation for the test sets for all methods.}
        \label{fig:main_eval}
    \end{figure*}

\begin{table}[]
\caption{Evaluation of DTI prediction on the drugbank\_approved dataset based on 5-fold cross-validation under 6 evaluation metrics. We report the mean and standard deviation for the test sets.}
\centering
\begin{tabular}{|l|l|l|l|l|l|l|l|}
\hline
 & & Acc          & Sen  & Prec    & Spec    & Mcc     & Auc\\\hline
SAE+RF*      & mean & 82.60 & 82.75 & 82.39 & 82.54 & 66.05 & 88.31 \\
             & std  & 1.24  & 1.35  & 1.16  & 1.07  & 1.29  & 0.31\\\hline
DeepDTA*     & mean & 82.13 & \textbf{83.61} & 81.29 & 80.65 & 64.34 & 87.36 \\
             & std  & 0.96  & 1.76  & 2.15  & 2.94  & 1.92  & 0.77  \\\hline
DeepConv-DTI & mean & 84.54 & 83.44 & 85.31 & 85.65 & 69.02 & 91.73 \\
             & std  & 0.65  & 0.72  & 0.83  & 0.98  & 1.21  & 0.34  \\\hline
LRF*         & mean & 86.11 & 83.45 & 88.18 & 88.78 & 72.38 & 92.20 \\
             & std  & 0.45  & 2.40  & 1.13  & 1.56  & 0.77  & 0.75  \\\hline
\model      & mean & \textbf{87.67} & 81.12 & \textbf{93.34} & \textbf{94.21} & \textbf{76.00} & \textbf{92.30} \\
             & std  & 0.95  & 2.24  & 0.60  & 0.65  & 1.69  & 0.83  \\\hline

\end{tabular}

\label{tab:drugbank_eval}
\end{table}

\begin{figure}[!t]
    \centering

        \includegraphics[width=\linewidth]{images/ez_cn.pdf}

        \label{fig:hyper1}

    \caption{Performance evaluation on using different hyperparameters for PDI prediction on the Enzyme dataset. The accuracy is reported.}\label{fig:hyper}

\end{figure}

\begin{table}[]
\caption{Evaluation of DTI predictions by different methods on the independent study set.}
\centering
\begin{tabular}{|l|l|l|l|l|l|l|}
\hline
             & Acc   & Sen   & Prec  & Spec  & Mcc   & Auc   \\ \hline
SAE-RF*       & 64.20 & 55.30 & 69.30 & 77.30 & 32.10 & 71.20 \\ \hline
DeepDTA*      & 64.82 & 67.33 & 59.80 & 62.75 & 29.95 & 73.04 \\ \hline
DeepConv-DTI & 68.60 & 58.50 & 73.20 & 78.60 & 37.90 & 74.30 \\ \hline
LRF*          & 71.38 & 45.89 & 73.19 & \textbf{92.37} & 44.02 & 75.70 \\ \hline
\model      & \textbf{72.90} & \textbf{67.36} & \textbf{75.54} & 78.39 & \textbf{46.04} & \textbf{78.42} \\ \hline

\end{tabular}

\label{tab:indep_eval}
\end{table}

\begin{figure*}
        \centering
        \begin{subfigure}[b]{0.495\textwidth}
            \centering
            \includegraphics[width=\textwidth]{images/tsne_ez_protein.pdf}
            \caption[Enzyme]%
            {{Protein nodes grouped by top connected drugs}}
            \label{fig:tsne_protein}
        \end{subfigure}
        \hfill
        \begin{subfigure}[b]{0.495\textwidth}
            \centering
            \includegraphics[width=\textwidth]{images/tsne_ez_drug.pdf}
            \caption[]%
            {{Drug nodes grouped by top connected proteins}}
            \label{fig:tsne_drug}
        \end{subfigure}
        \caption[t-SNE visualization of the node embeddings generated by \model on the Enzyme dataset.]
        {t-SNE visualization of the protein (a) and drug (b) nodes embeddings generated by \model on the Enzyme dataset.}
        \label{fig:tsne_all}
    \end{figure*}

We present the experimental evaluation of the proposed framework on the binary DTI prediction task. The experiments are conducted on the following datasets.

\subsection{Datasets}
We use the interaction data between drugs and target proteins collected by \cite{yamanishi2008prediction}, available at http://web.kuicr.kyoto-u.ac.jp/supp/yoshi/drugtarget. The dataset is collected from various databases like SuperTarget \citep{gunther2007supertarget}, DrugBank \citep{wishart2008drugbank}, KEGG BRITE \citep{kanehisa2006genomics}, and BRENDA \citep{schomburg2004brenda}.
This dataset includes four main subsets: enzymes, ion channels (IC), G-protein-coupled receptors (GPCR) and nuclear receptors (NR). The statics of the interaction are shown in Table~\ref{tab:data}. We obtained the protein sequence information from UniProt \citep{uniprot2019uniprot}, and the drug fingerprint data from PubChem \citep{kim2019pubchem}. Note that there are a few proteins and drugs that have been removed in the newer version of databases, so we did not include them in our training.

In addition to the above four benchmark datasets, we also use a dataset called drugbank\_approved \citep{ba2016daspfind}, which contains all FDA-approved drugs and their corresponding protein targets in the DrugBank database \citep{wishart2008drugbank}. After removing the non-existing proteins and drugs, the processed dataset contains 1555 drugs and 1591 targets.

\subsection{Evaluation protocol}
Following the settings in previous works \citep{wang2018rfdt, wang2018computational, shi2019predicting}, we conduct 5-fold cross-validation (CV) on the five datasets of the enzyme, ion channel, GPCR, nuclear receptor and the drugbank\_approved.
Under the 5-fold CV setting, the data is equally divided into 5 non-overlapping subsets, and each subset has a chance to train and to test the model so as to ensure an unbiased evaluation. We repeat the prediction model with 10 trials and record the mean and standard deviation of results.
We aggregate fix metrics on the test cases of each fold, i.e. the overall \emph{accuracy (ACC)}, \emph{precision (PR)}, \emph{sensitivity (SE)}, \emph{specificity (SP)}, \emph{Matthews correlation coefficient (MCC)} and \emph{Area under the curve (AUC)}.
All these metrics are preferred to be higher to indicate better performance.

\subsection{Baseline methods}
We compare \model with the following two groups of baselines:
\begin{enumerate}
    \item Classical statistical learning models:
    SVM\citep{cao2012large}, Silico DVM\citep{li2017silico}, and LRF\citep{shi2019predicting}.
    \item Deep learning models:
    SAE+RF \citep{wang2018computational}, DeepDTA \cite{ozturk2018deepdta}, and DeepConv-DTI \cite{lee2019deepconv}.
\end{enumerate}

Note that LRF has been modified to perform data augmentation on the training set only, to ensure validation and training samples are fully separated.
Also, note that DeepDTA is designed for Drug-Target Affinity Regression instead. We adapt it by using a threshold of 0.5 to do a binary classification in order to obtain the interaction.

\subsection{Experimental results}
\subsubsection{Comparison with other methods}
For the prediction of drug-target interactions, many prediction methods have been proposed. Figure~\ref{fig:main_eval} details the comparison between \model with other baseline methods on enzymes, ion channels, GPCRs, nuclear receptors and Table~\ref{tab:drugbank_eval} shows the results on the larger drugbank\_approved datasets.

As shown in Figure~\ref{fig:main_eval}, in general, the statistical baselines perform better than the deep learning methods. This shows that deep learning methods are not able to obtain enough information to generalize.
The best performing model among them is Silico DVM, a statistical learning model that trains a discriminative vector machine classifier with a local binary pattern generated from PSSM.
Our model \model can generalize using additional information from the graph, thus achieving better performance than the baseline models. \model outperforms Silico DVM on Enzyme, Ion channel and GPCR by 0.014, 0.054, 0.011 on MCC, but Silico DVM outperforms \model on nuclear receptor by 0.018 on MCC. This attribute to the fact that \model is able to extract more generalizable information from the graph of interaction, especially on a large, dense graph.

Specifically, we can see that for the dataset of the enzyme, the accuracy rate of \model reaches 93.75\%, which is higher than other prediction methods. In addition, the AUC of the prediction model reaches 96.32\%, comparable with the best performing model LRF of 96.64\%.
In the ion channel dataset, the accuracy of prediction in this paper is 94.38\%, which is also higher than other prediction methods. Our AUC of 96.51\%is the highest among other baselines. Similarly in the GPCR dataset, \model outperforms all baselines in accuracy, precision, specificity, MCC and AUC. As for the nuclear receptor dataset, our prediction method accuracy rate is 91.11\% and our AUC is 90.49\%, slightly lower than the best performing Silico DVM. This is mainly due to the fact that the nuclear receptor dataset only contains a very limited number of samples for training and all the deep learning based methods suffers from it. However, we still drastically outperform other neural network based methods including SAE+RF, DeepDTA and DeepConv-DTI.

The results on the drugbank\_approved dataset are shown in Table~\ref{tab:drugbank_eval}. With this larger dataset, we focus on comparing our model with four other methods including LRF, DeepConv-DTI, DeepDTA and SAE+RF.
We can see from the table that the performances for all the methods drop by 5\% to 10\% comparing to the previous cross-validation results. This is because the larger drugbank\_approved dataset contains more complicated and new protein-drug interactions, which make the learning process harder.
However, the accuracy rate of \model reaches 87.67\% and the AUC of \model reaches 92.30\%, which are still higher than all the baseline methods and relatively robust in identifying new interactions.

\subsubsection{Independent Study}

To show the generalizability of \model, we evaluate the performance of different methods on an independent dataset.
We construct the independent dataset using the five datasets we discussed in Table~\ref{tab:data}. This independent study dataset consists of a training set of enzyme, IC, GPCR and NR. The testing set comprises of the drugbank\_approve dataset, with the protein-drug pairs that appear in the training set removed. Since the proteins and drugs have different names in different sources, we perform the matching and removal by directly comparing the PsePSSM and the fingerprints.
After removing the similar sequences, we obtain the independent dataset with 1883 drugs and 2048 proteins. Among them, 5113 interactions are used for training and the rest 5315 interactions are used for independent testing.
Using this dataset, we can examine the ability of \model to generalize the prediction to a larger protein network from a subset of networks.

We compare \model with four baselines including SAE+RF, DeepDTA, DeepConv-DTI, and LRF.
As shown in Table~\ref{tab:indep_eval}, \model outperforms other methods by achieving the highest accuracy of 72.9\%, sensitivity of 67.36\%, MCC of 46.04\% and AUC of 78.42\%. Notice that LRF achieves high specificity with the cost of getting very low sensitivity, meaning that there are many false negatives (predicting many interactions to be non-interactions), which is not optimal for the DTI prediction task.
The comparative results of \model on the independent set demonstrate that \model can generalize better than other methods to drugs and targets that share low sequence similarity with the training set.

\subsubsection{Parameter analysis}
In our work, the main hyperparameter is the common neighbor threshold, $\theta$. It is the key factor of similarity edge, $e_s$, for which links the protein and drug domain.

To further discuss the influence of common neighbors, we choose the Enzyme dataset, which has the most nodes. The result of different settings is illustrated by Figure~\ref{fig:hyper}. $\theta = 1$ means once two nodes have common neighbors, they will be linked. In our observation, there are some clustered graphs, many different drug nodes are linked to the same protein node. Once $\theta = 1$, these drug nodes will be joined into a fully connected graph, the DTI graph structure and similarity information will be destroyed. That's why the performance of $\theta = 1$ is lower than the others.  As the $\theta$ increases, the DTI graph includes more and more useful similarity edges, leads to higher accuracy. What's more, if we delete all the $e_s$, means $\theta$ is infinite, no similarity edge, the score is only 87.01, much lower than others, that proves it is very important.

\subsubsection{t-SNE visualization of the node embeddings by \model}
The effectiveness of the learned feature representations for each node may also be investigated qualitatively. For this purpose, we provide a visualization of the t-SNE \citep{maaten2008visualizing} transformed feature representations extracted by the output layer of the GAT model on the Enzyme dataset.
The t-SNE visualizations are generated for both the view of protein nodes and the view of drug nodes as shown in Figure~\ref{fig:tsne_protein} and Figure~\ref{fig:tsne_drug}, respectively.

To show the 2D-projection of protein features, we choose the top 4 drugs as labels ordered by their number of connections to all proteins. We then filter out proteins that are not connecting to any of the selected drugs or ambiguously connecting to multiple selected drugs. The feature vectors of the rest proteins outputted by GAT are then transformed into 2D space by t-SNE.
Similarly, we choose the top 4 proteins as labels and conduct the same filtering process on the drugs to visualize the drug node features.

As shown in Figure~\ref{fig:tsne_protein}, each dot represents a protein and each color represents the drug that the protein is interacting with. Most proteins except the two blue ones are forming discernible clusters. Note that these clusters correspond to the labels of the common interacting drugs. Similarly in Figure~\ref{fig:tsne_drug}, each dot represent a drug and each color represent the protein that the drug is interacting with. We can also see four clusters are formed based on the learned features of each drug. These two different views of visualizations in 2D space by t-SNE demonstrate that our learned attended vector can capture the important interactions while preserving the properties of each node.

To demonstrate the biological interpretability of the learned features, in addition to the internal distance within each cluster, we also examine the distance among different clusters. The distance also correlates with the pathway of the labeled proteins. For example, in Figure~\ref{fig:tsne_drug}, the blue hsa:759 cluster has only metabolic pathways, and the green hsa:43 and red hsa:4129 clusters both participate in metabolism and other functions. These 3 clusters share the metabolic pathway.
The purple one, hsa:1636 instead has pathways in the Renin system. Therefore, hsa:759 is closer to hsa:43 and hsa:4129 comparing to the distance between hsa:759 and hsa:1636.

\subsubsection{Attention weight analysis}

\begin{figure}[!t]
    \centering
    \includegraphics[width=\columnwidth]{images/att_out.png}

    \caption{Attention graph of \model on NUC.}
    \label{fig:att_val}

\end{figure}

To further illustrate the effectiveness of the graph attention approach, we also compute an attention graph from the training, shown in Figure~\ref{fig:att_val}. We generate this figure using the attention value of the edges trained from the Nuclear Receptor dataset. From this graph, the proteins and drugs are able to form groups based on their similarities in the functions. We focus on analyzing two main groups.

First, we analyze the large group around proteins hsa:2099  (estrogen receptor alpha), hsa:2100 (estrogen receptor beta), and hsa:5241 (progesterone receptor), shown in the purple circle in the figure. The functions of these proteins are very similar. Therefore they are shown as connected and interact with many drugs. The drugs to the right of these proteins consist of a few progesterone receptor agonists (D00182, D00951, D01294, D00066, D00950, D00954, D01217, D02367). For some of the drugs, they only act as progesterone receptors, such as D00951 and D01217, and therefore have a large weight connecting to the progesterone receptor. On the other hand, drugs like Progesterone (D00066) also activate the estrogen pathway and therefore are connected to the estrogen receptor pathway.
Because of there similarity, these drugs also give attention to other drugs with a similar function, lowering the individual attention weight in the cluster.
On the smaller cluster on the left of the estrogen receptors, we can also see a similar cluster of estrogen receptor agonists (D00554, D00105, D00067, D00312, D00577, D00898) and estrogen receptor antagonists (D01161). These proteins do not activate progesterone receptors, with a few of them only activate the alpha receptor (D00962, D02217).

In addition, we analyze the smaller cluster on the top right of the graph, shown in the green box. This cluster is disjoint from the rest of the graph, centered around Vitamin D Receptor (hsa:7421). There are 5 drugs around this receptor, which are the following:

\begin{itemize}
    \item Vitamin D2 (Ergocalciferol, D00187)
    \item activated Vitamin D2 (Paricalcitol, D00930)
    \item Vitamin D3 (Cholecalciferol, D00188)
    \item activated Vitamin D3 (Calcitriol, D00129)
    \item a synthetic vitamin D analog (Dihydrotachysterol, D00299)
\end{itemize}

For these drugs, we can see that the receptor has placed more attention weights on the activated versions of the vitamin D (D00930 and D00129). And for these two drug, there are less attention on the receptor. This is because they pay more attention on itself, lowering their attention weight to the receptor. However, our graph is not able to predict the connection between the vitamins and their activated version due to insufficient information.

We can see that \model is able to learn the importance of each interaction from the interaction graph of the dataset. This allows the model to effectively predict the interaction between the drug and the protein.

\section{Conclusion}
In this paper, we introduce a novel and comprehensive learning framework to predict the drug-target interactions.
Our proposed framework, \model, operates on the graph-structured data with the attention mechanism based on the deep neural network architecture.
We provide an approach to transform the feature representations of proteins and drugs into a protein-drug interaction graph. We also emphasize the importance of adding the drug-drug and protein-protein similarities to the graph.
With the attention mechanism, \model can automatically extract the important high-level relationships by assigning different weights to each edge.
Extensive experiments of both cross-validation and independent tests conducted on the five datasets demonstrate the promising performance of \model.
Moreover, we provide case studies to show the interpretability of \model.
As a future direction, we plan to explore other external knowledge including gene ontology and PPI networks, which can be integrated into \model. To further improve the performance, we seek to incorporate the knowledge graph representation learning framework \citep{hao2019universal}.

\bibliographystyle{natbib}

Quantitative Methods (q-bio.QM); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)


Evolution of electronic structure of Ru-doped single-crystal iridiates, Sr$_2$Ir$_{1-x}$Ru$_x$O$_4$

Abstract: We investigated Ru-doped single-crystal 5$d$ iridiates, Sr$_2$Ir$_{1-x}$Ru$_x$O$_{4}$, at three different doping concentrations ($x =$ 0.01, 0.07 and 0.10) using optical spectroscopy. The undoped pristine compound (Sr$_2$IrO$_{4}$) is known as a novel $J_{eff}$ = 1/2 Mott insulator. Remarkably, the optical conductivity spectra of all three samples exhibited the insulating behavior, although we observed weak Drude components in the optical conductivity spectra down to the lowest temperature of 30 K. The charge-carrier densities of the Ru-doped iridiates estimated from the Drude components are significantly smaller than the expected values estimated from the nominal Ru-doping concentrations. Herein, we provide temperature- and doping-dependent electronic structure evolution of Ru-doped iridiates. We expect that our results will be useful for understanding the intriguing Ru-doping-dependent properties of 5$d$ iridiate Sr$_2$IrO$_{4}$.

\section{Introduction}
A 5$d$ ternary compound, Sr$_2$IrO$_{4}$, was discovered many years ago\cite{randall:1957}. Since then, 5$d$ transition metal (iridium) oxides have been extensively investigated\cite{cava:1994,moon:2008,kim:2008,moon:2009,kim:2009a,lee:2012,ye:2013,sohn:2014,clancy:2014,cao:2016}. Because the 5$d$ transition metal oxides exhibit a significantly larger bandwidth than that of 3$d$ metal oxides, the Coulomb interaction is reduced and the spin-orbit coupling enhanced. These differences may spawn interesting new electronic ground states of 5$d$ transition metal oxides because of the three competing energy scales of bandwidth ($W$), Coulomb repulsion ($U$), and spin-orbit coupling ($E_{SO}$). Appropriate doping can be used to control these electronic tuning parameters\cite{lee:2012}. By replacing 5$d$ Ir atom with 4$d$ Ru, the band-filling (hole doping), $E_{SO}$, and the ratio of the Coulomb repulsion to the bandwidth ($U/W$) through structural distortion ($W$) and orbit extension difference ($U$) can be tuned\cite{subramanian:1994,lee:2012}. An optical spectroscopic study on Ru-doped Sr$_2$IrO$_4$ (Sr-214) thin films at room temperature was performed and it provided interesting doping-dependent electronic evolutions\cite{lee:2012}. In a recent optical study on La-doped Sr-214 ((Sr$_{1-x}$La$x$)$_2$IrO$_4$) single crystals, pesudogap effects were observed below the onset temperature of short-range antiferromagnetic order and the anomalous charge dynamics were attributed to the electronic correlations\cite{seo:2017}. In another optical study on La-doped Sr-214 single crystals, a soft collective mode and a Drude mode were observed simultaneously in over 10\% La-doped Sr-214 and part of its Fermi surface was ungapped at low temperatures\cite{wang:2018}. Other studies have been performed on Rh-doped Sr-214\cite{clancy:2014,cao:2016}. One may expect that these Rh-doped Sr-214 systems are isovalently doped. However, studies revealed that Rh-doped Sr-214 became hole-doped owing to the introduction of Rh$^{3+}$/Ir$^{5+}$. An intriguing question is whether doped Sr-214 can be used as a platform for superconductivity. A study on angle-resolved photoemission spectroscopy observed a gap with the $d$-wave symmetry in electron-doped Sr-214\cite{torre:2015,kim:2016}. However, superconductivity has not been clearly observed in doped Sr-214. A Raman study on Ru-doped Sr-214 systems revealed that a small percentage of Ru-doping had a strong effect on spin excitations and lattice dynamics\cite{glamazda:2014}. An optical spectroscopy study on Ru-doped Sr-214 single crystals may provide interesting new information, particularly on Ru-doping-dependent evolutions of charge-carrier density and electronic structure.

In this study, we investigated Ru-doped Sr-214 single crystals, Sr$_2$Ir$_{1-x}$Ru$_x$O$_{4}$ ($x =$ 0.01, 0.07, and 0.10), using optical spectroscopy. Optical spectroscopy is a suitable experimental technique for studying the charge-carrier dynamics, electronic ground state, and electronic structures of a particular material system. We expected that the hole-doping level of Sr-214 could be controlled by Ru-doping because the 4$d$ Ru atom has one less electron than the 5$d$ Ir atom. However, our results indicate that the doping effect is negligibly small in terms of charge-carrier density. The charge-carrier densities obtained from measured optical spectra were significantly smaller than those estimated using the energy dispersive spectrometry (EDS) technique. However, we obtained the Ru-doping-dependent electronic band structure evolution of Sr-214 from the results of Drude-Lorentz model analysis\cite{wooten,tanner:2019} and discussed about the electronic band structure evolution.

\section{Experiments}

\begin{figure}[!htbp]
  \vspace*{-0.3 cm}%
  \centerline{\includegraphics[width=4.5 in]{Figure0new.pdf}}%
  \vspace*{-0.7 cm}%
\caption{(Color online) Measured DC resistivity data of Sr$_2$Ir$_{1-x}$Ru$_x$O$_4$ in three different Ru-doping levels of $x$ = 0.01, 0.07, and 0.10. The Arrhenius plots are shown in the inset. The dashed lines are linear-fits. }
 \label{fig0}
\end{figure}

High-quality single crystals of Sr$_2$Ir$_{1-x}$Ru$_x$O$_4$ ($x$ = 0.01 , 0.07, and 0.10) were grown using a self-flux method\cite{kim:2009a}. To verify Ru concentrations of Sr$_2$Ir$_{1-x}$Ru$_x$O$_4$ crystals, the energy dispersive spectrometry (EDS) technique was employed using a JEOL-7800F (JEOL Ltd.) field emission scanning electron microscope equipped with an energy dispersive x-ray spectrometer working at 5.0 kV. The estimated Ru concentrations were 0.018$\pm$0.002, 0.070$\pm$0.005, and 0.104$\pm$0.011, respectively, for the samples with $x =$ 0.01, 0.07 and 0.10. The sample with $x =$ 0.01 shows a larger concentration than the nominal one. DC resistivity data were measured using a conventional 4-point probe technique and a physical property measurement system (PPMS, Quantum Design, Inc.). The measured DC resistivity data of the all three samples are shown in Fig. \ref{fig0}, which interestingly exhibit insulating temperature-dependent behavior. In the inset, we show the Arrhenius plots of the DC resistivity data of the three samples in a temperature range from 300 to 200 K. From the Arrhenius plot, we estimated the activation energy $E_a$. The estimated $E_a$ are 55, 50, and 37 meV for the samples with $x =$ 0.01, 0.07, and 0.10, respectively. The prepared single crystals were considerably small ($<$ 1 mm$^2$ area) and brittle. The size of the sample became smaller as the doping increased. However, because the surfaces of all samples were flat and bright, we could obtain reliable spectra. Because the size of Ru ion is similar to that of Ir ion, the unit cell volume with respect to Ru-doping remains within 0.5\% up to 10\% Ru-doping concentrations \cite{cava:1994}. Therefore, all three samples had the same crystal symmetry: a tetragonal symmetry (space group $I4_1$/a)\cite{torchinsky:2015}.

\begin{figure}[!htbp]
  \vspace*{-0.3 cm}%
  \centerline{\includegraphics[width=4.5 in]{Figure1newi.pdf}}%
  \vspace*{-0.7 cm}%
\caption{(Color online) Measured reflectance spectra of Sr$_2$Ir$_{1-x}$Ru$_x$O$_4$ in three different Ru-doping levels at various selected temperatures: (a) $x$ = 0.01, (b) $x$ = 0.07, and  (c) $x$ = 0.10.}
 \label{fig1}
\end{figure}

We measured near-normal reflectance spectra in the spectral range from 80 to 13,000 cm$^{-1}$ at several selected temperatures from 30 to 300 K using a commercial Fourier-transform infrared spectrometer (Bruker Vertex 80v) and a continuous liquid helium flow cryostat. To obtain accurate reflectance spectra in a wide spectral range up to 20,000 cm$^{-1}$, we employed an {\it in-situ} gold/aluminum evaporation technique\cite{homes:1993}. We obtained reflectance spectra of the sample with $x=$ 0.01, which was the largest sample, from 8000 to 20,000 cm$^{-1}$ at the selected temperatures, and used the measured reflectance spectra to extend the data up to 20,000 cm$^{-1}$ for all three samples. We note that, for samples with $x =$ 0.07 and 0.10, the extensions to 20,000 cm$^{-1}$ were approximates; we used them just for the Kramers-Kronig analysis. The measured reflectance spectra of the three samples at various selected temperatures are shown in Fig. \ref{fig1}. Because the size of the sample became smaller with increasing the doping the low-frequency cutoff of the measured spectra was $\sim$150 cm$^{-1}$ for the highest doped (10\%) sample due to the diffraction limit. As we expect from the DC resistivity data, all three samples exhibited insulating temperature-dependent behavior; the reflectance at the low-frequency region below 2000 cm$^{-1}$ was enhanced as the temperature increased. The overall reflectance was enhanced as the doping increased. However, the doping-dependent enhancement of reflectance at low-frequency region is not very dramatic, comparing to that of 3$d$ transition metal oxides such as cuprates\cite{uchida:1991}. We used the well-developed Kramers-Kronig analysis to obtain the phase of the reflection coefficient from the measured reflectance. For performing the Kramers-Kronig analysis, the measured spectrum should be extrapolated to zero and infinity \cite{wooten}. For the extrapolation to zero, we employed the Hagen-Rubens relation, i.e., $1-R(\omega) = \sqrt{2\omega \rho/\pi}$, using the measured DC resistivity data, $\rho$ (see Fig. \ref{fig0}). For the extrapolation to infinity, we used $R(\omega) \propto \omega^{-1}$ from 20,000 to $10^{6}$ cm$^{-1}$ and above $10^{6}$ cm$^{-1}$ we assumed the free electron behavior, i.e., $R(\omega) \propto \omega^{-4}$. We obtained the complex optical conductivity using the Fresnel formula and well-known relationships between the optical constants\cite{wooten,tanner:2019}.

\section{Results and discussion}

\begin{figure}[!htbp]
  \vspace*{-0.3 cm}%
  \centerline{\includegraphics[width=4.5 in]{Figure2newj.pdf}}%
  \vspace*{-0.7 cm}%
\caption{(Color online) Optical conductivity spectra of Sr$_2$Ir$_{1-x}$Ru$_x$O$_4$ in three different Ru-doping levels at various selected temperatures: (a) $x$ = 0.01, (b) $x$ = 0.07, and (c) $x$ = 0.10.}
 \label{fig2}
\end{figure}

Fig. \ref{fig2} shows the optical conductivity spectra of our three samples (Sr$_2$Ir$_{1-x}$Ru$_{x}$O$_4$: $x =$ 0.01, 0.07, and 0.10) at various selected temperatures from 30 to 300 K. The overall level of the optical conductivity in the high frequency region above 10,000 cm$^{-1}$ is slightly higher than that of undoped Sr-214 obtained using an ellipsometry\cite{sohn:2014}. It is worth to be noted that the optical conductivity in the high frequency region can be influenced by extrapolations to infinity for the Kramers-Kronig analysis. Below 10,000 cm$^{-1}$, we observed two prominent interband transitions, which were previously observed in undoped Sr-214\cite{kim:2008}. We also observed another interband transition above 15,000 cm$^{-1}$, which has been observed and assigned as a $pd$-transition\cite{sohn:2014}. We adopted the previous naming of the two prominent modes. We denoted them as $\alpha$ (near 4000 cm$^{-1}$) and $\beta$ (near 7500 cm$^{-1}$) modes. These modes appear owing to the strong spin-orbit-coupling ($E_{SO}$) in Sr-214, which splits the $t_{2g}$ orbital into $J_{eff}$ = 1/2 and 3/2 orbitals, and the Coulomb repulsion ($U$), which splits the half-filled $J_{eff}$ = 1/2 state into the lower Hubbard band (LHB) and upper Hubbard band (UHB)\cite{kim:2008,kuriyama:2010}. Therefore, the ground state of the undoped Sr-214 is insulating. The $\alpha$ mode has been identified as the optical transition from the occupied $J_{eff}$ = 1/2 LHB to the unoccupied $J_{eff}$ = 1/2 UHB and the $\beta$ mode as the transition from the occupied $J_{eff}$ = 3/2 band to the unoccupied $J_{eff}$ = 1/2 UHB. Undoped Sr-214 has been known as a $J_{eff}$ = 1/2 Mott insulator. In a previous study\cite{lee:2012}, measured optical spectra at room temperature of Ru-doped Sr-214 thin films exhibited strong Drude components, indicating that the  $J_{eff} =$ 1/2 LHB is partially filled by charge carriers introduced by Ru-doping. However, as shown in Fig. \ref{fig2}, the optical conductivity spectra of Ru-doped Sr-214 single crystals were significantly different from those in a previous optical study on Ru-doped Sr-212 thin films\cite{lee:2012}. In the previous study, a thin film with a continuous doping gradient was prepared on a substrate using a pulsed laser deposition; different spots in the film have different doping levels. The optical conductivity was estimated through a numerical subtraction of the substrate contribution using an approximate formula\cite{okazaki:2006}. The resulting conductivity spectra may contain unexpected uncertainty, which may result from the subtraction process. Additionally, the measured spectra might be affected by some extrinsic effects such as a lattice mismatch between the thin film and substrate. In fact, one earlier study demonstrated that the strain could be used to tune the electronic bandwidth, which caused changes in the overall electronic band diagram\cite{nicols:2013,bhandari:2019}. However, the causes of differences between the two data sets obtained from thin films and single crystals have not been clearly understood yet.

\begin{figure}[!htbp]
  \vspace*{-0.3 cm}%
  \centerline{\includegraphics[width=4.7 in]{Figure3newm.pdf}}%
  \vspace*{-0.5 cm}%
\caption{(Color online) Optical conductivity data and Drude-Lorentz model fits of Sr$_2$Ir$_{1-x}$Ru$_x$O$_4$ ($x$ = 0.01, 0.07, and 0.10) at two representative temperatures: 30 and 300 K. The dotted, solid, and dashed lines represent the inner-gap excitation (or the $I$ mode), $\alpha$ mode, and $\beta$ mode, respectively. The dot-dashed and dot-dot-dashed lines represent the Drude modes and $pd$-transitions, respectively.}
 \label{fig3}
\end{figure}

Fig. \ref{fig3} shows the optical conductivity spectra and Drude-Lorentz model fits of our three samples at two representative temperatures, 30 and 300 K. In the Drude-Lorentz model, the real part of the optical conductivity can be expressed as
\begin{equation}\label{eq1}
\sigma_1(\omega)=\frac{\Omega_{D,p}^2}{4\pi}\frac{1/\tau_{imp}}{\omega^2+(1/\tau_{imp})^2} + \sum_k\frac{\Omega_{k,p}^2}{4\pi}\frac{\gamma_k\omega^2}{(\omega_k^2-\omega^2)^2+\gamma_k^2\omega^2},
\end{equation}
where $\Omega_{D,p}$ is the Drude plasma frequency and $1/\tau_{imp}$ is the elastic impurity scattering rate. $\omega_k$, $\Omega_{k,p}^2$, and $\gamma_k$ are the center frequency, strength, and width (or damping parameter) of the $k$th Lorentz oscillator, respectively. Ru-doping introduces another new mode below the $\alpha$ mode, which was observed and interpreted as an inner-gap excitation in previous optical studies on Ru-doped Sr-214 thin films\cite{lee:2012} and La-doped Sr-214 single crystal samples\cite{seo:2017,wang:2018}. For the fitting, we used one Drude mode, three ($I$, $\alpha$, and $\beta$) Lorentz modes, and one additional interband transition located at the high frequency region, known as a $pd$-transition\cite{sohn:2014}. Here, we denote the inner-gap excitation as the $I$ mode. Similar inner-gap excitations have been observed in 3$d$ transition metal oxides when they are doped\cite{imada:1998}. This new mode may be related to the famous mid-infrared absorption in cuprates\cite{uchida:1991,quijada:1999}. The absorption can be explained as the incoherent component of the spectral weight of charge carriers induced by a strong correlation\cite{hwang:2008a}. However, the origin of the inner-gap excitation in Ru-doped Sr-214 systems is not clearly known yet. We separately show the Drude mode, the three prominent Lorentz modes, and the $pd$-transition in Fig. \ref{fig3}. We ignored the sharp phononic absorption peaks located in the low-frequency region below 800 cm$^{-1}$ for these fittings.

\begin{figure}[!htbp]
  \vspace*{-0.4 cm}%
  \centerline{\includegraphics[width=4.7 in]{Figure4newp.pdf}}%
  \vspace*{-0.5 cm}%
\caption{(Color online) (a,b) Normalized relative temperature-dependencies, $\Delta [\Omega_{D,p}(T)]^2/[\Omega_{D,p}(300K)]^2$ and $\Delta [1/\tau_{imp}(T)]/[1/\tau_{imp}(300K)]$ of the two Drude fitting parameters ($\Omega_{D,p}$ and $1/\tau_{imp}$)). (c) Temperature-dependent DC resistivity obtained from the Drude fitting parameters.}
 \label{fig4}
\end{figure}

In Fig. \ref{fig4}(a) and \ref{fig4}(b), we show normalized relative temperature-dependencies, $\Delta [\Omega_{D,p}(T)]^2/[\Omega_{D,p}(300 K)]^2$ and $\Delta [1/\tau_{imp}(T)]/[\tau_{imp}(300 K)]$, of two fitting parameters (plasma frequency and impurity scattering rate) of the Drude mode for the three Ru-doped samples ($x =$ 0.01, 0.07, and 0.10). The normalized relative temperature-dependencies are defined by $\Delta [\Omega_{D,p}(T)]^2/[\Omega_{D,p}(300 K)]^2 \equiv \{[\Omega_{D,p}(T)]^2-[\Omega_{D,p}(300 K)]^2\}/[\Omega_{D,p}(300 K)]^2$ and $\Delta 1/\tau_{imp}(T)/[1/\tau_{imp}(300 K)] \equiv [1/\tau_{imp}(T)-1/\tau_{imp}(300 K)]/[1/\tau_{imp}(300 K)]$. Pairs of $\Omega_{D,p}(300 K)$ and $1/\tau_{imp}(300 K)$ are 1250 and 800, 2550 and 1100, and 3700 and 1290 cm$^{-1}$, for the three samples with $x$ = 0.01, 0.07, and 0.10, respectively. The Drude plasma frequencies of all three samples exhibit significant temperature dependencies. The plasma frequency squared, $\Omega_{D,P}^2$, which is proportional to the charge-carrier density, monotonically increases with raising the temperature, probably due to thermal excitations. In general, the plasma frequency of a good metal is independent of temperature. The impurity scattering rates of all three samples decrease with lowering the temperature, which is a generic temperature-dependent trend of the impurity scattering rate of a good metal. Both plasma frequency and impurity scattering rate increase with the Ru-doping. Using these two Drude fitting parameters, we estimated dc resistivity ($\rho$), i.e., $\rho = 4\pi(1/\tau_{imp})/\Omega_{D,p}^2$. The estimated dc resistivity data of the three samples are shown as functions of temperature in Fig. \ref{fig4}(c). The obtained dc resistivity data of all three samples exhibit insulating temperature-dependent behavior, which is consistent with the directly measured DC resistivity data shown in Fig. \ref{fig0}. We can see huge discrepancy in low temperatures between the two sets of DC resistivity data. At low temperatures, the Drude component is very (or almost too) weak to get reasonable fitting parameters. Therefore, the very weak Drude component may cause the huge discrepancy between the two sets of the DC resistivity data at low temperatures.

\begin{figure}[!htbp]
  \vspace*{-0.3 cm}%
  \centerline{\includegraphics[width=4.7 in]{Figure5newk.pdf}}%
  \vspace*{-0.5 cm}%
\caption{(Color online) Experimentally estimated number of charge-carriers per Ir atom of the three samples (Sr$_2$Ir$_{1-x}$Ru$_x$O$_4$; $x$ = 0.01, 0.07, and 0.10) as functions of temperature (in the main frame) and the charge-carriers per Ir atom for the three samples (in the narrow right frame) estimated using the EDS technique.}
 \label{fig5}
\end{figure}

Fig. \ref{fig5} shows the effective charge-carrier number per Ir atom, $N_{eff, exp}$, of the three Ru-doped Sr-214 samples obtained from the Drude plasma frequencies. The the effective charge-carrier number per Ir atom ($N_{eff, exp}$) can be estimated using the relationship between the plasma frequency and carrier density, i.e., $N_{eff, exp} = \Omega_{D,p}^2 m_e V_{Ir}/(4\pi e^2)$, where $m_e$ is the electron mass, $e$ the elementary charge, and $V_{Ir}$ the volume per one formula unit. The experimentally estimated $N_{eff,exp}$ exhibited strong Ru-doping and temperature dependencies. As the temperature increased, $N_{eff, exp}$ significantly increased due to the thermal excitation effect, which are closely related to the small activation energies estimated from the DC resistivity data (see the inset of Fig. \ref{fig0}). The estimated $N_{eff, exp}$ were significantly smaller than the expected ones ($N_{eff, EDS}$), which were estimated using the EDS technique. The $N_{eff,EDS}$ estimated by the EDS are shown in the narrow frame on the right in Fig. \ref{fig5}, where the vertical axis scale is around 3 times larger than that of the main frame. This kind of discrepancy between the estimated values from the optical spectroscopy and EDS technique can be observed between two infrared studies of La-doped Sr-214 system (Sr$_{2-y}$La$_y$IrO$_4$)\cite{seo:2017,wang:2018}. The related problem of doping may be seen between two samples (Sr$_{2-y}$La$_y$IrO$_4$ with $y$ = 0.1 and 0.18) in an optical study\cite{wang:2018}. It is not clear why the estimated $N_{eff,exp}$ is significantly smaller than the EDS one. A possible interpretation of this low charge-carrier density may be associated with the effective mass. For the estimation, we considered the mass of charge carriers as the bare electron mass; however, the mass can be enhanced owing to the correlations. Typically, the spectral weight is divided into two components (coherent and incoherent) owing to the correlation\cite{hwang:2008a,basov:2011a}. Here, the Drude spectral weight ($\propto N_{eff,exp}$) can be approximately that of the coherent component, and the total spectra weight can be the EDS spectral weight ($\propto N_{eff,EDS}$). With these values at 30 K, we obtained the coupling constants ($\lambda$), which can be defined as $\lambda +1 \equiv N_{eff,EDS}/N_{eff,exp}$. The coupling constants of the three samples are 459, 1287, and 14.2 for the samples with $x =$ 0.01, 0.07, and 0.10, respectively), which seemed to be way too large. Hence, other unknown doping-related phenomena may exist in Ru-doped Sr-214, as in the Rh-doped Sr-214\cite{clancy:2014,cao:2016}.

\begin{figure}[!htbp]
  \vspace*{-0.4 cm}%
  \centerline{\includegraphics[width=6.5 in]{Figure6newj.pdf}}%
  \vspace*{-0.5 cm}%
\caption{(Color online) Normalized relative temperature-dependent center frequencies (a-c), strength ($\Omega_p^2$) (d-f), and damping parameters (g-i) of the inner-gap excitation ($I$ mode), $\alpha$, and $\beta$ modes for the three Ru-doped Sr-214 samples.}
 \label{fig6}
\end{figure}

Fig. \ref{fig6} shows the doping- and temperature-dependent fitting parameters of three Lorentz ($I$, $\alpha$, and $\beta$) modes for the three samples. We took a relative value to the value at 300 K and normalized with the value at 300 K, i.e., $\Delta A(T)/A(300 K) \equiv [A(T)-A(300 K)]/A(300 K)$, where $A$ is a representative quantity. The temperature-dependent properties of the $\alpha$ and $\beta$ modes of undoped Sr-214 have been previously reported in the literature\cite{moon:2009,sohn:2014}. Our sample at the lowest doping concentration exhibited a similar temperature-dependence as that reported previously. As the temperature increases, the $\alpha$ mode shifts to a lower frequency. The temperature-dependent trend remains the same as that of undoped Sr-214 (Fig. \ref{fig6}(b)). As the Ru-doping increases, the $I$ mode shifts to a lower frequency as well (see Table I). The $\alpha$ and $\beta$ modes exhibit similar doping- and temperature-dependent trends as the $I$ mode. Similar doping-dependent shifts of the $\alpha$ and $\beta$ modes were observed in a recent optical study on La-doped Sr-214\cite{seo:2017}. We also observe that the temperature-dependent shift of the $\alpha$ mode ($\Delta \omega_{\alpha}(T) \equiv \omega_{\alpha}(T)-\omega_{\alpha}(300K)$) is twice that of $\beta$ mode ($\Delta \omega_{\beta}(T) \equiv \omega_{\beta}(T)-\omega_{\beta}(300K)$), as reported in a previous paper\cite{sohn:2014}, i.e.,$\Delta \omega_{\alpha}(T) = 2\Delta \omega_{\beta}(T)$. A previous study proposed that the observed shift ratio between the two interband ($\alpha$ and $\beta$) transition energies is associated with the electron-phonon coupling\cite{sohn:2014}. This energy shift ratio between the $\alpha$ and $\beta$ modes is not dependent on Ru-doping. We note that $2\omega_{\alpha}(300K) \cong \omega_{\beta}(300K)$ (see Table I). At each temperature, all three modes shifts to a lower frequency as the doping increases (see Table I). The normalized relative temperature-dependent strength ($\Omega_p^2$) (d-f) and damping parameters (g-i) of the three Lorentz modes are shown in the right two columns in Fig. \ref{fig6}. We also provide all fitting parameters of the three Lorentz modes at 300 K in Table I. We observe systematic temperature-dependent trends of the strength and damping parameter. However, as we can see in Table I, the strength of the $I$ mode increases with the Ru-doping, while those of both $\alpha$ and $\beta$ modes decrease, which are similar doping-dependent trends in the strengths of the three Lorentz modes observed in La-doped Sr-214 systems\cite{seo:2017}. With lowering the temperature, the intensity of $I$ mode decreases, while those of both $\alpha$ and $\beta$ modes increase, indicating that the spectral weight of the $I$ mode shifts to the $\alpha$ and $\beta$ modes. All three damping parameters monotonically decrease with lowering the temperature, which is a general temperature-dependent trend of the damping parameter.

\begin{table}[h!]
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
  \hline\hline

   samples & $\omega_I$ & $\omega_{\alpha}$ & $\omega_{\beta}$ & $\Omega_{p,I}$ & $\Omega_{p,\alpha}$ & $\Omega_{p, \beta}$ & $\gamma_I$ & $\gamma_{\alpha}$ & $\gamma_{\beta}$ \\
  \hline\hline
  $x$ = 0.01 & 2970 & 3770 & 7675 & 9700 & 6050 & 17350 & 4600 & 2000 & 7060 \\
  \hline
   0.07 & 2700 & 3750 & 7530 & 11300 & 4850 & 15200 & 4800 & 2300 & 5750 \\
  \hline
   0.10 & 2550 & 3700 & 7400 & 11200 & 5550 & 12662 & 4500 & 2800 &6150 \\
  \hline\hline
\end{tabular}
\caption{All fitting parameters of the three ($I$, $\alpha$, and $\beta$) Lorentz modes at 300 K. Note that units for the all values are cm$^{-1}$.}
\label{table:1}
\end{table}

\begin{figure}[!htbp]
  \vspace*{-0.4 cm}%
  \centerline{\includegraphics[width=6.5 in]{Figure7newh.pdf}}%
  \vspace*{-0.5 cm}%
\caption{(Color online) Temperature-dependent (a) ratio of the Coulomb repulsion to the bandwidth ($U/W$) and (b) spin-orbit coupling energy ($E_{SO}$) for the three Ru-doped Sr-214 samples as functions of temperature. On the right, a schematic band diagram is shown.}
 \label{fig7}
\end{figure}

Fig. \ref{fig7} shows the temperature-dependent ratio of the Coulomb repulsion to the bandwidth ($U/W$) and the spin-orbit coupling energy ($E_{OS}$) for the three samples. As shown in the schematic band diagram in the right figure, $U$ is the same as the center frequency of the $\alpha$ mode and $W$ is the same as the width of the $\alpha$ mode. $E_{SO}$ is obtained from the center frequencies of $\alpha$ and $\beta$ modes, i.e., $E_{SO} = \omega_{\beta} - \omega_{\alpha}$. The schematic band diagram is based on our optical results. As the temperature increases, $U/W$ decrease because the thermal broadening of the $\alpha$ mode makes $W$ larger and the shifting to lower energy of the $\alpha$ mode makes $U$ smaller (see Fig. \ref{fig6}(b) and (h)). As the doping increases, $U/W$ also significantly decreases because the center frequency of the $\alpha$ mode decreases while its width increases with increasing the doping as shown in Table I. As the temperature increases, $E_{SO}$ increases because the temperature-dependent changes between the $\alpha$ and $\beta$ modes is different; the center frequency of the $\beta$ mode decreases less than that of the $\alpha$ mode as we discussed in the previous paragraph, i.e., $\Delta \omega_{\alpha}(T) = 2\Delta \omega_{\beta}(T)$. As doping increases, $E_{SO}$ decreases because the center frequency of the $\beta$ mode decreases more than that of the $\alpha$ mode with increasing the doping as shown in Table I, which is consistent with a previously reported one\cite{lee:2012}.

\section{Conclusions}

We measured reflectance spectra of three differently Ru-doped Sr-214 single crystal samples. We obtained the optical conductivity spectra of the three samples from the measured reflectance spectra using the Kramers-Kronig analysis. We fitted the optical conductivity spectrum with the well-known Drude-Lorentz model. The charge-carrier number per Ir atom estimated from the Drdue plasma frequency was significantly smaller than the expected value estimated from the nominal doping concentration. The origin of low charge-carrier number has not yet been understood. All three samples exhibited insulating temperature-dependent behavior, indicating that the electronic ground state is insulating. Three major interband optical transitions ($I$, $\alpha$, and $\beta$) were observed. From the Drude-Lorentz fitting parameters, we obtained temperature- and Ru-doping-dependent $U/W$ and $E_{SO}$. With increasing temperature, the $U/W$ decreased while $E_{SO}$ increased. With increasing doping, both $U/W$ and $E_{SO}$ decreased. We hope that our experimental results will be useful for understanding the physical properties of Ru-doped Sr-214 systems.

\acknowledgments J.H. acknowledges financial support from the National Research Foundation of Korea (NRF-2017R1A2B4007387 and 2021R1A2C101109811). The work at Yonsei University was supported by the National Research Foundation of Korea (grant numbers NRF-2017R1A5A1014862 (SRC program: vdWMRC center), NRF-2019R1A2C2002601, and NRF-2021R1A2C1006375).

\bibliographystyle{apsrev4-1}

Strongly Correlated Electrons (cond-mat.str-el)