What are the galaxies contributing to the Cosmic Infrared Background ?

Abstract: Recent optical observations have led to a significant progress in our understanding of galaxy formation and evolution. However, our view on the deep universe is currently limited to the starlight which directly escapes from high-redshift galaxies, since we so far ignore the fraction of luminosity absorbed by dust and released in the IR/submm wavelength range. A new constraint is set by the possible detection of the Cosmic Infrared Background. We briefly review the observations and use a semi-analytic model of galaxy formation and evolution to predict number counts consistent with the level of the background. It turns out that the predictions fairly accomodate preliminary data at 175 and 850 microns. This suggests that a significant fraction of star/galaxy formation at high z is hidden by dust.

\section{Introduction}

The picture of galaxy formation and evolution which is progressively
emerging from deep optical surveys is fascinating. The
measurement of the cosmic star formation rate
(hereafter SFR) density of the universe up to $z \sim 4$
from rest--frame UV fluxes seems to show a strong peak at $z \sim 1.5$
and suggests that we could have seen the bulk
of star formation in the universe
(Lilly {\it et al.} 1996; Madau {\it et al.} 1996).
This high SFR seems to be correlated with
the decrease of the cold--gas comoving
density in damped Lyman--$\alpha$ systems
between $z=2$ and $z=0$ (Storrie--Lombardi {\it et al.} 1996).
These results nicely fit in
a view where star formation in bursts triggered by interaction/merging
consumes and enriches the gas content of galaxies as time goes on.
Indeed, such a scenario is qualitatively predicted within the paradigm of
hierarchical growth of structures in which galaxy formation is a continuous
process.

However, we have only a partial view on galaxy
evolution since most of the observational data come from optical
surveys which probe the rest--frame UV and visible emission of
high--$z$ galaxies.
A still unknown fraction of star/galaxy formation is hidden by dust which
absorbs UV/visible starlight and re--radiates at larger wavelengths.
At low $z$, the IRAS satellite has discovered a sequence of IR
properties, from ``normal spirals'' to the ``luminous IR galaxies''
(hereafter LIRGs), mostly interacting systems, and the
spectacular ``ultraluminous IR galaxies'' (hereafter ULIRGs),
which are mergers
(Sanders and Mirabel 1996) and emit more than 95 \% of their
energy in the IR.

\begin{figure}[htbp]
\vbox
{\vskip -0.8truecm
\psfig{figure=guiderdonib1.ps,width=0.7\textwidth}
\vskip -0.8truecm
\caption{FIRAS residuals in the cleanest regions of the sky
(solid line), and the CIRB (dotted line).}
\psfig{figure=guiderdonib2.ps,width=0.7\textwidth}
\vskip -1truecm
\caption{Diffuse backgrounds in the FIR/submm and in the optical. Solid
triangles: COBE/DIRBE upper limits (Hauser 1995).
Thick solid line: CIRB (Puget {\it et al.}
1996; Guiderdoni {\it et al.} 1997a). Solid hexagons:
Cosmic Optical Background from faint galaxy counts (Williams {\it et al.}
1996). Long--dashes--and--short--dashes: predictions
without evolution. Dots--and--short--dashes: quiescent star formaiton
history. Solid line : scenario
A without ULIRGs. Dots--and--long--dashes: scenario E with ULIRGs, fitting
the CIRB. See Guiderdoni {\it et al.} (1997b) for details.}
}
\end{figure}

In contrast, we know very little
about galaxy evolution at high $z$ in this wavelength range.
Faint galaxy counts with IRAS seem to show a strong luminosity and/or
density evolution at $z \la 0.2$, but it is difficult to extrapolate
this trend to higher redshifts on a firm ground.
The Cosmic Infrared Background (hereafter CIRB) detected by Puget {\it et al.}
(1996) in FIRAS residuals sets strong constraints on the presence of a
population of IR/submm sources. We hereafter briefly review the observational
case (in Sec. 2)
and use a semi-analytic modelling of galaxy formation and evolution
to predict number counts (in Sec. 3).

\section{The diffuse background due to galaxies}

The epoch of galaxy formation can be observed in
the background radiation which is produced by the accumulation
of the light of extragalactic sources along the line of sight.
The search for the ``Cosmic Optical Background''
currently gives only upper limits. The shallowing
of the HDF faint counts suggests that we are now close to
convergence and that an estimate of the COB can
be obtained by summing up the contributions of faint galaxies
(Williams {\it et al.} 1996).

\begin{center}
\begin{figure}[htbp]
\psfig{figure=guiderdonib3.ps,width=0.7\textwidth}
\vskip -1truecm
\caption{Evolution of the ``cosmic constraints'' for scenario A (solid line)
without ULIRGs and scenario E (dots and long dashes) with a fraction of ULIRGs
increasing with $z$. {\it Upper panel:}
comoving star formation rate density as
computed from rest--frame UV luminosity densities, by using Salpeter IMF with
slope $1.35$ {\it without} extinction.
{\it Middle panel:} cold gas density parameter in damped
Lyman--$\alpha$ absorbers. {\it Lower panel:} rest--frame luminosity
densities.}
\end{figure}
\end{center}

The re--analysis of COBE/FIRAS residuals
between 200 $\mu$m and 2 mm has led Puget {\it et al.} (1996) to
discover the presence of an isotropic component which is likely to
be the long--sought CIRB.
Several steps are necessary in order to
remove the foregound Galactic components and extract the isotropic residual
identified as the CIRB. Whereas, at the wavelengths probed by FIRAS, the
interplanetary emission is small and easily removed, the
emission from interstellar dust mixed with the different gas phases of the
interstellar medium is the dominant component. In order to
address this problem, the original method of Puget {\it et al.}
was applied again by Guiderdoni {\it et al.} (1997a),
but only in the cleanest regions with very low HI column densities
($N_{HI} \leq 1 \times 10^{20}$ atoms cm$^{-2}$ instead of
$N_{HI} \leq 4.5 \times 10^{20}$ atoms cm$^{-2}$). In that case, the residual
component totally dominates the emission, as shown in Fig. 1.
Fig. 2 shows that the CIRB intensity per frequency decade $\nu
I_{\nu} \simeq (7 \pm 0.3)~10^{-9}$ W m$^{-2}$ sr$^{-1}$ near
300 $\mu$m is a factor of 5
higher than the no--evolution prediction obtained by a simple extrapolation of
the IR luminosities of local galaxies.

Such a detection yields the first
``post--IRAS'' constraint on the high--$z$
evolution of galaxies in the IR/submm range,
before the era of ISO results. Its level is comparable to the
optical background estimated by summing up faint galaxy counts down
to the deepest limit so far available which is given by the Hubble Deep Field
(Williams {\it et al.} 1996), and suggests
that a significant fraction of the energy of young stars is absorbed by
dust and released in the IR/submm.

The DIRBE instrument on COBE has given so far
upper limits on the IR background at wavelengths between 2 and 300 $\mu$m
(Hauser 1995). However Schlegel {\it et al.} (1997) extract a uniform
background radiation from DIRBE observations at the levels $\nu
I_{\nu} \simeq (17 \pm 4)~10^{-9}$ W m$^{-2}$ sr$^{-1}$ at 240 $\mu$m and $\nu
I_{\nu} \simeq (32 \pm 13)~10^{-9}$ W m$^{-2}$ sr$^{-1}$ at
140 $\mu$m. At least
the value at 240 $\mu$m, where the foregrounds are less important, seems
to be consistent with the estimate from FIRAS. A more thorough analysis of the
DIRBE data is currently undertaken by the DIRBE team.

\section{Semi--analytic modelling}

In order to
analyse the current data and make predictions for forthcoming observations,
we have chosen to model galaxy evolution by
using the so--called ``semi--analytic'' approach which
has been rather successful in reproducing the overall
properties of galaxies in the optical range. We have elaborated an
extension of this type of method to the IR/submm range. Details of the
modelling, and extensive predictions are
given elsewhere (Guiderdoni {\it et al.} 1997a,b).
In the Standard Cold Dark Matter cosmological
scenario ($H_0=50$ km s$^{-1}$ Mpc$^{-1}$, $\Omega_0=1$, $\Lambda=0$,
$\Omega_b=0.05$, $\sigma_8=0.67$), we have designed a family of plausible
evolutionary scenarios with different fractions of ULIRGs. In the following,
scenario A has no ULIRGs and scenario E has a fraction of
ULIRGs increasing with $z$.

Fig. 3 shows the predicted comoving SFR, gas, and luminosity
densities in the universe, for these
scenarios. The strong episode of star formation at $z \sim 1$
corresponds to the decrease of the gas density in the universe.
The local luminosity densities in the UV, visible and IR are
accommodated by all our evolutionary scenarios, though they predict
high--$z$ IR luminosity densities which are strongly different.
 The predicted backgrounds generated with our scenarios are displayed
in Fig. 2. Scenario E nicely fits the level and shape of the CIRB.

\begin{center}
\begin{figure}[htbp]
\psfig{figure=guiderdonib4.ps,width=0.6\textwidth}
\caption{Predictions for faint galaxy counts at 15 $\mu$m,
60 $\mu$m, 175 $\mu$m, and 850 $\mu$m for scenario E fitting the CIRB.
See text for the keys of the symbols.}
\end{figure}
\end{center}

The correponding
predictions for faint galaxy counts are given in Fig. 4, with data from
IRAS counts at 60 $\mu$m (Lonsdale {\it et al.} 1990; open stars),
the ISO--HDF follow--up with ISOCAM
at 15 $\mu$m (Oliver {\it et al.} 1997; solid squares) to be compared
with data from the IRAS survey (Rush {\it et al.} 1993; solid dot),
preliminary data for the
Lockman Hole (Kawara {\it et al.} 1998, in preparation; open dot)
and the Marano field (Puget {\it et al.} 1998, in preparation; solid
triangles)
with ISOPHOT at 175 $\mu$m, and the first deep field observed with
the JCMT/SCUBA at 850 $\mu$m (Smail {\it et al.} 1997; open squares).
The agreement
of the predictions with the data seems good enough to suggest that
these counts do probe the evolving
population contributing to the CIRB. The model
shows that 15 \%  and 60 \% of the CIRB respectively at 175 $\mu$m
and 850 $\mu$m are built up by objects brighter than
the current limits of ISOPHOT and
SCUBA deep fields. It is also possible to get predicted redshift
distributions for these counts. The predicted median redshift of the ISO--HDF
is $z \sim 0.8$. It increases to $z \sim 1.5$ for the deep ISOPHOT
surveys, and to $z \ga  2$ for SCUBA, even if the latter value seems to be
very sensitive to the details of evolution.

\section{Conclusions}

With ISO and SCUBA, a new window is now open to deep extragalactic
surveys. A large number of
objects are expected from current models of galaxy evolution in the
IR/submm. These models accommodate local IR data and follow
the high--redshift evolution of the ``cosmic constraints'' (that is, the
comoving SFR, gas and UV/optical/NIR luminosity densities), as well as
the high level of the isotropic component in COBE residuals
which is probably the CIRB. The data seem to be consistent with a strong
evolution. Significant progress in resolution
and/or sensitivity expected from forthcoming satellites such as WIRE,
SIRTF, and FIRST will show the high--redshift counterparts of
local ULIRGs.
The ``discovery potential'' of
the forthcoming IR/submm observations is very exciting.

\acknowledgments

I thank my collaborators F.R. Bouchet, E. Hivon, G. Lagache,
B. Maffei, and J.L. Puget.


Data-Efficient Classification of Radio Galaxies

Abstract: The continuum emission from radio galaxies can be generally classified into different morphological classes such as FRI, FRII, Bent, or Compact. In this paper, we explore the task of radio galaxy classification based on morphology using deep learning methods with a focus on using a small scale dataset ($\sim 2000$ samples). We apply few-shot learning techniques based on Twin Networks and transfer learning techniques using a pre-trained DenseNet model with advanced techniques like cyclical learning rate and discriminative learning to train the model rapidly. We achieve a classification accuracy of over 92\% using our best performing model with the biggest source of confusion being between Bent and FRII type galaxies. Our results show that focusing on a small but curated dataset along with the use of best practices to train the neural network can lead to good results. Automated classification techniques will be crucial for upcoming surveys with next generation radio telescopes which are expected to detect hundreds of thousands of new radio galaxies in the near future.

\section{Introduction}

This is a simple template for authors to write new MNRAS papers.
See \texttt{mnras\_sample.tex} for a more complex example, and \texttt{mnras\_guide.tex}
for a full user guide.

All papers should start with an Introduction section, which sets the work
in context, cites relevant earlier studies in the field by \citet{Others2013},
and describes the problem the authors aim to solve \citep[e.g.][]{Author2012}.

\section{Methods, Observations, Simulations etc.}

Normally the next section describes the techniques the authors used.
It is frequently split into subsections, such as Section~\ref{sec:maths} below.

\subsection{Maths}
\label{sec:maths}

Simple mathematics can be inserted into the flow of the text e.g. $2\times3=6$
or $v=220$\,km\,s$^{-1}$, but more complicated expressions should be entered
as a numbered equation:

\begin{equation}
    x=\frac{-b\pm\sqrt{b^2-4ac}}{2a}.
	\label{eq:quadratic}
\end{equation}

Refer back to them as e.g. equation~(\ref{eq:quadratic}).

\subsection{Figures and tables}

Figures and tables should be placed at logical positions in the text. Don't
worry about the exact layout, which will be handled by the publishers.

Figures are referred to as e.g. Fig.~\ref{fig:example_figure}, and tables as
e.g. Table~\ref{tab:example_table}.

\begin{figure}

	\includegraphics[width=\columnwidth]{example}
    \caption{This is an example figure. Captions appear below each figure.
	Give enough detail for the reader to understand what they're looking at,
	but leave detailed discussion to the main body of the text.}
    \label{fig:example_figure}
\end{figure}

\begin{table}
	\centering
	\caption{This is an example table. Captions appear above each table.
	Remember to define the quantities, symbols and units used.}
	\label{tab:example_table}
	\begin{tabular}{lccr}
		\hline
		A & B & C & D\\
		\hline
		1 & 2 & 3 & 4\\
		2 & 4 & 6 & 8\\
		3 & 5 & 7 & 9\\
		\hline
	\end{tabular}
\end{table}

\section{Conclusions}

The last numbered section should briefly summarise what has been done, and describe
the final conclusions which the authors draw from their work.

\section*{Acknowledgements}

The Acknowledgements section is not numbered. Here you can thank helpful
colleagues, acknowledge funding agencies, telescopes and facilities used etc.
Try to keep it short.

\section{Introduction}

The radio continuum sky contains a wide variety of sources. Some are compact sources while others are extended.
Compact sources are unresolved sources with a single source component whereas
extended sources are any whose sizes are greater than the effective resolution of the telescope.

In some galaxies, emission from the central Active Galactic Nucleus (AGN) is seen in the form of relativistic jets which manifest as radio synchrotron emission. These radio jets usually come in pairs and extend outwards in diametrically opposite directions. When they collide with the denser parts of the intergalactic medium, they give rise to the radio lobes. The morphology of the twin jets and radio lobes is used to broadly classify radio galaxies into FRI and FRII types \citep{fanriley74}. In FRI galaxies, the peak of the emission is closer to the central core and the emission brightness decreases farther away from the centre. On the other hand, the radio emission in FRII galaxies increases in brightness outwards culminating in bright hotspots at the outer edge of the jet emission.
See Figure~\ref{fig:FR-sample} for typical examples of FRI and FRII galaxies drawn from our sample.

\begin{figure}
    \centering
    \includegraphics[width=0.5\linewidth]{figures/FRI-mag-invert.jpeg}\\
    \includegraphics[width=0.5\linewidth]{figures/FRII-mag-invert.jpeg}
    \caption{A typical example of an FRI (top) and FRII (bottom) type radio galaxy from our sample.}
    \label{fig:FR-sample}
\end{figure}

Apart from these two broad classes, some radio galaxies also show a more complex morphology. If the angle between the jets in the radio galaxy is significantly less than $180^{\circ}$, then it can be classified as a Bent-Tail (BT) galaxy. BT galaxies can be further subclassified as Narrow-Angle Tail (NAT) or Wide Angle Tail (WAT) galaxies based on the angle between the two jets \citep{rudwen76}. In NATs, the jets are so bent that the entire radio structure lies on one side of the optical galaxy \citep{miley72} whereas in WATs the angle of projection between the jets is $\sim 60^{\circ}$ \citep{dehghan14}. Bent Tail galaxies are believed to form in dense environments such as galaxy clusters. The bent tail phenomenon is usually attributed to the large ram-pressure exerted on the jets as a result of the motion of the host galaxy through the cluster medium \citep{miley72,rudwen76,burns90} or due to the turbulent movement of the ICM (Intra-Cluster Medium) during a merger \citep{burns90,burns96,roettiger96}. As such, they can be effectively used as tracers for the presence of galaxy clusters, particularly at high redshifts \citep{blanton00,blanton03,mao09,mao10,wington11,dehghan14}. It is expected that the upcoming Evolutionary Map of the Universe (EMU) survey \citep{norris11} will be able to detect many new galaxy clusters via this method.

Traditionally, radio galaxy classification has been carried out by visual inspection of radio continuum images. Large area radio surveys like the NVSS \citep{nvss98}, FIRST \citep{first95}, SUMSS \citep{sumss99} have been used to identify and classify radio galaxies based on their morphology. The Radio Galaxy Zoo (RGZ) project aims to use citizen scientists to assist in the identification and classification of candidate radio galaxies \citep{rgz15}. In the future, next generation radio telescopes like MeerKAT \citep{2016mks..confE...1J}, ASKAP \citep{askap08} and eventually the SKA \citep{ska04} will significantly improve in sensitivity and detect millions of new radio galaxies. Given these astronomical numbers, manual classification of radio galaxies would be highly impractical, if not impossible. As a result, astronomers need to identify an automated way to precisely classify radio galaxies based on their morphology.

Automated classification algorithms using machine learning and deep learning techniques are extremely useful for this purpose. Machine learning techniques have been widely applied to astronomical problems in recent years. See \citet{Fluke2020} for a recent review.

Classification of optical galaxies using artificial neural networks has been a particularly active field of research \citep{philip02,dieleman15,hc15,ostrovski17,schawinski17,hocking18,martin20,cheng20}. \cite{aniyan17} were the first to employ a CNN (Convolutional Neural Network) to classify radio galaxies using FIRST survey images. Since then several other researchers  \citep{wathela18, 2018MNRAS.476..246L, ma19, 10.1093/mnras/stz1289, wu19, tang19} have used different variations of the standard CNN architecture to attempt radio galaxy classification.

Recently, deep learning \citep{lecun2015deeplearning} algorithms have shown remarkable efficacy in image classification and speech recognition tasks across diverse fields.  McCulloch \& Pitts introduced the McCulloch-Pitts Neuron \citep{mcculloch43a} in 1943, which is considered as the initial version of the Artificial Neural Network. Further work by \cite{hebb-organization-of-behavior-1949}, \cite{rosenblatt1958perceptron} laid the foundation for modern neural networks. The current deep learning era started with the introduction of Deep Belief Networks by \cite{hinton06} (see \citet{DBLP:journals/corr/WangRX17} for a comprehensive history of deep learning). The term deep learning itself is associated with the collection of new techniques mainly based on Neural Networks. In these networks, inputs are fed into the input layer propagating through one or more hidden layers and finally linking to the output layer. A layer is mainly a set of nodes, sometimes referred to as units, connected to the immediately preceding and following layers. The input layer nodes are associated with the inputs from the dataset of interest. In deep neural networks, multiple layers are present (hence the term deep) and starting with the second layer, each layer essentially performs the feature construction based on the output of the previous layers which in turn acts as the input to the current layer. The output from the final layer is compared with the expected output and the difference (also called the `error') is propagated backward (called backpropagation \citep{Rumelhart:1986we}), through all the layers to adjust the units in each layer and the process is repeated. This backpropagation process helps in better optimisation of the network for each input dataset.

In this work, we have implemented a method to perform automated morphological classification of radio galaxies using deep learning into four important classes - FRI, FRII, Bent, and Compact.
The novel feature of this work is the focus on creating a classifier with fewer data samples while still striving for maximum accuracy. \cite{roh2018survey} provide an overview of the role of data in machine learning and deep learning techniques. An approximate rule of thumb is that we need at least 1000 samples per class in the task. The State of the Art results for Computer Vision tasks \citep{DBLP:journals/corr/abs-1905-00546, 2019arXiv191211370K} based on ImageNet \citep{deng2009imagenet} and MNIST \citep{lecun2010mnist} datasets have used architectures like ResNet \citep{he2016deep}, DenseNet \citep{huang2017densely} for  ideal performance but the sample size for these tasks has been at least 1000 images per class. \cite{8599448} discuss the impact of data sample size in image classification tasks and show that sample size affects the performance as we increase the number of classes in the task. For the transfer learning approach on image classification tasks, \cite{10.1007/978-3-319-46349-0_5} discuss that in case of a limited data sample, training only specific layers can provide performance similar to a model trained from scratch with a large dataset.

The previously mentioned approaches to build a classifier for radio galaxies have used a large amount of data (at least 10000 samples in each class) for experiments. Where necessary, sample sizes have been increased with the use of techniques like oversampling and data augmentation \citep{NIPS2012_4824}. In \citet{wathela18}, authors achieved an overall accuracy above 94\% (with the same 4 classes as this work - FRI, FRII, Bent, Compact for classification) using more than 21000 samples. \citet{aniyan17} performed a 3 class (FRI, FRII and Bent) classification using a small original dataset (716 total samples) but above 25000 samples for experiments with substantial use of augmentation. \citet{ma19} implemented an unsupervised pre-training technique on the unlabeled samples and later use this pre-trained model to classify the labeled samples as a 6 class classification task. They use 1442 labeled and 14245 unlabeled samples and at least 4900 samples per class in case of experiments with augmentation in their two-step approach. \citet{2018MNRAS.476..246L} performed 4 class (Compact, Single extended, Two-Component extended and Multi-component extended) classification using a total of 141032 samples via augmentation (21933 original samples) for 4 classes. \citet{wu19} used object detection and classification techniques while using more than 10000 samples for their experiments. The authors also leverage the transfer learning technique by loading pre-trained model weights based on ImageNet dataset. \citet{tang19} implemented transfer learning approach while maintaining the idea of domain transfer. The authors used NVSS \citep{nvss98} and FIRST \citep{first95} catalog samples and performed 1) pre-training of the model on NVSS data and transfer the weights for FIRST data and 2) pre-train model on FIRST data, transfer weights for NVSS data. Their task is 2 class (FRI and FRII) i.e. binary classification task. \citet{10.1093/mnras/stz1289} performed a 3 class (Unresolved, FRI and FRII) classification using a model pre-trained on ImageNet dataset and 15936 total image samples.
We have limited our data sample size to $\sim$2000 images for initial experiments and $\sim$2700 images for the experiments with the balanced classes. As we increase the number of data samples, we can capture the distribution of the data in a better way. However, obtaining a large number of samples is not always feasible. Thus, we need to develop algorithms that can perform better even with a limited amount of data. In this work, we apply few-shot learning and transfer learning techniques. Our results show that with the use of a curated dataset and a well designed neural network architecture, higher accuracy can be obtained even with very less amount of data for the classification task.

The paper is organised as follows: Section~\ref{sec:methodology} presents the overall approach towards the problem and includes the data sample construction used for the experiments, the initial approach of few-shot learning technique and its intuition to work efficiently with less amount of data, transfer learning introduction with a description of pre-trained models and the specific architecture used for our experiments. Section~\ref{sec:expresults} presents the experimental details where we try out and apply the optimum parameters to obtain the results. In section~\ref{sec:discussion}, we discuss the results, choice of hyperparameters and the scope for further work.

\section{Methodology}
\label{sec:methodology}
\subsection{Data Sample construction}
\label{sec:data}

Our radio galaxy sample was compiled using data from several catalogs. We used the catalog by \cite{proctor11} for bent-tail galaxies (BENT). For compact sources (Compact) we combined the FR0 catalog by \cite{baldi17} and the CoNFIG (Combined NVSS-FIRST Galaxies) sample \citep{gendre08, gendre10}. We only used the sources classified as ``Compact (C)'' for this sample. The sample of FRI galaxies included the FRICAT by \cite{capetti17a} as well as the sources classified as FRI in the CoNFIG sample. Similarly, the FRII sample of galaxies was combined from the FRIICAT by \cite{capetti17b} and the sources classified as FRII in the CoNFIG sample. We did not use sources classified as ``unresolved'' from the CoNFIG catalog.

The complete data sample initially contained 412 Bent, 332 Compact, 321 FRI, and 606 FRII sources. We manually inspected the FIRST image for every galaxy in our sample and discarded a few galaxies based on 1) Presence of multiple sources in a single image, 2) High noise component in the image, 3) Source present but concentrated at the extreme (L/R) of the image. Through this manual procedure, we removed 7 samples from the Bent set, 26 samples from the Compact set, 32 samples from the FRI set, 29 samples from the FRII set.
About $\sim$25\% samples from each class were used for validation purposes.
The test set images were obtained separately per category. The numbers of objects in each category are listed in Table~\ref{table:Table 1}.

Similar to \cite{aniyan17} some pre-processing was performed on the images before being used. Noise statistics were calculated in every image and all pixels below $3\sigma$ were clipped.

We handle the class imbalance in accordance with the majority class (FRII) via upsampling the other 3 classes. The upsampling is performed via duplication of randomly selected samples from the original set and combining these with the original set. The numbers of objects in balanced dataset for each category are listed in Table~\ref{table:Table 2}.

To work with this amount of data, we initially implemented the Few-Shot learning approach and used Siamese Networks for the classification purpose.

\begin{table}
    \centering
    \caption[Centre]{Final data sample distribution: number of total images, training, validation and testing images (\textbf{Original set}).}
    \label{table:Table 1}
    \begin{tabular}{lcccr}
        \hline
        Type & Number of samples & Train & Validation & Test\\
        \hline
        Bent & 508 & 305 & 100 & 103\\
        Compact & 406 & 226 & 80 & 100\\
        FRI & 389 & 215 & 74 & 100\\
        FRII & 679 & 434 & 144 & 101\\
        \hline
        Total & 1982 & 1180 & 398 & 404\\
        \hline
    \end{tabular}
\end{table}

\begin{table}
    \centering
    \caption[Centre]{Final data sample distribution with balanced classes: number of total images, training, validation and testing images.}
    \label{table:Table 2}
    \begin{tabular}{lcccr}
        \hline
        Type & Number of samples & Train & Validation & Test\\
        \hline
        Bent & 680 & 433 & 144 & 103\\
        Compact & 675 & 431 & 144 & 100\\
        FRI & 674 & 430 & 144 & 100\\
        FRII & 679 & 434 & 144 & 101\\
        \hline
        Total & 2708 & 1728 & 576 & 404\\
        \hline
    \end{tabular}
\end{table}

\subsection{Few-Shot Learning}

A variety of algorithms in deep learning have performed efficiently in terms of pattern recognition for image data. These algorithms often need large amounts of data and break down when forced to make predictions about data for which little supervised information is available. The ability to acquire and recognize new patterns is strong in humans. In particular, people seem to be able to understand new concepts quickly and then recognize variations on these concepts in future cases \citep{Lake_oneshot}. The deep learning algorithms are unable to learn the features for a class from a single image and generalize this information for the unseen images from the same class. The few shot learning approach is used to provide the machine with a quick understanding of the different images and their corresponding classes.

The few shot learning techniques are devised to classify the image samples correctly even when the availability of the supervised data is limited. It is a procedure that resembles human-like learning.\smallskip

\subsubsection{Advantages of Few Shot Learning (FSL):}
\begin{description}
\item[$\bullet$] Few Shot Learning can help in the reduction of the data-gathering efforts as well as the computation time and cost that arises with a large amount of data. The image classification accuracy ({\it P}) improves with the experience ({\it E}) obtained by a few labeled images for each class of the target ({\it T}), and the prior knowledge extracted from other classes, such as raw images to co-training \citep{colt98/blum}. Methods succeed in this task usually have higher generality, therefore they can be easily applied for tasks with large training samples.
\item[$\bullet$] Few Shot Learning can help in training suitable models for rare cases when the supervised data are limited.  Although this is a common situation in many astronomical problems, to our knowledge, this work represents the first application of few shot learning in astronomy.
\end{description}

\subsubsection{Siamese networks}

Siamese networks \citep{koch2015siamese} are neural networks containing two or more identical subnetwork components. A siamese network can be visualized as shown in Figure~\ref{fig:Siamese_arch}.

\begin{figure}
\centering
 \includegraphics[width=1.0\linewidth]{figures/Siamese_nets.png}
 \caption{Architecture of a simple Siamese Network.}
 \label{fig:Siamese_arch}
\end{figure}

The network is called `Siamese' as the subnetworks have identical architecture and the weights are also shared among them. Siamese network architecture is based on the idea that it can learn useful data descriptors that assist to compare the inputs of the respective subnetworks. The network handles different types of input such as numerical data (in this case the subnetworks are usually formed by fully-connected layers), image data (with Convolutional Neural Networks as subnetworks, current case) or even sequential data such as sentences or time signals (with Recurrent Neural Networks as subnetworks). In training deep neural networks, an activation function is used to define the output of a given node. It helps to avoid the linearity between the outputs of subsequent layers and thus the network can learn the complex patterns from the data.

Batch normalization \citep{DBLP:journals/corr/IoffeS15} (also referred to as BatchNorm) is another technique to normalize the inputs to each layer during training, which is performed using mean and standard deviation of the values in the current batch. This helps to reduce the difference in input parameters (obtained from a variety of input samples) to each layer. Siamese Networks performs pairwise learning and learns to find the similarity between the pair of images as opposed to a traditional CNN that learns to classify a single image predominantly via loss function like cross-entropy loss.

Although accuracy is often the primary evaluation metric for the performance of classification problems, to quantify the performance of our network\textquotesingle s classification ability, we use other metrics such as Precision, Recall, and F1-score. These metrics provide detailed insights based on the performance of the classifier for each class. The precision P (purity) and recall R (completeness) are defined as follows:

\begin{equation} \label{precision}
    \begin{gathered}
        Precision (P) = \frac{TP}{TP + FP}
    \end{gathered}
\end{equation}

\begin{equation} \label{recall}
\begin{gathered}
Recall (R) = \frac{TP}{TP + FN}
\end{gathered}
\end{equation}
\newline
The F1 score is interpreted as the weighted average of precision and recall. The accuracy is the overall accurate prediction by the classifier across all the classes.

\begin{equation} \label{F1-score}
\begin{gathered}
F1-score = \frac{2 \times \text{Precision} \times \text{Recall}}{Precision + Recall}
\end{gathered}
\end{equation}

\begin{equation} \label{accuracy}
\begin{gathered}
Accuracy = \frac{TP + TN}{TP + FP + TN + FN}
\end{gathered}
\end{equation}
\newline
where TP=true positive, TN=true negative, FP=false
positive, FN=false negative.
\begin{description}
\item[$\bullet$] True Positive (TP) is when the source is predicted as `x' and it is actually `x'.
\item[$\bullet$] False Positive (FP) is when the source is predicted as `x' and it is not actually `x'.
\item[$\bullet$] True Negative (TN) is when the source is predicted as not `x' and it is actually not `x'.
\item[$\bullet$] False Negative (FN) is when the source is predicted as not `x' and it is actually `x'.
\end{description}
The precision, recall, F1 score, and accuracy metrics were calculated for both the validation and test data sets to assess the performance in each experiment with the corresponding model.
We also provide the confusion matrix, another tool to visualize the performance of the classifier for each experiment. Confusion matrix helps in calculation of the metrics like accuracy, precision, recall and thus F1-score per class.

We perform the experiments in two different settings: original imbalanced dataset and the balanced dataset obtained via upsampling of minority classes with respect to the majority class. Since we will be referring to two set of experiments frequently, we use the following notations. Version 1: original imbalanced dataset experiments, Version 2: balanced dataset experiments.

\subsubsection{\textbf{Few-Shot Learning experiments and results}}
We used the open source PyTorch \citep{paszke2019pytorch} framework, Python version 3.6.7 for our experiments and Nvidia P100 graphics processing unit (GPU) for the hardware support.

We implemented a customized data loader in PyTorch to feed input pairs to the Siamese Networks. We use the architecture described for Siamese networks which involves five convoluted layers with batch normalization and two fully connected layers with ReLU activation function. The loss function used here is Contrastive Loss Function \citep{Hadsell:2006:DRL:1153171.1153654}. In the testing phase, K-nearest neighbors technique is used to classify images. The test image is compared to the embeddings of all the images in training data (stored earlier) and the class of images that are closest to this image (k nearest neighbors) is assigned to the test image.

The performance of Siamese Networks on the original dataset provided an overall accuracy of 71.53\%. The performance metrics can be observed in Table~\ref{table:Test_siamese_original}. The true positive values per class can be observed in the Figure ~\ref{fig:confmat_siamese_orig}.

Further, we tried the same network on balanced dataset. The overall accuracy improved marginally to a value of 74.26\%. Performance metrics can be observed in Table~\ref{table:Test_siamese_balanced}. The details per class can be observed in Figure ~\ref{fig:confmat_siamese_balanced}.

\begin{figure}
\centering
 \includegraphics[width=1.15\linewidth]{figures/siamese_orig_grey.png}
 \caption{Siamese Networks, Dataset Version 1: test data confusion matrix.}
 \label{fig:confmat_siamese_orig}
\end{figure}

\begin{table}

    \centering
    \caption[Centre]{Test metrics: Siamese Networks, original dataset}
    \label{table:Test_siamese_original}
    \begin{tabular}{lcccr}
        \hline
        Class & Precision & Recall & F1-Score & Support\\
        \hline
        Bent & 53.41\% & 45.63\% & 49.21\% & 103\\
        Compact & 96.77\% & 90\% & 93.26\% & 100\\
        FRI & 67.2\% & 84\% & 74.67\% & 100\\
        FRII & 69.39\% & 67.33\% & 68.34\% & 101\\
        \hline
        avg/ total & 71.69\% & 71.74\% & 71.37\% & 404\\
        \hline
    \end{tabular}
\end{table}

\begin{figure}
\centering
 \includegraphics[width=1.15\linewidth]{figures/siamese_balanced_grey.png}
 \caption{Siamese Networks, Dataset Version 2: test data confusion matrix.}
 \label{fig:confmat_siamese_balanced}
\end{figure}

\begin{table}

    \centering
    \caption[Centre]{Test metrics: Siamese Networks, balanced dataset}
    \label{table:Test_siamese_balanced}
    \begin{tabular}{lcccr}
        \hline
        Class & Precision & Recall & F1-Score & Support\\
        \hline
        Bent & 61.03\% & 45.63\% & 52.22\% & 103\\
        Compact & 98.88\% & 89\% & 93.68\% & 100\\
        FRI & 66.41\% & 89\% & 76.06\% & 100\\
        FRII & 72.81\% & 74.26\% & 73.53\% & 101\\
        \hline
        avg/ total & 74.78\% & 74.47\% & 73.87\% & 404\\
        \hline
    \end{tabular}
\end{table}

Although the results of Siamese Networks in Few Shot Learning are encouraging, we wanted to see if the use of other more sophisticated techniques produces better results. We therefore switched to the Transfer Learning approach using the same input data.

\begin{table*}
    \centering
    \caption[Centre]{DenseNet-161 architecture}
    \label{table:densenet-161}
    \begin{tabular}{lccccccr}
        \hline
        \# Layer & Function & Input tensor size & Activation & Stride & Output tensor size & No. of parameters & \# Block \\
        \hline

        0 & Input & 3 x 224 x 224 & - & - & - & 0 & \\

        1 & \textbf{conv} & 3 x 224 x 224 & ReLU & 2 & 96 x 112 x 112 & 14112 & - \\
        1 & pooling & 96 x 112 x 112 & - & 2 & 96 x 56 x 56 & 0 & \\
        \hline

        2-13 & [1 x 1 \textbf{conv}, 3 x 3 \textbf{conv}] x 6 & 96 x 56 x 56 & ReLU & 2 & 48 x 56 x 56 & 82944 & 1 \\
        \hline

        14 & \textbf{conv} & 384 x 56 x 56 & ReLU & 2 & 192 x 56 x 56 & 73728 & - \\
        14 & pooling & 192 x 56 x 56 & - & 2 & 192 x 28 x 28 & 0 & \\
        \hline

        15-38 & [1 x 1 \textbf{conv}, 3 x 3 \textbf{conv}] x 12 & 192 x 28 x 28 & ReLU & 2 & 48 x 28 x 28 & 82944 & 2 \\
        \hline

        39 & \textbf{conv} & 768 x 28 x 28 & ReLU & 2 & 384 x 28 x 28 & 294912 & - \\
        39 & pooling & 384 x 28 x 28 & - & 2 & 384 x 14 x 14 & 0 & \\
        \hline

        40-111 & [1 x 1 \textbf{conv}, 3 x 3 \textbf{conv}] x 36 & 384 x 14 x 14 & ReLU & 2 & 48 x 14 x 14 & 82944 & 3 \\
        \hline

        112 & \textbf{conv} & 2112 x 14 x 14 & ReLU & 2 & 1056 x 14 x 14 & 2230272 & - \\
        112 & pooling & 1056 x 14 x 14 & - & 2 & 1056 x 7 x 7 & 0 & \\
        \hline

        113-160 & [1 x 1 \textbf{conv}, 3 x 3 \textbf{conv}] x 24 & 1056 x 7 x 7 & ReLU & 2 & 48 x 7 x 7 & 82944 & 4 \\
        \hline

        161 & BatchNorm2d & 48 x 7 x 7 & - & - & 2208 x 7 x 7 & 4416 & - \\
        161 & avgpool & 2208 x 7 x 7 & - & 2 & 2208 x 1 x 1 & 0 & - \\
        161 & maxpool & 2208 x 1 x 1 & - & 2 & 2208 x 1 x 1 & 0 & - \\
        161 & Flatten & 2208 x 1 x 1 & - & - & 4416 & 0 & - \\
        161 & BatchNorm1d & 4416 & - & - & 4416 & 8832 & - \\
        161 & Dropout & 4416 & - & - & 4416 & 0 & - \\
        161 & Linear & 4416 & ReLU & - & 512 & 2261504 & - \\
        161 & BatchNorm1d & 512 & - & - & 512 & 1024 & - \\
        161 & Dropout & 512 & - & - & 512 & 0 & - \\
        161 & Linear & 512 & - & - & 4 & 2052 & - \\

        \hline
    \end{tabular}
\end{table*}

\subsection{Transfer Learning}

Transfer learning \citep{pratt1993discriminability} is a technique in machine learning which focuses on storing the knowledge gained while solving one problem and applying it to a different but related problem (See \citet{2018arXiv180801974T} for an
overview of deep transfer learning). The definition of transfer learning is given in terms of domain and task. The domain D consists of: a feature space X and a marginal probability distribution $P(X)$, where $X = {x_1,...,x_n}$ belong to $X$. Given a specific domain, $D = \{X, P(X)\}$, a task $T$ consists of two components: a label space $Y$ and an objective predictive function $f$ (denoted by $T = \{Y, f\}$), which is learned from the training data consisting of pairs $\{x_i, y_i\}$, where $x_i$ belongs to $X$ and $y_i$ belongs to $Y$. The function $f$ can be used to predict the corresponding label, $f(x)$, of a new instance $x$ \citep{pan2009survey}. In retrospect, transfer learning is the ability to utilize existing knowledge from the source learner in the target task.\\

\noindent\textbf{Pre-trained models}\\
One important requirement for transfer learning is the availability of models that perform well on source tasks. Pre-trained models are trained on a large amount of data with a variety of classes. The weights of such a network are set to obtain the low-level features like shapes, edges identified by the initial layers of the network efficiently. In this work, we use the method of transfer learning by loading weights pre-trained on ImageNet dataset and the architecture used is DenseNet and its variants. We also worked with some other architectures than the final one used here with no improvement in the performance, but we do not claim to have explored the complete parameter set.

We could not get satisfactory results with a custom convolutional neural network architecture and thus opted for DenseNet architecture.

\subsubsection{DenseNet Architecture}
In Convolutional Neural Networks, the input image goes through multiple convolutions to obtain high-level features.
Feature maps are the collective output of the convolution operation applied over the whole input image by all the convolution filters. Skip connections are the connections to the current layer from layers other than the immediately preceding layer. In DenseNet, the feature maps are concatenated from each of the previous layers along with the addition of the skip connections
obtained from the previous layers. Thus, each layer receives collective knowledge from all the previous layers. Traditional CNN with `L' layers have `L' connections between the subsequent layers whereas DenseNet has $L\times(L+1)/2$ direct connections. This helps to avoid the learning of redundant feature maps and a reduction in the requirement of total parameters of the network. The important element in DenseNet is the dense block which is composed of the Batch Normalization layer \citep{pmlr-v37-ioffe15}, ReLU activation function \citep{nair2010rectified} and $3\times3$ convolution layer and the dimensions of the feature maps remain constant within a block. The dense blocks contain transition layers in between them consisting of Batch Normalization, Convolution, and Pooling layers. Growth rate (K) of the network due to concatenation of the feature maps at each layer `L' can be formulated as $K(L) = K(0) + K\times(L-1)$ and it regulates the amount of information added to the network at each layer. At the end of the last block, global average pooling is performed and then a softmax activation is attached to get the final result.

\smallskip
The DenseNet architecture has two important advantages:
\begin{enumerate}
    \item The availability of connections between all the layers helps to propagate the error or the gradient more directly. The earlier layers can get feedback from the final layer rapidly.
    \item The input received by the current layer has knowledge from all the preceding layers leading to diversified features and richer patterns. This helps to obtain smoother decision boundaries. It also explains why DenseNet performs well when training data are insufficient - the central insight for our experiments.
\end{enumerate}

\subsubsection{\textbf{Experimental setup and parameter selection}}
\label{parameters}
We used the Fastai \citep{howard2018fastai} library, Python version 3.6.7 for our experiments and Nvidia P100 graphics processing unit (GPU) for the hardware support. We set the batch size of 32 and the image size of $224\times224$, the data samples are normalised using the ImageNet stats.
We use the learning rate finder method based on the cyclical learning rates \citep{2015arXiv150601186S}
to obtain the optimal learning rate. This method performs a mock training run starting with a smaller value for the learning rate which is modified after each batch. It searches the maximum learning rate at which there is still some improvement in terms of a decrease in the error. The learning rate either increases or decreases based on the outcome from the latest batch in the epoch. Large neural networks take a considerable amount of time for the training and the choice of the learning rate substantially affects this process. Using cyclical learning rates technique helps in the selection of a greater value for learning rate while still maintaining the improvement of the model. Based on the data distribution, after using the learning rate finder method, we set the learning rate for each experiment individually. We use a weight decay of $1e-3$ and categorical cross-entropy (equation \ref{CELoss}) as the cost function for all the tasks. In equation \ref{CELoss}, x,y denote the classes \& observations respectively, $t_{x,y}$ represent targets and $p_{x,y}$ represent the predictions. We have limited the training to 10 epochs for our experiments.
\begin{equation} \label{CELoss}
    \begin{gathered}
        L_x = - \sum_{y}t_{x,y} log(p_{x,y})
    \end{gathered}
\end{equation}

The growth rate {\it k}  in the DenseNet is the number of output feature maps of a layer, plays an important role in propagating the information through the network. We tried different versions of the DenseNet architecture and the best performance for our task was shown by the DenseNet-161 version. Hereafter, we focus on fine-tuning the selected version. The number 161 refers to the total number of layers in the particular version of the architecture. This can be described as follows: DenseNet-161 consists of 4 dense blocks and hence 3 transition layers between the blocks. There is one convolution layer at the beginning and one classification layer (fully connected layer) at the end. The four dense blocks consist of 12, 24, 72, 48 `conv' layers respectively where `conv' layer is combined BatchNorm, ReLU activation and Convolution. Table \ref{table:densenet-161} describes the functions and the classification head at the end for the model. While the total number of parameters in DenseNet-161 is 28,745,412 freezing initial layers and training only the last layer reduces the number of trainable parameters to 2,493,348.

Initially, we train the last layer for 1-2 epochs. The intuition behind training this way is to train only the fully connected layers completely and freezing the other layers for faster training. This way, we use the weights obtained from the pre-trained model and check the performance of the model. Gradually unfreezing the layers works better than unfreezing and retraining all layers of the network \citep{yosinski2014transferable, DBLP:journals/corr/abs-1801-06146}.
Here, we use a set of learning rates instead of a single value to train the network. This technique known as discriminative learning \citep{howard2018universal}, is used to obtain the best outcome out of different layers which might need different hyperparameters as per the feature maps in the specific layer. Discriminative learning uses the idea to train the earlier layers with a very low learning rate and the later layers with higher learning rates. This helps to avoid altering the earlier weights and massively affect the end layer weights which are more crucial for the specific case. To update the weights in the network, we require an optimizer and here we use the method of AdamW \citep{loshchilov2018decoupled} to achieve super-convergence \citep{2017arXiv170807120S} which leads to fast training over the data sample even with large architectures such as the one used for our experiments - DenseNet. With the use of a small scale dataset, the model is prone to overfitting as the decision boundaries are based on a few specific samples, here using an efficient regularization technique helps to avoid the overfitting of the model.

\section{Experiments and results}
\label{sec:expresults}
\subsection{Version 1: Original set}

\begin{figure}
  \includegraphics[width=1.0\linewidth]{figures/161-lr-bw.png}
  \caption{Graph of learning rate finder to get optimal learning rate for DenseNet-161 version 1, the black point in the graph depicts the learning rate suggested by the lr\_find method.}
  \label{fig:learningrate_1}
\end{figure}

From Figure~\ref{fig:learningrate_1},
we observe that the learning rate between the range $1e-4$ to $1e-3$ follows a non-increasing curve and can be used for optimal performance. Using this learning rate and a weight decay of $1e-3$ we trained Densenet-161 for 10 epochs. The model required 1 minute 10 seconds per epoch leading to a total time of approximately 12 minutes for training.

The training loss starts with a value of 0.6056 (Figure~\ref{fig:lossVsbatch_v1}) and later follows a downward path with the increase in the number of batches processed. The validation loss starts with a value above 1 (1.07) and later decreases steadily. Near the end of the batch processing, the training and the validation loss reach a value of 0.17 and 0.25 respectively.

\begin{figure}
\centering
 \includegraphics[width=1.0\linewidth]{figures/161-trval-grey.png}
 \caption{Plot of training and validation losses as a function of batches processed, DenseNet-161 version 1.}
\label{fig:lossVsbatch_v1}
\end{figure}

This model performed well on the validation set and the top losses can be observed in some common cases where FRII in predicted as Bent in Figure~\ref{fig:TopLosses_v1}. Individual class metrics for validation set can be observed from Table~\ref{table:valid_v1}.

\begin{table}

    \centering
    \caption[Centre]{Validation metrics: DenseNet-161 version 1.}
    \label{table:valid_v1}
    \begin{tabular}{lcccr}
        \hline
        Class & Precision & Recall & F1-Score & Support\\
        \hline
        Bent & 81.57\% & 93\% & 86.91\% & 100\\
        Compact & 100\% & 100\% & 100\% & 80\\
        FRI & 95.08\% & 78.37\% & 85.92\% & 74\\
        FRII & 92.25\% & 91.61\% & 91.93\% & 143\\
        \hline
        Mean over all classes & 92.22\% & 90.74\% & 91.19\% & 397\\
        \hline
    \end{tabular}
\end{table}

The performance of the model on the test set provided an overall accuracy of 92.3\% The accuracy values per class can be observed from the Figure~\ref{fig:confmat_v1}. Individual class metrics for test set can be observed from Table~\ref{table:Test_v1}.

\begin{figure}
\centering
 \includegraphics[width=1.15\linewidth]{figures/d161-v1_grey.png}
 \caption{DenseNet-161 version 1: test data confusion matrix.}
 \label{fig:confmat_v1}
\end{figure}

\begin{table}

    \centering
    \caption[Centre]{Test metrics: DenseNet-161 Version 1.}
    \label{table:Test_v1}
    \begin{tabular}{lcccr}
        \hline
        Class & Precision & Recall & F1-Score & Support\\
        \hline
        Bent & 85.21\% & 95.14\% & 89.90\% & 101\\
        Compact & 98.95\% & 95\% & 96.93\% & 100\\
        FRI & 91.92\% & 91\% & 91.46\% & 100\\
        FRII & 94.68\% & 88.12\% & 91.28\% & 103\\
        \hline
        Mean over all classes & 92.69\% & 92.31\% & 92.39\% & 404\\
        \hline
    \end{tabular}
\end{table}

\begin{figure}
\centering
 \includegraphics[width=1.0\linewidth]{figures/losses-161-mag.png}
 \caption{The top losses and incorrect classification by DenseNet-161 Version 1.}
 \label{fig:TopLosses_v1}

\end{figure}

\subsection{Version 2: balanced dataset}

\begin{figure}
\centering
 \includegraphics[width=1.0\linewidth]{figures/aug-lr-bw.png}
 \caption{Graph of learning rate finder to get optimal learning rate for DenseNet-161 Version 2.}
 \label{fig:aug-lr_v2}
\end{figure}

In order to improve the performance, we tried with the balanced data sample (details in section~\ref{sec:data}). Based on the learning rate finder method,
here, the learning rate between the range $3e-5$ to $2e-4$ (Figure~\ref{fig:aug-lr_v2}) follows a decreasing curve and can be used for optimal performance. Using this learning rate and a weight decay of $1e-3$ we trained the Densenet-161 pre-trained model with the upsampled classes in the dataset for 10 epochs. The model took a time of 1 minute 38 seconds per epoch leading to a total time of approximately 16 minutes for training.

\begin{figure}
\centering
 \includegraphics[width=1.0\linewidth]{figures/balanced_loss_gray.png}
 \caption{Plot of training and validation losses as a function of batches processed, DenseNet-161 Version 2.}
 \label{fig:lossVsbatch_v2}
\end{figure}

The training loss starts with a value of 0.5416 (Figure~\ref{fig:lossVsbatch_v2}) and later follows a steadily downward path with the increase in the number of batches processed. It reaches a minimum of 0.1588 at the end of the last epoch. The validation loss starts with a value of 0.5116, reaches a minimum value of 0.2598 near the end of the batch processing.

This model performed very well on the validation set with slightly better performance than Version 1 of the DenseNet-161 experiment. The few cases where the sources are classified incorrectly are shown in Figure~\ref{fig:aug_losses_v2}.

\begin{figure}
\centering
 \includegraphics[width=1.0\linewidth]{figures/losses-161-balanced.png}
 \caption{The top losses and incorrect classification by DenseNet-161 Version 2.}

 \label{fig:aug_losses_v2}
\end{figure}

\begin{table}

    \centering
    \caption[Centre]{Validation metrics: DenseNet-161 Version 2.}
    \label{table:Valid_v2}
    \begin{tabular}{lcccr}
        \hline
        Class & Precision & Recall & F1-Score & Support\\
        \hline
        Bent & 84.90\% & 93.75\% & 89.10\% & 144\\
        Compact & 98.61\% & 98.61\% & 98.61\% & 144\\
        FRI & 93.23\% & 86.11\% & 89.53\% & 144\\
        FRII & 92.14\% & 89.58\% & 90.84\% & 144\\
        \hline

        Mean over all classes & 92.22\% & 92.01\% & 92.02\% & 576\\
        \hline
    \end{tabular}
\end{table}

\begin{figure}
\centering
 \includegraphics[width=1.15\linewidth]{figures/d161-v2_grey.png}
 \caption{DenseNet-161 version 2 (balanced dataset): test data confusion matrix.}
 \label{fig:confmat_v2}
\end{figure}

The performance of the model on the test set provided an overall accuracy of 92.82\%. The accuracy values per class can be observed from the Figure~\ref{fig:confmat_v2}. Individual class metrics for test set can be observed from Table~\ref{table:Test_v2}.

\begin{table}

    \centering
    \caption[Centre]{Test metrics: DenseNet-161 Version 2.}
    \label{table:Test_v2}
    \begin{tabular}{lcccr}
        \hline
        Class & Precision & Recall & F1-Score & Support\\
        \hline
        Bent & 87.5\% & 88.35\% & 87.93\% & 101\\
        Compact & 99\% & 99\% & 99\% & 100\\
        FRI & 93\% & 93\% & 93\% & 100\\
        FRII & 92\% & 91.09\% & 91.54\% & 103\\
        \hline
        Mean over all classes & 92.87\% & 92.86\% & 92.87\% & 404\\
        \hline
    \end{tabular}
\end{table}

\section{Discussion and Conclusion}
\label{sec:discussion}

The important part of our work has been the focus on getting accurate classification with few training examples. This factor mainly depends on the task and its complexity. Naturally, a model learns to generalize better with a wide variety of examples rather than just a large number of examples. Deep learning algorithms generate the features from the input data at runtime (training process) and thus avoids the need for feature engineering. To generate useful features, these algorithms require a large amount of data compared to traditional machine learning algorithms. The general idea is to manage the sample size and architecture coherently. In our task, since the number of classes is very less, the number of different parameters per class has not been too high, and thus even with less than 1000 images per class, we have achieved accuracy above 92\%. The use of techniques like cyclical learning rate, AdamW optimizer to cope with the less amount of data have helped us to obtain good results with short training time. Though DenseNet has a large number of parameters (section \ref{parameters}), our training methods help to converge the model quickly, leading to a low cost of training.

In comparison to previous methods, our method outperforms some of these while showing significantly close performance to the other methods. The classification task closely resembles with the work by \citet{wathela18} (similar 4 classes - FRI, FRII, Bent, Compact for classification) in which the authors achieved an overall accuracy above 94\% while using more than 21000 samples with 400 epochs for training. In this work, we obtain a close performance with just over 2708 samples in total and 10 epochs for training. \citet{aniyan17} obtained average precision and recall values of 88\% and 86\% respectively in their 3 class (FRI, FRII and Bent) classification task using above 25000 samples with augmentation, whereas our classifier obtains average precision of 93.11\% and recall of 92.82\% for the 4 class classification. \citet{ma19} achieved values of 92\% and 84\% for precision and recall respectively for the 6 class classification task in their two-step approach. Our classifier provides better values for overall precision and recall. \citet{2018MNRAS.476..246L} performed 4 class (Compact, Single extended, Two-Component extended and Multi-component extended) classification and obtained an overall accuracy of 94.8\% while using a total of 141032 samples via augmentation (21933 original samples) for 4 classes. \citet{wu19} used object detection and classification techniques and obtained mean average precision (mAP) of 84\% while using more than 10000 samples. \citet{tang19} implemented transfer learning approach while maintaining the idea of domain transfer in their binary classification task (FRI and FRII) and even with the domain transfer approach, authors achieved overall accuracy of 90.1\% and 83.9\% for FIRST and NVSS samples respectively. \citet{10.1093/mnras/stz1289} performed a 3 class (Unresolved, FRI and FRII) classification using a model pre-trained on ImageNet dataset and 15936 image samples in total to obtain overall precision of 94.4\%.

Few shot learning methods are being developed specifically to make the models efficient in terms of data usage while providing similar results to experiments with larger datasets. In our case, Siamese networks did not perform well but the results are promising. With new techniques being developed in the domain of FSL, better performance can be expected.

Our experiments are driven by the idea to get the best possible performance with as little data as possible. The classifier presented here does not outperform all the previous attempts but we obtain comparable performance with limited data. Convolutional Neural Networks have traditionally worked well with large amounts of data but the recent developments in the training methods have helped to get significant outcome even with fewer samples in the dataset.

The concatenation technique for feature maps in DenseNet helps to capture diverse characteristics rapidly and thus the architecture works well with less amount of data. This has been the primary motivation to utilize DenseNet in our experiments.

Since we are using a network that has already seen a lot of images and learned to distinguish between the classes, we can teach this network to classify the new classes with few examples. The technique has been applied mostly in the same domain tasks but the use of optimal architecture and discriminative learning has helped to obtain better results in a different domain here. \cite{wu19} have mentioned regarding the possible negative impact of transfer learning when applied for different domain transfer based on the suggestions from \cite{pan2009survey}. \cite{Rosenstein2005TransferLW} suggested using an ensemble of tasks A for a better transfer to task B mainly due to strong feature space creation. The idea of negative transfer does not practically hold true as seen from our experiments.

The use of data augmentation technique provided only a marginal improvement in our experiments but here we use augmentation only to remove the class imbalance and not extensively to increase the data sample size as performed by previous classifiers.

In both the methods applied in our approach, we observed that the `Bent' category has shown the highest confusion. In the incorrect cases, the FRII samples are classified as Bent and vice-versa. As suggested by \cite{aniyan17} this is most likely because many Bent galaxies exhibit FRII like appearance and hence can be classified incorrectly as such.

The learning rate finder method based on cyclical learning has provided a way to adapt to the optimal learning rate parameter for all the experiments. The general idea to set the batch size has an aspect of the computational architecture linked to it and thus we set the batch size as a power of 2 i.e. 32, 64 and so on. Larger batch size ($>$ 64) can lead to memory overflow but it depends on the network architecture and the data sample. We have maintained a batch size of 32 throughout the experiments. In practice, the batch size and the number of epochs are handled interdependently. The batch size of 32 along with 10 epochs instead of a batch size of 64 and less number of epochs worked better in our experiments. Another important factor, the image size is set as $224\times224$ for our experiments based on the applications of the DenseNet architecture for ImageNet dataset. In order to utilize the weights of the DenseNet obtained for ImageNet dataset, using the ImageNet stats for our data is highly encouraged. The optimizer used in our experiments - AdamW is the improvement over the original Adam \citep{kingma2014method} optimizer and performs better for computer vision tasks. The hyperparameter setup worked best for our case as per the hardware and GPU support but it can be adjusted for different and possibly better results.

Our system can be very useful to classify the objects of any current or future survey. The dataset, experiments and the models are available at \href{https://github.com/kiryteo/RG_Classification_code}{https://github.com/kiryteo/RG\_Classification\_code}. Users of the system can try to get their own small and curated dataset for the training, tweak with the hyperparameters mentioned above get their own results. Since we are providing the trained model for the inference, users can try using the model as a pre-trained model, perform hyperparameter tuning and get improved results, which is consistent with the domain transfer idea from transfer learning. With the ongoing research for the optimization of neural network architectures, applying the techniques presented in the paper with another architecture can lead to independent results with the same training data.

\bsp
\label{lastpage}
\bibliographystyle{mnras}

Instrumentation and Methods for Astrophysics (astro-ph.IM); Machine Learning (cs.LG)


Quark-Gluon Plasma

Abstract: An elementary introduction to the physics of quark-gluon plasma is given. We start with a sketchy presentation of the Quantum Chromodynamics which is the fundamental theory of strong interactions. The structure of hadrons built up of quarks and gluons is briefly discussed with a special emphasis on the confinement hypothesis. Then, we explain what is the quark-gluon plasma and consider why and when the hadrons can dissolve liberating the quarks and gluons. The heavy-ion collisions at high-energies, which provide a unique opportunity to get a droplet of the quark-gluon plasma in the terrestrial conditions, are described. We also consider the most promising experimental signatures of the quark-gluon plasma produced in nucleus-nucleus collisions. At the end, the perspectives of the quark-gluon plasma studies at the future accelerators are mentioned.

\section{Introduction}

The quark-gluon plasma is a state of the extremely dense matter with
the quarks and gluons being its constituents. Soon after the Big Bang
the matter was just in such a phase. When the Universe was expanding
and cooling down the quark-gluon plasma turned into hadrons - neutrons
and protons, in particular -  which further formed the atomic nuclei.
The plasma is not directly observed in Nature nowadays, but the astrophysical
compact objects as the neutron stars may conceal the quark-gluon nuggets
in their dense centers. The most exciting however are the prospects to study
the plasma in the laboratory experiments. A broad research program of
the heavy-ion collisions, which provide a unique opportunity to produce
the quark-gluon droplets in the terrestrial conditions, is underway.
While the question, whether the plasma is already produced with
the currently operating heavy-ion accelerators, is right now vigorously
debated, there are hardly any doubts that we will have a reliable evidence
of the plasma within a few years. Then, there will be completed the
accelerators of a new generation: the Relativistic Heavy Ion Collider
(RHIC) at the Brookhaven National Laboratory and the Large Hadron Collider
(LHC) at CERN.

The aim of this article is to give a very elementary introduction to the
physics of quark-gluon plasma. We start with a few words on the Quantum
Chromodynamics which is a fundamental theory of strong interactions.
Then, the structure of hadrons, which are built up of quarks and gluons,
is briefly discussed. A particular emphasis is paid on the confinement
hypothesis. In the next section we try to explain why and when the
hadrons can dissolve liberating the quarks and gluons from their interiors.
Finally, the generation of the quark-gluon plasma in heavy-ion collisions
is considered.

There is a huge literature on every topic touched in this article.
Instead of citing numerous original papers, we rather recommend two
collections of the review articles \cite{Hwa90,Hwa95}. The more recent
progress in the field can be followed due to the proceedings of the
regular ``Quark Matter" conferences \cite{Pro93,Pro95,Pro96}.

Throughout the article we use the natural units where the velocity of
light $c$, the Planck $\hbar$ and Boltzmann $k$ constants
are all equal unity. Then, the mass, momentum and temperature have the
dimension of energy, which is most often expressed in MeV. The distances
in space or time are either measured in the length units, which are
usually given in fm $( 1\;{\rm fm} = 10^{-13}\:{\rm cm})$, or in the
inverse energy units. One easily recalculates the length into the
inverse energy or vice verse keeping in mind that
$\hbar c = 197.3 \;\; {\rm MeV\!\cdot fm}$.

\section{Quantum Chromodynamics}

Quantum Chromodynamics (QCD) strongly resembles the Quantum
Electrodynamics (QED). While QED describes the interaction of electric
charges - usually electrons and their antiparticles positrons - with the
electromagnetic field represented by photons, QCD deals with the
quarks and gluons corresponding to, respectively, the electrons and
photons. The quarks are, as electrons, massive and curry a specific
charge called color, which is however not of one but of three types:
red, blue and green. The gluons, which are, as photons, massless,
but not neutral. In contrast to photons, they curry color charges being
the combinations of the quark ones. The electromagnetic interaction
proceeds due to the photon exchanges. Analogously the quarks
interact exchanging the gluons. Although the photons being neutral
cannot interact directly with each other, there are forces acting
between gluons.

QCD has emerged as a theory of quarks and gluons which built up
the hadrons i.e. strongly interacting particles such as neutrons and
protons. However, there is no commonly accepted model of the hadron
structure. The difficulty lies in the very nature of the strong interaction -
its strength. While the perturbative expansion, where the noninteracting
system is treated as a first approximation, appear to be the only effective
and universal computational method in the quantum field theory, a large
value of the QCD coupling constant  excludes applicability of the method
for the system of quarks and gluons. However, QCD possesses a remarkable
property called the {\it asymptotic freedom}. The coupling constant $\alpha_s$
effectively depends on the four-momentum $Q$ transferred in the interaction as
\begin{equation}\label{alpha}
\alpha_s(Q) = {12 \pi \over (33 - 2N_f ) \, {\rm ln}Q^2/\Lambda_{QCD}^2} \;,
\end{equation}
where $N_f$ is the number of the number of the quark flavours (types) and
$\Lambda_{QCD}$ is the QCD scale parameter, $\Lambda_{QCD} \cong 200$
MeV. Eq. (\ref{alpha}) shows that the coupling constant is small when
$Q^2 \gg \Lambda_{QCD}^2$. Therefore, the interactions with a large
momentum transfer can be treated in the perturbative way. QCD appears to
be indeed very successful in describing the hard processes, such as a
production of jets in high energy proton-antiproton collisions, which
proceed with high $Q$.

\section{Hadron structure}

The description of the soft processes in QCD, which, in particular, control
the hadron structure, remains a very serious unresolved  problem of the strong
interactions. One has to rely on the phenomenological models, the validity of
which can be only tested by confrontation against the experimental data.
Dealing with the soft QCD one often uses the concept of the constituent
quarks which should be distinguished the current quarks. The latter ones are
the fundamental elementary spin 1/2 particles which are present in the QCD
lagrangian. The current up ($u$) and down ($d$) quarks are light with the
mass of a few MeV. The constituent $u$ and $d$ quarks are supposed to be
the effective quasiparticles with the (large) masses of  about 300 MeV
(1/3 of the nucleon) generated by the interaction. The difference between
the current and constituent strange ($s$) quark masses is less dramatic.
They are, respectively, about 150 and 450 MeV. In the case of heavy
quarks - charm ($c$), bottom ($b$) and top ($t$) - the distinction is no
longer valid.

Within the constituent quark model the baryon is described as three
bounded quarks. The meson is then the system of quark and antiquark.
In terms of current quarks the hadron is seen as a cloud of quarks
and gluons. The baryon is then no longer built of three quarks and the
meson of quark and antiquark. Instead, the hadron curries the quantum
numbers of, respectively, three quarks or the quark and antiquark.
Therefore, the hadron is composed of the valence quarks (quark and
antiquark in the case of meson and three quarks for the baryon) and
the sea constituted by the quark-antiquark pairs and gluons.

The gluons are believed to glue together the quarks which form
the hadron, but a satisfactory theory of the hadron binding is still
missing. Such a theory has to explain the {\it hypothesis of confinement}
which is the fundamental element of our understanding of the hadron
world. While the electric charges tend to form electrically neutral atoms
and molecules, in the case of the chromodynamic interactions there
seems to be a strict rule that the color charges occur only within the
white configurations. The existence of  the separated color objects as
quarks and gluons is excluded. They must be confined in the colorless
systems such as hadrons. The three quarks forming a baryon curry three
fundamental colors which give all together the white object. In the
meson case the quark color is complementary to the color of the antiquark.
In principle, the confinement hypothesis allows for the existence of not
only the meson (quark-antiquark) and baryon (three quark) configurations
but for any white one such as a dibaryon which is the six quark system.
However, in spite of hard experimental efforts, the reliable evidence for
the hadrons different than baryons and mesons is lacking.

There are many phenomenological approaches to the confinement. Let us
briefly present here the string model inspired by the Meissner effect which
is the expelling of the magnetic field from the superconducting material.
The model assumes that the vacuum behaves as a diaelectric medium,
where the chromodynamic field cannot propagate but is confined in thin
tubes or strings which connect the field sources. In Fig. 1 we show the
electric field generated by the two opposite charges which are in the (normal)
vacuum (a) and in the diaelectric medium (b). Let us compute the potential
which acts between charges in the latter case. Using the Gauss theorem one
immediately finds the electric field as $E = q /\sigma $, where $q$ denotes
the charge and $\sigma $ the cross section of the tube. If  $\sigma$ is
independent of  the distance $r$ between the charges, their potential energy
equals
\begin{equation}\label{linear}
V(r) = {q^2 \over \sigma} \, r \;.
\end{equation}
As seen the potential energy grows lineary with $r$ when the charges are
put in the diaelectric medium.

Keeping in mind the result (\ref{linear}), the confinement hypothesis,
which is illustrated in Fig. 2, can be understood as follows. Let us
imagine that one tries to burst the meson up separating the quark from
the antiquark. Stretching the meson requires pumping of the energy
to the system. When the energy is sufficient to produce the quark-antiquark
pair, the string breaks down and we have two mesons instead of one.

At the end of this section we mention another commonly used model of the
hadron structure the bag model. One assumes here that the vacuum exerts
the pressure $B$ on the (colored) quarks and gluons. Then, the hadron is
the bag with quarks and gluons similar to the bubble of vapour in the liquid.
The bag is usually spherical but the deformations are possible. The model
appears to be really successful in describing a very reach mass spectrum
of hadrons.

\section{Quark-Gluon Plasma}

The quark-gluon plasma is the system of quarks and gluons, which are no
longer confined in the hadron interiors, but can propagate in the whole
volume occupied by the system. Thus, the quark-gluon plasma resembles
the ionized atomic gas with quarks and gluons corresponding to electrons
and ions and hadrons being the analogs of  atoms. One may wonder whether
the existence of quark-gluon plasma is not in conflict with the confinement
hypothesis. We note that the plasma is white as a whole. Thus, the color
charges are still confined in the colorless system. However, it is still
unclear why and when the hadrons can dissolve liberating quarks and gluons.
We will consider this issue in the next section. Here we would like to
discuss a somewhat unexpected consequence of the asymptotic freedom.

Due to the mentioned difficulties of the soft QCD, the properties of the
hadronic matter - either composed of hadrons or of quarks and gluons -
are poorly known at the moderate temperatures. The situation changes
qualitatively when the system temperature $T$ is  much larger than the
QCD scale parameter $\Lambda_{QCD}$. In this limit $T$ is the only
dimensional parameter which describes the system. In particular, it
determines the average momentum transfer in the interaction of quarks
and gluons. Therefore, $\langle Q^2 \rangle = c\, T^2$ where $c$ is a
dimensionless constant. On the basis of the dimensional argument we
expect to achieve the asymptotic freedom regime (smallness of the
coupling constant) when $T \gg \Lambda_{QCD}$. Then, the
quark-gluon plasma is a weakly interacting gas of massless quarks
and gluons.

The detailed calculations performed within the thermal QCD show that
the asymptotic freedom regime is indeed obtained in the high temeprature
limit. Here we will only add a simple physical argument to the dimensional
one used above. The momentum transfer, which enters the formula of the
running coupling constant (\ref{alpha}), corresponds (due to the Fourier
transform) to the distance between the interacting partons. The distance
is proportional to the inverse momentum transfer. When the plasma
temperature increases, the density of the quark-gluon system grows as
$T^3$ in the high temperature limit. (One can refer here again to the
dimensional arguments or simple calculations presented in
the next section.) Since the average particle separation in the gas of the
density $\rho$ equals $\rho^{-1/3}$, the average inter-particle distance
decreases with $T$ as $T^{-1}$. Consequently, the average coupling
constant vanishes when $T \rightarrow \infty $.

\section{Deconfinement phase transition}

The transformation of the hadron gas into the quark-gluon plasma is called
the deconfinement phase transition. There are many independent indications
that such a transition indeed takes place in the dense hadronic medium.
First of all one should mention the results obtained within the lattice
formulation of QCD, where the space continuum is replaced by the discrete
points. The Monte Carlo simulations show that there are two phases in the
lattice QCD, which are identified with the hadron and quark-gluon phase,
respectively. Here we would like to discuss simple arguments in favour
of the existence of the quark-gluon plasma.

We start with the observation that hadrons are not point-like objects
but their size is finite. The hadron radius is about 1 fm. So, let us
consider the hadron gas which is so dense that the average separation
of hadrons is about 1 fm. There is no reason to think  that the confining
potential still operates in such a medium at the distances which are
significantly larger than 1 fm.  When the quark and antiquark forming
a meson are pulled way from each other there is no vacuum, which
expels the chromodynamic field, but there are other hadrons all around.
Therefore,  the confining potential is expected to be screened at the
distances comparable to the average inter-particle separation.

There are two ways, as illustrated in Fig. 3,  to produce the dense
hadronic matter. The first one is evident - squeezing of the nuclear matter.
Since the baryon number, which is curried by neutrons and protons, is
conserved, the nucleons cannot disappear and will start to overlap when
the average inter-particle distance is smaller than the nucleon radius
$r_N \cong 1$ fm. The inter-nucleon separation is smaller than $r_N$
at the densities exceeding
$\rho = r_N^{-3} \cong 1 \; {\rm fm}^{-3} \cong 6 \: \rho_0$,
where $\rho_0 = 0.16 \; {\rm fm}^{-3}$ is the so-called saturation
or normal nuclear density being approximately equal the nucleon
density in the nucleus center.

The second method to produce the dense hadronic matter is to heat up
the nuclear matter or the hadron gas.  The point is that in the contrast
to the baryon or lepton number, which are the conserved quantities,
the particle number is not. Therefore, when the gas temperature (measured
in the energy units) becomes comparable to the particle mass, further
heating leads not only to the increase of the average particle kinetic energy
but to the growth of the average particle number. Of course, the particle
number growth cannot violate the conservation laws. Therefore, the particles,
which curry a conserved charge, must be produced in the particle-antiparticle
pairs. For example, to keep the baryon number of the system fixed we can
add to the system only pairs of the baryon and antibaryon. The essentially
neutral particles such as $\gamma$ or $\pi^0$, which do not curry conserved
charges, can be added to the system without restrictions, although the
average particle number is controlled by the equilibrium conditions. More
specifically, the numbers are determined by the minimum of the system free
energy when the system temperature and volume are fixed.

Let us now consider the gas of noninteracting pions. Its temperature
is assumed to be so high that the pions can be treated as massless particles.
(The approximation appears to work quite well even for the temperatures
which are close to the pion mass equal 140 MeV.) The pion density is then
\begin{equation}\label{rho-pi}
\rho_{\pi} =  \int {d^3 p \over  (2\pi )^3} {g_{\pi} \over e^{E/T} -1 }
= {g_{\pi} \zeta(3) \over \pi^2} \: T^3 \cong  0.73 \: T^3 \;,
\end{equation}
where $E \equiv |{\bf p}|$; $\zeta(z)$ is the Riemann function and
$\zeta(3) \cong 1.202$; $g_{\pi}$ is the number of the particle internal
degrees of freedom, which is 3 for the gas of  $\pi^+$, $\pi^0$ and
$\pi^-$. One immediately finds from (\ref{rho-pi}) that the inter-pion
separation is smaller than 1 fm for $T > 219$ MeV.

The deconfinement phase transition has been studied in the numerous
phenomenological approaches. We consider here the simplest possible
model, where the phase transition is assumed to be of the first order.
Then, one can apply the Gibbs criterion to construct the phase diagram.
The hadron phase is modeled by the ideal gas of  massless pions of three
species ($g_{\pi}=3$) while the quark-gluon one by the ideal gas of
quarks and gluons, which is baryonless i.e. the total baryon charge
vanishes due to the equal number of quarks and antiquarks. Let us compute
the number of the internal degrees of freedom in the quark-gluon gas. One
should distinguish here the fermionic degrees of freedom of quarks $g_q$
and the bosonic of gluons $g_g$. There are two light quark flavours
($u$ and $d$); there are quarks and antiquarks; we have two spin and
three color quark states. So, $g_q = 2 \times 2 \times 2 \times 3 = 24$.
The gluons are in two spin and eight color states, which give
$g_g = 2 \times 8 = 16$.

Since the pressure of the ideal gas of massless particles equals one third
of the energy density, the pressure exerted by the pion gas is
\begin{equation}\label{press-pi}
p_{\pi} = {1 \over 3} \int {d^3 p \, E \over  (2\pi )^3}
{g_{\pi} \over e^{E/T} -1 }
= {g_{\pi}  \pi^2 \over 90} \: T^4 \cong  0.33\: T^4 \;,
\end{equation}
while that of the quark-gluon plasma equals
\begin{equation}\label{press-qg}
p_{qg} = {1 \over 3} \int {d^3 p \, E\over  (2\pi )^3}
\bigg[ {g_g \over e^{E/T} -1 } + {g_q \over e^{E/T} + 1 }\bigg]
= \Big(g_g + {7 \over 8} g_q \Big)\, {\pi^2 \over 90} \: T^4
\cong  4.1\: T^4 \;.
\end{equation}

According to the Gibbs criterion the phase, which generates the higher
pressure at a given temperature, is realized. Then, one finds from
eqs. (\ref{press-pi}, \ref{press-qg}) that the pressure of the quark-gluon
plasma is always greater than that of pions. Therefore, we should have,
in conflict with the experiment, the quark-gluon phase at any temperature.
However, we have not taken into account the pressure exerted by the
vacuum on quarks and gluons. Subtracting the bag constant $B$ from the
r.h.s. of eq. (\ref{press-qg}), one finds that below the critical temperature
$T_c$ there is the pion gas and above the quark-gluon plasma. The critical
temperature is given as
$$
T_c = \Bigg[{ 90 B \over \pi^2 \big( g_g + {7 \over 8}g_q
- g_{\pi}\big)} \Bigg]^{1/4} \cong 0.72 \:B^{1/4} \;.
$$
Taking $B^{1/4} = 200$ MeV, we get $T_c = 144$ MeV.

In Fig. 4 we show a schematic phase diagram of the strongly interacting
matter. The baryon density is measured in the units of the normal nuclear
one. The point at $\rho = \rho_0$ and $T=0$ represents normal nuclei.
In fact, this is the only point in the diagram which is really well known
and understood. At $ \rho > \rho_0$ and $T = 0$ there is a region of
the dense nuclear matter with several exotic forms being suggested.
When the baryon density exceeds $2 - 3 \rho_0$ one expects
a transition to the quark-gluon phase. At even higher densities the plasma
is believed to be perturbative i.e. weakly interacting. So, one deals
with the quasi ideal strongly degenerated quark gas. The nuclear matter at
the temperature larger than a few MeV is traditionally called a hadron gas
being mostly composed of pions when the baryon density vanishes. Then,
we have quite reliable QCD lattice results which tell us that there is the
deconfinement phase transition at $T \cong 180$ MeV. With the significantly
larger temperatures we approach again the perturbative regime where the
plasma is weakly interacting. In the next section we discuss how the phase
diagram can be explored by means of the heavy-ion collisions.

\section{Heavy-Ion Collisions}

As already mentioned in the introduction, the heavy-ion collisions provide
a unique opportunity to study the quark-gluon plasma in the laboratory
experiments. More precisely, a drop of a dense hadronic matter can be
created in the collisions; the question whether the matter is for some
time in the deconfined phase is a separate issue.

The physics of heavy-ion collisions essentially depends on the collisions
energy. A good measure is not the whole energy of the incoming nucleus
but the energy per nucleon. The point is that at the energy of a few GeV
per nucleon, which is the lowest energy being interesting from the point
of view of the quark-gluon plasma, the nucleus does not interact
as a whole but there is an interaction of the overlapping parts of the
colliding nuclei as depicted in Fig. 5. The nucleons to be found in these
parts are called the {\it participants} while the remaining ones the
{\it spectators}.  One often distinguishes the central from the peripheral
collisions. In the latter ones, which proceed with a large value of the
impact parameter, most of the nucleons are spectators. In the case of the
central collisions basically all nucleons from the smaller nucleus (target
or projectile) are participants and the interaction zone is the largest.
Obviously the central collisions are the most interesting in the context
of the quark-gluon plasma searches. Unfortunately, the nucleus-nucleus
cross section is dominated by the peripheral collisions. The cross section
contribution of the collisions with a given impact parameter
$b$ is $2 \pi b db$. Therefore, the contribution vanishes when
$b \rightarrow 0$.

The high-energy nucleus-nucleus collision proceeds according to the
following scenario. The overlapping parts of the colliding nuclei
strongly interact and a dense and hot hadronic system, which is often
called a {\it fireball}, is created. Initially it is formed either
by the quarks and gluons or by the hadrons. Then, the system expands
and cools down. If the fireball matter has been initially in the deconfined
phase, the system experiences the hadronization i.e. the quarks and
gluons are converted into the hadrons. The fireball further expands.
When its density is so low that the mean free path of the hadrons
is close to the system size or the expansion velocity is comparable
the that of the individual particles, the whole system decouples into
the hadrons which do not interact with each other any longer.
However, the unstable particles can still gradually decay into the final
state hadrons. The moment of decoupling is called the {\it freeze-out},
because the particle momenta  are basically fixed at that time. Thus,
the final state hadrons, which are observed by experimentalists in
the particle detectors, characterize the fireball at the freeze-out.

As we discussed in Sec. V, the dense hadronic matter can be obtained
either due to the nuclear matter compression or the nuclear matter heating.
It appears that the significant effect of compression at the early stage of
heavy-ion collisions, which is still insufficient for the quark-gluon
generation, is observed at the relatively low energies, not larger than
1 GeV. Then, one can study the dense nuclear matter of rather small
temperature. At higher energies the atomic nuclei appear to be rather
transparent i. e. the colliding nuclei traverse each other. The participants
are only slightly deflected from their straight line trajectories due to
the interaction. However, they lose a sizable portion of their energy which
is further manifested in the form of produced particles, mainly pions.
Therefore, the baryon density of the produced hadronic system is not much
increased even at the early collisions stage. When the produced system
expands, the baryon density soon gets a value which is smaller than the
normal nuclear density. The strongly interacting matter however is
significantly heated up. If the temperature exceeds the deconfinement
transition temperature, the matter is expected to be in the quark-gluon
phase.

A comment is in order here. We use the concept of the temperature which
explicitly assumes that the system is in the thermodynamic equilibrium.
It is far not obvious that this is really the case. Any physical system needs
some time to reach the state of equilibrium. The hadronic matter produced
in heavy-ion collisions cannot achieve the {\it global} equilibrium but the
theoretical as well as experimental arguments suggest that the {\it local}
quasi equilibrium is possible. The global equilibrium is characterized by
the thermodynamic parameters which are unique for the whole system.
Before the global equilibrium is achieved a system is usually for some time
in the local equilibrium state. The system's parts are then already in the
equilibrium but the thermodynamic parameters - temperature, density,
hydrodynamic velocity - vary from part to part. The hadronic matter produced
in heavy-ion collisions is however not kept in any container but it immediately
starts to expand, mainly along the beam axis. Consequently there is a sizable
variation of the hydrodynamic velocity in this direction.

\section{Plasma Signatures}

The plasma creation is expected to occur at the early collision stage, but
it must hadronize i.e. experience the transition to the hadron gas, when
the matter is expanding and cooling down at the later stages. Therefore,
we always observe hadrons in the final state of the collisions and it is
really difficult to judge whether the plasma has been present or not.
Although it is hard to imagine a smoking gun proof, a few signatures of
the plasma creation has been proposed. We discuss below the two which
seem to be the most promising.

It has been argued that the presence of the plasma at the early collision
stage increases the number of strange particles observed in the final state.
The point is that the mass of strange quark mass appears to be significantly
smaller than that of strange particles. The strange $(s)$ quark mass is
about 150 MeV. Therefore, one needs 300 MeV to produce the $s-\bar s$
pair. The strange quarks must be, of course, produced in pairs because the
strangeness is conserved in the strong interactions. The most energetically
favorable way to produce the strangeness at the hadron level proceeds
in the reaction $\pi + N \rightarrow K + \Lambda $. Here the incoming
particles - the pion and nucleon - must curry over 500 MeV in the center
of mass frame. Thus, it is easier to produce strangeness at the quark-gluon
than hadron level. Once the strange quarks appear in the plasma, they cannot
disappear - a rather rare annihilation process of $s-\bar s$ pairs can be
neglected - and consequently, they are distributed among the final state hadrons.
Quantitative comparison of the two scenarios with and without the
plasma indeed shows that the presence of the plasma leads to the
significant strangeness enhancement.

The second plasma signatures deals with the $J/\psi$ particle or
charmonium which is a bound state of  the charm quark $c$ and
antiquark $\bar c$. The $J/\psi$ particle is expected to dissolve
much easier in the quark-gluon environment than in the hadron one.
This can be understood as a result screening of the potential, which
binds the $c$ and $\bar c$ quarks, by the color charges of the plasma
particles. Therefore, the number of the $J/\psi$ particles in the final
state should be significantly reduced if the plasma is present at the
collision early stage.

The two predicted quark-gluon plasma signatures have been indeed
experimentally observed in the central heavy-ion collisions at the
energy of about 200 GeV per nucleon which have been recently
studied at CERN. The whole set of the experimental data however
does not fit to the theoretical expectations. There have been
advocated the mechanisms of the strangeness enhancement
and $J/\psi$ suppression which are different than those mentioned
above. Therefore, it is a matter of hot debate whether the plasma
is produced at the energies which currently available. The plasma
generation at higher energies seems to be guaranteed.

\section{Perspectives}

In the near future the nucleus-nucleus collisions will be studied at the
accelerators of a new generation: Relativistic Heavy-Ion Collider (RHIC)
at Brookhaven and Large Hadron Collider (LHC) at CERN. The collision
energy will be larger by one or two orders of magnitude than that of
the currently operating machines. The heavy-ion experiments have been
performed till now in the beam-targets system where the accelerated ion
is smashed against the target nucleus which is initially at rest. RHIC
and LHC will use another principle - there will be accelerated two
intersecting ion beams. The energy of each beam will be 100 GeV
per nucleon at RHIC and 3 000 GeV at LHC. Thus, the collision
energy in the center of mass frame will equal, respectively, 200 and
6 000 GeV which should be compared to 20 GeV presently available
in the beam-target systems.

The proton-proton collisions have been already studied experimentally
at the energy domain of RHIC. The perturbative QCD, which is hardly
available at lower energies, is extensively used here. This is possible
because the average momentum transfer grows with the collision
energy and the QCD coupling constant is ten relatively small. Therefore,
the theoretical understanding of the collisions, paradoxically, improves
with the growing energy.

The nuclear collisions at RHIC or LHC are expected to be so violent
that the quarks and gluons comprising the nucleons will be easily
deconfined, for some time of course. At such huge energies the nucleon
is visualized as a cloud of partons which breaks up into the parton
showers as a result of collision with another nucleon. Thus, the creation
of the quark-gluon plasma seems to be unavoidable at RHIC and LHC.
The method of its detection however remains to a large extend an open
question.

\vspace{1cm}

I am very grateful to Marek Ga\'zdzicki for critical reading of the
manuscript.

Quark-Gluon Plasma