Na order and Co charge disproportionation in Na$_{x}$CoO$_{2}$

Abstract: We have synthesized and characterized different stable phases of sodium cobaltates Na$_{x}$CoO$_{2}$ with sodium content $0.65<x<0.80$. We demonstrate that $^{23}$Na NMR allows to determine the difference in the susceptibility of the phases and reveals the presence of Na order in each phase. $^{59}$Co NMR experiments give clear evidence that Co charge disproportionation is a dominant feature of Na cobaltates. Only a small fraction ($\approx$ 25%) of cobalts are in a non-magnetic Co$^{3+}$ charge state whereas electrons delocalize on the other cobalts. The magnetic and charge properties of the different Co sites are highly correlated with each other as their magnetic shift $K_{ZZ}$ scales linearly with their quadrupolar frequency $nu_Q$. This reflects the fact that the hole content on the Co orbitals varies from site to site. The unusual charge differentiation found in this system calls for better theoretical understanding of the incidence of the Na atomic order on the electronic structures of these compounds.

\section{Introduction}
\label{Introduction}
The influence of the dopant atoms on the electronic properties of conducting layers has initiated large debates in High Temperature Superconducting (HTSC) cuprates as well as in other complex layered oxides of transition elements. Whereas in many systems this influence is masked by miscellaneous effects, there are many experimental evidences that in the sodium cobaltates Na$_{x}$CoO$_{2}$ a large interplay between the Na atomic ordering and the electronic density on the Co sites occurs.

As HTSC cuprates the sodium cobaltates Na$_{x}$CoO$_{2}$ are layered oxide materials and the charge doping of the CoO$_{2}$ layers is controlled on a large range by variation of the Na content. This can be put in parallel with the doping of the cuprates by chemical substitutions on the layers separating the CuO$_{2}$ planes. One significant difference with the cuprates is that the Co of the CoO$_{2}$ plane are ordered on a triangular lattice and not on a square lattice as for the CuO$_{2}$ plane of the cuprates. In this configuration the large crystal field on the Co site favors a low spin state \cite{SinghPRB61} in which orbital degeneracy influences significantly the electronic properties and may yield large thermoelectric effects \cite{TerasakiTEP}. A rich variety of other physical properties ranging from ordered magnetic states \cite{Mendels05}, high Curie-Weiss magnetism and metal insulator transition \cite{Foo}, superconductivity \cite{TakadaNature} \emph{etc} have then been observed on the cobaltates. These differences with the cuprates have apparently recently stimulated an increased interest of theorists as monitored by recent publications \cite{Karim2014,Chubukov2014,Wilhelm2014}.

In sodium cobaltates the low spin configuration for the Co ions should corresponds to 3$^{+}$ or 4$^{+}$ charged states with spin $S=0$ and $S=\frac{1}{2}$ respectively. Many experiments and theoretical calculations have considered that the Co magnetism either is uniform or that there is an Co$^{3+}$/Co$^{4+}$ charge segregation with localized magnetic moments. However it has been evidenced by NMR by many groups that for $x\geq 1/2$ the Na$^{+}$ displays an atomic ordering associated with Co charge disproportionation and itinerant magnetism in the planes  \cite{CoPaper,ImaiPRL1,Ray,Gavilano1,Ishida07,EPL2008,MHJulien075,H67_CoNMR}. Na ordered atomic structures have been observed in sodium cobaltates by TEM \cite{Zandbergen}, neutrons \cite{Roger}, and x-rays \cite{TaiwanPRB2009,H67NQRprb,FouryPRB}. However, the experimental situation that prevails so far is quite unusual in solid state physics, as most experiments do not permit altogether to establish reliably the relation between the local order proposed, the actual Na content and the local magnetic properties of the studied samples.

On the contrary $^{23}$Na and $^{59}$Co NMR experiments have proved to be excellent probes allowing to evidence not only the Na atomic order but also that Co charge disproportionation occurs. In this paper we summarize our NMR study of sodium cobaltates Na$_{x}$CoO$_{2}$ with sodium content $0.65<x<0.80$ which allowed us to establish this correlation for the specific $x=2/3$ \cite{EPL2009,H67NQRprb,H67_CoNMR} and $x=0.77$ phases \cite{Na077prb,Co77JETP,Co077prb}. We also compare the properties of the studied phases with properties of the $x = 1$ and $x = 1/2$ compounds.

\section{Samples}
\label{Samples}

We have synthesized our powder samples by standard solid state reaction of Na$_{2}$CO$_{3}$ and Co$_{3}$O$_{4}$ powders in flowing oxygen, with nominal
concentrations $x$ increasing by increments of 2 to 3/1000 in the range
$0.65<x<0.80$. X-ray powder diffraction data always exhibited the Bragg peaks corresponding to the two layer Co structure $P2$ ($P6_{3}/mmc$, n$^{\circ }$166) with a hexagonal unit cell. We found that $c$-axis parameter changes systematically with sodium content $x$, so the position of the (008) reflection for this structure allows to match them (Fig.~\ref{FigGrXrayEcrys}).

Also we systematically detected weak additional reflections indicative of three dimensional Na long range ordering. It has been immediately clear that the corresponding Na order is complicated and highly dependent of Na content. The systematic study of the powder x-ray spectra for various nominal Na contents allowed us to separate pure phases from multiphase samples with distinct $c$ axis parameters - the simple observation of the (008) Bragg peaks like in Fig.~\ref{FigGrXrayEcrys}(a) permits ensuring the absence of phase mixing. The concentrations for which single phase samples could be stabilized are sequenced in four distinct narrow $x$ non overlapping domains - see Fig.~\ref{FigGrXrayEcrys}(b). For intermediate concentrations between these, we always find mixed-phase samples with the two end phases. The crystallographic parameters for these 4 phases were reported in Ref.~\cite{EPL2008}.

We have found that keeping samples at room temperature in contact with humid air leads to destruction of the phase purity and loss of sodium content \cite{JETP4phasesNQR}. The high sodium content sample evolves progressively into a mixture of the detected stable phases until it reaches the $x = 2/3$ composition which appears to be the most stable phase in this part of phase diagram.

Single crystals of sodium cobaltates Na$_{x}$CoO$_{2}$ can be grown by the floating zone technique \cite{SCgrowPRB2004,SCgrowJCG2004}. To perform detailed NMR study of the $x=0.77$ phase of sodium cobaltates we also synthesized single phase crystals with this specific sodium content \cite{Co77JETP}.

\begin{figure}[tbp]
\centering
\includegraphics[width=0.9\linewidth]{GrXrayEcrys.EPS}
\caption{(Color online) (a) Part of the x-ray powder spectra between 68 and 70$^{\circ }$ reflection angles for pure phase samples in the sodium content range $0.65<x<0.8$. One can see that in these pure phases the simple observation of the (008) Bragg peaks permits ensuring the absence of phase mixing. (The double peak structure corresponds here to the two Bragg peaks associated with the Cu K$\protect\alpha _{1}$ and K$\protect\alpha _{2}$ radiations); (b) The $c$ parameter values (filled and empty circles) obtained for all pure phase samples we could synthesize in the sodium content range $0.65<x<0.8$ establishes a  calibration curve for the $c$ axis parameter versus Na content for the $P2$ hexagonal two layers phases (red dotted line). Intervals with no data point marked by grey bars correspond to composition gaps. The $c(x)$ curve reported in Ref.~\cite{NatureMat2010} (green dashed line) agrees well with our data.}
\label{FigGrXrayEcrys}
\end{figure}

\section{Magnetic susceptibility and $^{23}$Na NMR shifts}

SQUID measurements of the macroscopic susceptibility $\chi _{m}$ taken in 5~T field allow us to evidence that the different phases display different magnetic properties. For instance, as evidenced in Fig.~\ref{FigGrSuscEcrys}(a), the low T magnitude of $\chi_{m}$ decreases progressively with increasing $x$. The $x=0.77$ phase is furthermore found to be the only phase in which an antiferromagnetic (AF) order is detected in low applied field. However, as minute amounts of impurities or slight admixture of phases could spoil the bulk measurements, spectroscopic measurements with local probes give better determination of the susceptibility of each phase.

The magnetic properties of the compounds are probed at the local scale through the magnetic shifts of the different Na sites resolved in the central line ($-\frac{1}{2}\leftrightarrow \frac{1}{2}$ transition) of the $^{23}$Na NMR spectra presented in Fig.~\ref{FigGrNaSpecs}. As we have shown in Ref.~\cite{NaPaper} shift values $K$ which corresponds to the first moment (center of gravity) of the central line allows us to follow the spin susceptibility $\chi _{s}(T)$ of the system up to room $T$, even when the central lines of the different Na sites merge. The $T$ variations of $K$ are reported in Fig.~\ref{FigGrSuscEcrys}b, and are shown to be quite identical for $T>100$~K with a unique Curie-Weiss $(T-\Theta)^{-1}$ variation (with $\Theta\approx$-80~K). They differ markedly below 100~K, as does the SQUID data for $\chi _{m}$, the low $T$ enhancement of $\chi _{s}(T)$ observed for $x=2/3$ being progressively reduced for increasing $x$.

\begin{figure}[tbp]
\centering
\includegraphics[width=0.8\linewidth]{GrSuscEcrys.EPS}
\caption{(Color online) (a) $T$-dependencies of the bulk susceptibility $\chi_{m}$ measured in non-oriented powders of sodium cobaltates with $0.65<x<0.8$ with a DC SQUID in a 5T field; (b) $T$-dependence of the mean $^{23}$Na NMR magnetic shift. Identical behaviour above 100~K can be seen for the four phases with remarkable differences at low $T$. For comparison data for $x=0.5$ and $x=1$ phases also shown in panels (a) and (b), as well as in inset (data were taken from the Ref.~\cite{Bobroff05} and Ref.~\cite{LangNa1} correspondingly).}
\label{FigGrSuscEcrys}
\end{figure}

In the $x=0.77$ AF phase, the saturation of $K(T)$, that is $\chi _{s}^{\alpha }(T)$, seen at low $T$ in Fig.~\ref{FigGrSuscEcrys}b should be associated with the onset of AF correlations. In a uniform Heisenberg model, one would then assign the progressive increase of $\chi _{s}(T)$ at low $T$ with decreasing $x$ to a decrease of $T_{N}$ and of out of plane AF coupling strength. However this primary interpretation fails as NMR data taken down to 1.4~K (and $\mu $SR to 50~mK \cite{Mendels05}), did not evidence any frozen magnetic state in the three other phases, which are then paramagnets in their ground state, most probably metallic, as no electronic magnetic transition is detected.

For comparison we also show in Fig.~\ref{FigGrSuscEcrys} the corresponding data for $x=0.5$ and $x=1$ phases taken from the Ref.~\cite{Bobroff05} and Ref.~\cite{LangNa1} correspondingly. The bulk magnetic susceptibility of Na$_{1}$CoO$_{2}$ exhibits a small and flat susceptibility, with a significant increase only at low temperature which is attributed to spurious paramagnetic local moments - see Fig.~\ref{FigGrSuscEcrys}(a). The magnetic shift $^{23}$K value in this phase is extremely low with no measurable temperature dependence (Fig.~\ref{FigGrSuscEcrys}(b) and insert in it). Both these dependencies confirm that the Na$_{1}$CoO$_{2}$ composition corresponds to a band insulator made of filled shell nonmagnetic Co$^{3+}$.

In Fig.~\ref{FigGrSuscEcrys} the temperature dependencies of the bulk magnetic susceptibility and magnetic shift of Na$_{0.5}$CoO$_{2}$ compound are also displayed \cite{Bobroff05}. Both are very weakly temperature dependent as compared with  phases with sodium content $0.65<x<0.80$. Two kinks at $T$=51~K and $T$=86~K (see insert in Fig.~\ref{FigGrSuscEcrys}(b)) can be attributed to a metal-insulator transition (MIT) and long range antiferromagnetic order, correspondingly \cite{Foo}.

\section{$^{23}$Na NMR and Na order}

To perform the nuclear magnetic resonance (NMR) study of sodium cobaltates powders we prepared oriented samples. As the room $T$ susceptibility of sodium cobaltates is known to be anisotropic \cite{Wang}, we have used this anisotropy to align the single crystallites of powder samples in a 7~T field, by mixing the samples with Stycast 1266 epoxy resin which cured in the field. Thus in our samples the $c$  axes of the crystallites were aligned in the same direction, but the $ab$ planes of different crystallites are randomly distributed. Such Stycast epoxy matrix also perfectly protects the powder from environment (water) influence \cite{JETP4phasesNQR}. Samples with $x=2/3$ prepared almost nine years ago and packed in Stycast did not show any changes in the NMR spectra.

In Fig.~\ref{FigGrNaSpecs} the typical low $T$ $^{23}$Na NMR spectra for the pure phase oriented samples are shown. As $^{23}$Na has a spin $I=3/2$, the NMR spectrum for a single Na site displays a central transition and two satellites disposed symmetrically with respect to the central line. While the shift of the central line signal is governed by the magnetism of the near-neighbor Co sites, the distance $\Delta \nu $ between the satellites is linked to the quadrupole frequency $\nu _{Q}$ associated with the magnitude of the electric field gradient (EFG) at the Na site, which is governed by the distribution of ionic and electronic charges around the Na. For each Na site the largest splitting between satellites is observed when $H\parallel c$, which evidences that the principal axis $Z$ of the EFG tensor is close to the crystallographic $c$ axis for all sodiums. For $x=2/3$ phase in the spectra of Fig.~\ref{FigGrNaSpecs} one can clearly distinguish three pairs of quadrupole transitions which correspond to three Na sites with distinct local environments \cite{NaPaper}. Due to the more complex structures of the other phases there was no surprise in finding a larger number of resolved Na sites, although with similar magnitudes of their quadrupole splittings. For example detailed analysis done in the Ref.~\cite{Na077prb} for $x=0.77$ phase shows that there are 4 unequivalent Na position in this phase of sodium cobaltates. The fact that only limited number of sites are observed in each phase definitely proves that the neighboring charges of the Na sites are ordered.

\begin{figure}[tbp]
\centering
\includegraphics[width=0.8\linewidth]{GrNaSpecs.EPS}
\caption{(Color online) Low temperature $^{23}$Na NMR spectra for for pure phase oriented powder samples in the sodium content range $0.65<x<0.8$. The splitting of the satellite $^{23}$Na transitions reveals the existence of defined Na order at each specific sodium content. These NMR spectra were obtained by pulse NMR at fixed frequency $\nu _{0}$, the spin echo intensity being measured after two 2$~\mu$ sec radio frequency pulses separated by 100~$\mu$ sec, while the external magnetic field was increased by steps. The full NMR spectra were then constructed using a Fourier mapping algorithm \cite{Clark}.}
\label{FigGrNaSpecs}
\end{figure}

\section{2D Na order}

Na$_{1}$CoO$_{2}$ has been previously synthesized and its structure determined on both powder samples \cite{Na1Fouassier} and single
crystals \cite{Na1Takahashi}. In this phase the single Na site is located on top of the center of a Co triangle, in a configuration usually called Na2 - see Fig.~\ref{FigStructures}.

Numerical simulations \cite{Roger} and electronic structure calculations \cite{Hinuma} suggest that the Na2 vacancies formed for $x<1$, are ordered in the Na plane. These vacancies have a tendency toward clustering and, depending on $x$, these clusters induce altogether the appearance of isolated Na1 sites (on top of a Co) in divacancy clusters or of trimers of Na1 sites in trivacancy clusters \cite{Roger}. For $x=0.5$ the Na atoms are ordered in an orthorhombic superstructure commensurate with the Co lattice with the equal occupancy
of Na1 and Na2 sites \cite{Na05structure} (Fig.~\ref{FigStructures}).

For the $x=2/3$ phase our NMR/NQR experiments have allowed the determination of both the 2D Na ordered structure and the 3D stacking of Na/Co planes \cite{EPL2009} (Fig.~\ref{FigStructures}). This Na organization agrees with GGA calculations \cite{Hinuma}, and consists of two Na on top of Co sites (the Na1 sites) and six on top of Co triangles (the Na2 sites). This twelve Co unit cell and the stacking between planes has been confirmed later by x-ray diffraction experiments \cite{H67NQRprb}. It results in a differentiation of four cobalt sites in the structure, two nearly non-magnetic Co$^{3+}$ and two more magnetic sites constituting a kagom\'e sublattice of the triangular Co lattice on which the holes are delocalized \cite{H67_CoNMR}.

The existence of a sodium site with axial charge symmetry and the intensity ratio between the sets of $^{23}$Na NMR lines in the sodium cobaltates phase with $T_{N}=$22~K permits us to reveal that the two-dimensional structure of the Na order corresponds to 10 Na sites on top of a 13 Co sites unit cell, that is with $x=10/13\approx 0.77$ \cite{Na077prb} - see Fig.~\ref{FigStructures}. However contrary to the $x=1/2$ or $x=2/3$ phases the 3D stacking of the Na planes is not perfect for $x=0.77$ \cite{Co077prb} but this does not influence markedly the electronic properties of this phase.

\begin{figure}[tbp]
\centering
\includegraphics[width=0.7\linewidth]{Structures_sq.EPS}
\caption{(Color online) 2D Na unit cell for specific phases of sodium cobalates Na$_{x}$CoO$_{2}$ (thick green lines). The Na1 sites (filled red dots) and Na2 sites (empty black circles) are differentiated, and represented above the triangular lattice of Co sites (not reported) which are located at all intersections of thin black lines.}
\label{FigStructures}
\end{figure}

\section{Co charge disproportionation}

NMR/NQR is a powerful technique which allows to establish the relation between the local Na order and the local magnetic properties of the phases studied.

In the $x=1$ phase the $^{59}$Co NMR signal shows a single cobalt site with very slow nuclear relaxation \cite{LangNa1,MHJulienNa1}, as expected for homogeneous planes of Co$^{3+}$ ions with $S=0$.

$^{59}$Co NMR in the $x=1/2$ phase gives clear evidence for the presence of two Co sites. Both sites display a similar $T$ dependence of the spin contribution to the magnetic shift, suggesting that there is no charge segregation into Co$^{3+}$ and Co$^{4+}$ sites. The electric field gradient at the Co site does not change at the AF or MIT transitions, indicating the absence of any modification of the charge ordering. The analysis of hyperfine coupling constants for both Co sites allowed the authors of \cite{Bobroff05} to conclude, that in this $x=1/2$ phase the charge disproportionation on the Co sites is rather small: Co$^{3.5\pm \varepsilon }$ (with $\varepsilon <0.2)$.

\begin{figure}[tbp]
\centering
\includegraphics[width=0.8\linewidth]{GrT1curves.EPS}
\caption{(Color online) Spin-lattice relaxation curves measured on different
cobalt sites in the $x=2/3$ and $x=0.77$ pure phase samples in the same experimental conditions. Whereas in the $x=2/3$ there are two types of cobalts - slow (Co1a,b) and fast (Co2a,b) relaxing, in the $x=0.77$ phase there are three types - slow (Co1), intermediate (Co2a,b,c) and fast (Co3a,b) relaxing. The fits of the curves with the magnetization relaxation functions from Ref.~\cite{H67_CoNMR} with $T_1$ parameters from the Table~\ref{tab:CoParams} are also shown.}
\label{FigGrT1curves}
\end{figure}

Though $^{59}$Co NMR has been detected in many phases, the most well studied one has been the $x=2/3$ phase. As we have formerly established the in plane Na ordering and its three dimensional stacking in this phase, we succeed for the first time to completely resolve all the parameters of the Zeeman and quadrupolar Hamiltonians for all cobalt sites in the unit cell \cite{H67_CoNMR}. The three non-magnetic Co$^{3+}$ (Co1) in $x=2/3$ unit cell are in axially symmetric positions and the doped holes are delocalized on the nine complementary magnetic cobalt sites (Co2) of the atomic unit cell. This Co1/Co2 charge disproportionation can be clearly seen in the relaxation of nuclear magnetization which is slow for non-magnetic Co1 sites and fast for the Co2 magnetic sites. This is exemplified in Fig.~\ref{FigGrT1curves}(a) where the relaxation curves of the longitudinal nuclear magnetization (spin lattice relaxation) are shown for Co1a and Co2b sites.

However the moderately complicated atomic structure in the $x=2/3$ phase resumes then in a very simple electronic structure in which the electrons delocalize on the Co2 kagom\'e sublattice of the triangular lattice of Co sites. These sites display a strong in plane electronic anisotropy initially unexpected but which accords perfectly with an orbital ordering along the kagom\'e sublattice organization and could result from the distribution of holes between the axial $a_{1g}$ orbitals and the in plane $e_{g}^{\prime}$ orbitals on the Co2 sites \cite{H67_CoNMR}. The observation of a single temperature dependence of the spin susceptibilities indicates that a single band picture applies, and that the magnetic properties are dominated by the static and dynamic electronic properties at the Co2 sites.

The $^{59}$Co NMR study in the paramagnetic state of the $x=0.77$ phase with $T_{N}=22$~K permitted us to evidence that at least 6 Co sites could be differentiated \cite{Co77JETP}. They could be separated by their magnetic and relaxation behavior into three types: a single site (Co1) with cobalt close to non-magnetic Co$^{3+}$, two sites (Co3) with the most magnetic cobalts in the system, and the remaining three sites (Co2) displaying an intermediate behavior - see Fig.~\ref{FigGrT1curves}(b).

It is important to highlight that the non-magnetic Co1 sites correspond only to 25\% of total cobalts in the system for $x=2/3$ and 23\% for $x=0.77$. These facts are in complete contradiction with Co$^{3+}$/Co$^{4+}$ charge segregation model, for which the Co$^{3+}$ content should be proportional to the Na content.

\section{Estimation of the charge disproportionation for the different Co
sites}

Our $^{59}$Co NMR experiments in the $x=2/3$ and $x=0.77$ phases allowed us to determine the quadrupole frequencies $\nu_{Q}$, magnetic shifts $K_{ZZ}$ and nuclear spin lattice relaxation rates $1/T_{1}$ for distinct cobalt sites in each phase \cite{H67_CoNMR,Co077prb}. They are summarized in Table~\ref{tab:CoParams}. Unexpectedly we found that the magnetic and charge properties of the Co sites are highly correlated with each other as $K_{ZZ}$ and $(1/T_{1})^{1/2}$ scale linearly with $\nu _{Q}$ - see Fig.~\ref{FigCoCorrEcrys}.

\begin{table}[tbp]
\centering
\caption{Some NMR parameters of the different Co sites in the $x=2/3$ and  $x=0.77$ phases measured at $T$=80~K. $K_{ZZ}$ is the magnetic shift, $\protect\nu _{Q}$ is the quadrupolar frequency, $I_{n}$ is the number of cobalts corresponding to the given Co site, $T_{1}$ is the values of the longitudinal nuclear magnetisation relaxation time, $\protect\delta $ is the hole concentration (charge state of cobalt is Co$^{3+\protect\delta }$) deduced from the analysis of the correlation between $(1/T_{1 spin})^{1/2}$ and $\protect\nu _{Q}$ data.}
\label{tab:CoParams}%
\begin{tabular}{|c|c|c|c|c|c|}
\hline
     & $K_{ZZ}$, \% & $\nu_Q$, MHz & $I_{n}$ & $T_1$, ms & $\delta $ \\
\hline
\multicolumn{6}{|c|}{$x$=2/3} \\
\hline
Co1a & 2.30(1) & 1.165(5)& 2 & 4.59(2) & 0.05 \\
Co1b & 2.23(1) & 1.34(1) & 1 &         & 0.10 \\
Co2a & 2.40(1) & 2.19(1) & 3 &         & 0.35 \\
Co2b & 2.61(1) & 2.55(1) & 6 & 0.25(1) & 0.46 \\
\hline
\multicolumn{6}{|c|}{$x$=0.77} \\
\hline
Co1  & 2.38(1) & 0.568(5) & 3 & 2.98(3) & 0.02 \\
Co2a & 2.45(1) & 1.14(2)  & 4 & 0.91(5) & 0.20 \\
Co2b & 2.48(1) & 1.22(2)  & 1 & 0.75(4) & 0.23 \\
Co2c & 2.50(1) & 1.42(2)  & 2 & 0.71(4) & 0.29 \\
Co3a & 2.56(1) & 1.75(2)  & 2 & 0.30(1) & 0.39 \\
Co3b & 2.62(1) & 2.19(2)  & 1 & 0.17(1) & 0.53 \\
\hline
\end{tabular}
\end{table}

Such correlations can be understood if one takes into account that both measured values $\nu _{Q}$ and $K_{ZZ}$ contain terms proportional to the quadrupole moment of the $t_{2g}$ hole density distribution $\hat{q}$ which involves as a coefficient the hole concentration $\delta $ on the Co site \cite{Co077prb}. Therefore the linear variation with $\nu _{Q}$ found for the Knight shift $K_{ZZ}$ and $(T_{1})^{-1/2}$ just reflects the fact that the hole content on the Co orbitals varies from site to site in both phases. Also this observation means that the hyperfine coupling, or the local magnetic $\chi $ scales with the on site delocalized charge.

As observed experimentally for non-magnetic Co$^{3+}$ the spin lattice relaxation is extremely long \cite{LangNa1,MHJulienNa1} as the spin contribution $(1/T_{1spin})$ becomes negligibly small. Therefore in the plot of Fig.~\ref{FigCoCorrEcrys}(b) the pure Co$^{3+}$ state should correspond to the limit $(1/T_{1spin})^{1/2}$ $\rightarrow 0$. In that limit the corresponding $\nu _{Q}=\nu _{0}$ should be solely due to the lattice contribution $\nu _{Q}^{latt}$ to the EFG. We can assume that on each Co site the quantity $\nu _{Q}-\nu _{0}$ is directly related with the local charge contribution to the EFG, which is proportional to the hole concentration. Using the charge neutrality equation and knowing the number of cobalts corresponding to the given Co site we could estimate the hole concentration on each Co site in both phases - see last column in the Table~\ref{tab:CoParams} and Ref.~\cite{Co077prb} for details.

This analysis permitted us to deduce a variation of $\nu _{Q}$ with local charge $d\nu_{Q}/d\delta \approx$3.3~MHz per hole on the Co site for both phases.

\begin{figure}[tbp]
\centering
\includegraphics[width=0.8\linewidth]{FigCoCorrEcrys.EPS}
\caption{(Color online) (a) Magnetic shift $K_{ZZ}$, (b) $(1/T_{1})^{1/2}$ (closed symbols) and the corresponding pure spin contribution  $(1/T_{1spin})^{1/2}$ (open symbols) vs. $\protect \nu _{Q}$ for cobalt sites in the phases with $x=0.77$ (red squares) and $x=2/3$ (blue circles). All experimental points were obtained at T=80~K. Linear fits are shown as guides to the eyes.}
\label{FigCoCorrEcrys}
\end{figure}

We can apply our model to the in $x=1/2$ phase, in which a small charge disproportionation into Co$^{3.5\pm \varepsilon }$ is observed \cite{Bobroff05}. Using the measured $\nu _{Q}$ values 2.8~MHz and 4~MHz for the two Co sites in the $x=1/2$ phase we can estimate in our model that $\varepsilon =0.18$ which is in good agreement with the conclusion of Ref.~\cite{Bobroff05}.

The occurrence of a local charge contribution to the EFG of $d\nu_{Q}/d\delta $=3.3~MHz per hole on Co site implies that the local charge is not evenly distributed on the Co $t_{2g}$ orbitals. A theoretical estimate gives that single hole (or electron) on a given $t_{2g}$ orbital should correspond for $^{59}$Co to  $d\nu_{Q}/d\delta \approx$20~MHz per hole which is about six times larger than our experimental value \cite{Co077prb}. This allows us to suggest that in the cobaltates the holes are distributed over the three orbital states within the $t_{2g}$-shells due to the thermal/quantum fluctuations \cite{KHA2008}.

\section{Conclusion}

All NMR studies done so far have demonstrated that the layered Na cobaltates
undergo a charge disproportionation of the Co sites for large Na content. This has been ascertained by the observation of a slow relaxing $^{59}$Co NMR signal, that we call Co1, with an intensity which increased with hole content from $x=0.5$ to $0.8$. All experimental parameters, e.g.  magnetic shift or EFG allowed us to establish that those Co sites have nearly filled t$_{2g}$ orbitals that correspond to a Co$^{3+}$ with low spin (S=0) configuration. The NMR signals of the other Co sites have been much harder to study in detail as they usually display the metallicity with larger magnetic behavior which is detected in the macroscopic properties.

In our recent studies we have highlighted two specific cases in which the $^{59}$Co NMR spectra could be resolved completely, which permitted us to achieve a complete study of the charge disproportionation. For $x=2/3$ which is a paramagnet with a Curie Weiss like susceptibility increasing down to $T=0$, only a second Co site (Co2) with a different magnetic behaviour is detected, while for $x=0.77$, which is AF below $T_{N}=$22~K, at least two other types of disproportionated Co sites are distinguished. In either case the observed charge disproportionation does not correspond to the  segregation into Co$^{3+}$ and Co$^{4+}$ often proposed by theoreticians. The physical properties do not correspond either to a situation where a bath of conduction electrons would be diffused by local moments responsible for the magnetic properties. We rather showed that the Co$^{3+}$ sites are inert magnetically, while the bands associated with the other sites correspond roughly to Co$^{3.5+}$ and are involved in both
metallic and magnetic properties.

Let us point out altogether that these compounds display an ordered Na atomic structure for a limited number of $x$ values, and that phase mixing occurs for $x$ values in between two pure phases. In most of the pure phases the Na atomic order seems to be locked on that of the charge disproportionation of Co sites. However we cannot decide so far from the experimental side whether the Na order drives the charge disproportionation. The latter could as well be intrinsic to the electronic properties of the Co plane for a given charge filling and could conversely drive the pinning of the Na in an ordered structure.

Some theoretical studies favor the latter approach and indicated for instance that the kagom\'e structure of the Co2 sites found for $x=2/3$ could be intrinsic to the CoO$_{2}$ planes \cite{KoshibaeMaekawa}. The fact that we could not detect a 3D ordering of the Na in the $x=0.77$ phase would possibly support such a picture in which the disproportionation is specific to the Co layers. We certainly expect theoretical calculations involving electronic correlations to establish whether the  charge disproportionated state could occur in the absence of the Na potential.

Such a study on cobaltates has some analogy as well with those performed on
the cuprates, where charge density wave order has been detected and heavily
studied recently in YBCO$_{6+x}$. The most prominent static CDW have been
detected there for O contents $x$ for which the dopants are ordered on the
CuO chains in between the CuO$_{2}$ bilayers. This suggests that in
layered oxides the ordering of the dopants has indeed an incidence on the
Fermi surface of the layers and therefore on their electronic properties.

\section{Acknowledgments}

We would like to thank here F.~Bert, P.~Mendels and A.V.~Dooglav for their
help on the experimental NMR techniques and for constant interest and
stimulating discussions. This work was partially supported by the RFBR under project 14-02-01213a and performed according to the Russian Government Program of Competitive Growth of Kazan Federal University. I.R.M. thanks for the support of a visit to Orsay by ``Investissements d'Avenir'' LabEx PALM (ANR-10-LABX-0039-PALM).

\bibliographystyle{elsarticle-num}

Strongly Correlated Electrons (cond-mat.str-el)


Does enhanced shape bias improve neural network robustness to common corruptions?

Abstract: Convolutional neural networks (CNNs) learn to extract representations of complex features, such as object shapes and textures to solve image recognition tasks. Recent work indicates that CNNs trained on ImageNet are biased towards features that encode textures and that these alone are sufficient to generalize to unseen test data from the same distribution as the training data but often fail to generalize to out-of-distribution data. It has been shown that augmenting the training data with different image styles decreases this texture bias in favor of increased shape bias while at the same time improving robustness to common corruptions, such as noise and blur. Commonly, this is interpreted as shape bias increasing corruption robustness. However, this relationship is only hypothesized. We perform a systematic study of different ways of composing inputs based on natural images, explicit edge information, and stylization. While stylization is essential for achieving high corruption robustness, we do not find a clear correlation between shape bias and robustness. We conclude that the data augmentation caused by style-variation accounts for the improved corruption robustness and increased shape bias is only a byproduct.

\section{Appendix}

\end{document}

\section{Introduction}
\label{sec:intro}
\begin{figure}[h]
	\begin{center}
		\includegraphics[width=0.92\linewidth, height=0.2\textheight]{images/teaser_final_v2.png}

		\caption{Illustration of the effect of different training augmentations. While both style-based \citep{geirhos2018imagenettrained} and edge-based augmentation (this paper) reach the same validation accuracy, edge-based augmentation shows a stronger increase in shape bias as evidenced by lower accuracy on patch-shuffled images and higher rate of classifying according to the shape category for texture-shape cue conflicts. Nevertheless, only style-based augmentation shows a considerable improvement against common corruptions such as Gaussian blur. This challenges the hypothesis that increased shape bias causes improved robustness to corruption.}
		\vskip -0.15in
		\label{fig:teaser}
	\end{center}
\end{figure}
As deep learning is increasingly applied to open-world perception problems in safety-critical domains such as robotics and autonomous driving, its robustness properties become of paramount importance. Generally, a lack of robustness against adversarial examples has been observed \citep{szegedy_intriguing_2013,goodfellow_explaining_2015}, making physical-world adversarial attacks on perception systems feasible \citep{kurakin_adversarial_2016,eykholt_physical_2018,lee_2019}. In this work, we focus on a different kind of robustness: namely, robustness against naturally occurring common image corruptions. Robustness of image classifiers against such corruptions can be evaluated using the ImageNet-C benchmark \citep{hendrycks_benchmarking_2018}, in which corruptions such as noise, blur, weather effects, and digital image transformations are simulated. \citet{hendrycks_benchmarking_2018} observed that recent advances  in neural architectures increased performance on undistorted data without significant increase in relative corruption robustness.

One hypothesis for the lack of robustness is an over-reliance on non-robust features that generalize well within the distribution used for training but fail to generalize to out-of-distribution data. \citet{ilyas_adversarial_2019} provide evidence for this hypothesis on adversarial examples. Similarly, it has been hypothesized that models which rely strongly on texture information are more vulnerable to common corruptions than models based on features encoding shape information \citep{geirhos2018imagenettrained,hendrycks_benchmarking_2018}. Alternative methods for increasing corruption robustness not motivated by enhancing shape bias use more (potentially unlabeled) training data \citep{xie2019self} or use stronger data augmentation  \citep{lopes2019improving,hendrycks2019augmix}.
Note that our meaning of ``shape'' \& ``texture'' is built on the definitions by \citet{geirhos2018imagenettrained}.

In this paper, we re-examine the question of whether increasing the shape bias of a model actually helps in terms of corruption robustness. While prior work has found that there are training methods that increase both shape bias and corruption robustness \citep{geirhos2018imagenettrained,hendrycks_benchmarking_2018}, this only establishes a correlation and not a causal relationship. To increase the shape bias, \citet{geirhos2018imagenettrained} ``stylize'' images by imposing the style of a painting onto the image, leaving the shape-related structure of the image mostly unchanged while modifying texture cues so that they get largely uninformative of the class. Note that image stylization can be interpreted as a specific form of data augmentation, providing an alternative hypothesis for increased corruption robustness which would leave the enhanced shape bias as a mostly unrelated byproduct.

In this work, we investigate the role of the shape bias for corruption robustness in more detail. We propose two novel methods for increasing the shape bias:

 \begin{itemize}[noitemsep,topsep=0pt,parsep=0pt,partopsep=0pt, leftmargin=*]
	\item Similar to \citet{geirhos2018imagenettrained}, we pre-train the CNN on an auxiliary dataset which encourages learning shape features. In contrast to \citet{geirhos2018imagenettrained} that use stylized images, this dataset consists of the \emph{edge maps} for the training images that are generated using the pre-trained neural network of  \citet{liu2017richer} for edge detection. This method maintains global object shapes but removes texture-related information, thereby encouraging learning shape-based representations.
	\item In addition to pre-training on edge maps, we also propose \emph{style randomization} to further enhance the shape bias. Style randomization is based upon sampling parameters of the affine transformations of normalization layers for each input from a uniform distribution.
\end{itemize}

Our key finding is summarized in Figure \ref{fig:teaser}. While pre-training on stylized images increases both shape bias and corruption robustness, these two quantities are not necessarily correlated: pre-training on edge maps increases the shape bias without consistently helping in terms of corruption robustness. In order to explain this finding, we conduct a systematic study in which we create inputs based on natural images, explicit edge information, and different ways of stylization (see Figure \ref{fig:teaser_dataset} for an illustration).
We find that the shape bias gets maximized when combining edge information with stylization without including any texture information (Stylized Edges). However, for maximal corruption robustness, superimposing the image (and thus its textures) on these stylized edges is required.
This, however, strongly reduces shape bias. In summary, corruption robustness seems to benefit most from style variation in the vicinity of the image manifold, while shape bias is mostly unrelated. Thus, image stylization is best interpreted as a strong data augmentation technique that encourages robust representations, regardless whether these representations are shape-based or not.

Moreover, we present results for a setting where we fine-tune only parameters of the affine transformation of a normalization layer on the target distribution (stylized or corrupted images, respectively) for a CNN trained on regular images. Surprisingly, this is already sufficient for increasing the shape bias/corruption robustness considerably. We conclude that CNNs trained on normal images do learn shape-based features and features robust to corruptions but assign little weight to them. It may thus be sufficient to perform augmentation in feature space (extending \citet{nam2019reducing,li2020feature}) so that higher weights are assigned to features that are robust to relevant domain shifts.
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{images/teaser_2b_v2.png}
\caption{Overview of content and stylization variants used in this paper: Content is a natural image (IN) or an edge map (E).
Content is stylized in three different ways: No stylization, style transfer with \texttt{Painter by Numbers} as style source as proposed in \cite{geirhos2018imagenettrained} (SIN and SE), style transfer with a different in-distribution image as style source (I-SIN and I-SE). Additionally, we show a superposition (SE+IN) between natural (IN) and stylized edge image (SE).}
\label{fig:teaser_dataset}
\end{figure}
\section{Related work}
\label{sec:related_work}

\textbf{Texture-vs-Shape Bias}
\citet{geirhos2018imagenettrained} and \citet{baker2018deep} hypothesized that CNNs tend to be biased towards textural cues rather than shape cues.
This line of research is further supported by \citet{brendel2018approximating}, where the authors show that BagNets, Deep Neural Networks (DNN) trained and evaluated only on small restricted local image patches, already perform reasonably well on ImageNet.
Similarly, \citet{yin2019fourier} and \citet{jo2017measuring} showed using a Fourier space analysis that DNNs rely on surface statistical regularities and high-frequency components.
The texture-vs-shape bias can be quantified by evaluating a network either on images with texture-shape cue conflict \citep{geirhos2018imagenettrained} or on images which were patch-wise shuffled \citep{luo2019defective}.

\textbf{Robustness Against Common Corruptions}
Common corruptions are potentially stochastic image transformations motivated by real-world effects that can be used for evaluating model robustness.
\citet{hendrycks_benchmarking_2018} proposed the ImageNet-C dataset that contains simulated corruptions such as noise, blur, weather effects and digital image transformations.
\citet{geirhos2018generalisation} showed that humans are more robust to image corruptions than CNNs.

Approaches to improve corruption robustness include data augmentation \citep{lopes2019improving,yun2019cutmix,hendrycks2019augmix,cubuk2019randaugment}, self-training with more training data \citep{xie2019self}, novel architectures and building blocks \citep{zhang2019shiftinvar,hu2018squeeze}, and changes in the training procedure \citep{hendrycks2019using,Rusak2020a,NIPS2019_9237}.
Motivated by the texture-vs-shape hypothesis, \citet{geirhos2018imagenettrained} and  \citet{michaelis2019benchmarking} train their network on a stylized version of ImageNet.
The idea is that style transfer removes textural cues and models trained on stylized data thus have to rely more on shape information.
The observed increase in corruption robustness on this stylized data was attributed to the shape bias.
In this work, we provide evidence that contradicts this claim.

Similar to training on stylized images, \emph{Style Blending} \citep{nam2019reducing} employs style transfer in latent space by interpolating between feature statistics of different samples in a batch.
\citet{li2020feature} extend this idea and use feature space blending along with label interpolation. \citet{hendrycks2019using} considers self-supervised training with the prediction of image rotations as an auxiliary task.
The authors argue that predicting rotation requires shape information and thus improves robustness.
Similarly, \citet{shi2020informative} proposes Dropout-like algorithm to reduce the texture bias and thereby increase the shape bias to improve model robustness. However, the authors also discuss that a ``sweet spot" between shape and texture is needed for the model to be robust for domain generalization.
With \emph{Patch-wise Adversarial Regularization}, \citet{NIPS2019_9237} try to penalize reliance on local predictive representations in early layers and encourage the network to learn global concepts.
Other augmentation techniques that aim to improve common corruption robustness are \emph{PatchGaussian} \citep{lopes2019improving}, \emph{CutMix} \citep{yun2019cutmix}, \emph{AugMix} \citep{hendrycks2019augmix}, and \emph{RandAugment} \citep{cubuk2019randaugment}.
At this point, it remains unclear whether the increase in robustness caused by these augmentations is due to learning fundamentally different representations such as more shape-biased ones or to more incremental improvements in feature quality.

\textbf{Edge-based Representations}
A classical method for extracting edge maps is the Canny edge extractor \cite{canny1986computational}.
More recent approaches use DNNs \citep{xie2015holistically,liu2017richer} (see Figure \ref{fig:edgemaps}).
\citet{geirhos2018imagenettrained} evaluate their shape-biased models on edge maps obtained with a Canny edge detector.
ImageNet-Sketch \citep{NIPS2019_9237} is a newly collected sketch-like dataset matching the ImageNet validation dataset in shape and size. It is used to evaluate generalization to domain shifts. In contrast to these works, we generate the edge-based representations with an edge detector using Richer Convolutional Features (RCF) \citep{liu2017richer} (see Figure \ref{fig:edgemaps}) and use them explicitly for training. We provide evidence that edge-based representations enhance the shape bias, through an evaluation on images with induced texture-shape cue conflict and patch-shuffled images.

\section{Learning shape-based representations}
\label{sec:approach}

Similar to \citet{geirhos2018imagenettrained}, we aim to enhance the shape bias of a network so that it bases its decision more on shape details than on the style of objects encoded in textures.
While \citet{geirhos2018imagenettrained} augment training data with different styles (stylization), thereby making texture cues less predictive,  we extract edge information (edge maps) from the training images to maintain explicit shape details and remove texture-related information completely. Here, we consider grayscale intensity edge maps rather than separate edge maps for each color channel. We propose to train CNNs using the edge maps in addition to the standard training data to learn shape-based representations for more effective shape-based decision-making.

Besides training on the dataset with explicit shape cues, high capacity networks learn different feature representations when trained jointly on datasets from different distributions.
Despite edge maps encouraging CNNs to learn shape-based representations, we observe that the network learns to encode features with texture details when introduced to the standard image data during training.
We propose here to further restrain the network from learning texture details on standard image data.
We discuss below the extraction of edge details from images to create the edge map dataset and explain the technique to reduce the texture bias of the CNN.

\textbf{Edge dataset}
Given a standard image dataset, we construct a new dataset with edge maps (named the Edge dataset) by extracting the edge details of each image. The edge details are extracted by the CNN-based edge detector using richer convolutional features (RCF) proposed in \cite{liu2017richer}. RCF network produces a single-channel edge map that contains the pixel values between $[0,\,255]$.
We convert the non-binary edge map into a binary map with values in \(\{0, 255\}\) using a threshold of \(128\) and transform it into a \(3\)-channel RGB edge map by duplicating the channels, so we can use the edge maps as a direct input to train the CNNs. The edge maps from the Edge dataset are used as input and can be independently used to train or evaluate CNNs without necessarily being combined with the standard image data. Please refer to Section \ref{app_sec:edge_dataset} for the details of RCF network.

\textbf{Style Randomization (\emph{SR})}
While using a dataset with explicit shape cues enhances shape-based representations, we propose to further reduce the texture bias of the network when training on standard images.
It is shown in the literature of style transfer \citep{dumoulin2016learned,huang2017arbitrary} that the statistics of feature maps (e.g., mean and standard deviation) of a CNN effectively capture the style of an image and changing these statistics would correspond to a change in the style of an image. SIN dataset is generated using such style transfer technique and shown to reduce the texture bias of the networks. Inspired by this observation, we propose a simple technique to effectively reduce the texture bias using the feature statistics when being trained on standard training data. We modify the style of an image in the feature space so that the network becomes style-invariant. In particular, we randomize the style details, i.e. feature statistics, of an image during training such that the network can not rely on the texture cues. A similar approach named \emph{Style Blending} (SB) is proposed in \citet{nam2019reducing} which randomizes the style information by interpolating the feature statistics between different samples in a mini-batch. We propose here a slightly different approach to make the network invariant to style information.
Instead of interpolating the statistics of similar distribution of data i.e, training samples, we completely randomize the feature statistics (mean and standard deviation) by randomly sampling them from an uniform distribution.
Considering $X_i$ as the $i^{th}$ feature map of an intermediate layer in CNN, and $\mu_i$ \& $\sigma_i$ as the feature statistics of $X_i$, the style randomized feature map $\hat X_i$ is defined as:

\begin{equation}
\hat X_i := \hat\sigma_i * \left(\frac{X_i - \mu_i}{\sigma_i}\right) + \hat\mu_i
\end{equation}

where $\hat\sigma_i \sim$ Uniform$(0.1, 1)$  and $\hat\mu_i \sim$ Uniform$(-1, 1)$.
These specific choices of sampling for $\hat\sigma_i$ and $\hat\mu_i$ were found to perform best on our evaluations.
The style transfer technique described in \cite{huang2017arbitrary} replaces the feature statistics of content image with the statistics of a desired style image to change the style.
Similarly, we replace the statistics of content image with random statistics to change the style information.
Training the network with \emph{SR} reduces the texture bias and improves shape-based decision making.
An advantage of \emph{SR} over \emph{SB} is that the feature statistics are sampled from a different distribution than the training data, that encourages learning representations to generalize better to out-of-distribution data. We show in Section \ref{sec:evaluation} that \emph{SR} outperforms \emph{SB} and aids the network to induce stronger shape-based representations.

\section{Experimental settings}
\label{sec:experiments}

\textbf{Dataset}
We use a subset of \(20\) classes from ImageNet dataset (ImageNet20, or IN) that are chosen randomly, to study the role of shape bias towards corruption robustness; the main reason being that extensive experiments on this dataset are feasible with limited computation. Details about this dataset can be found in Section \ref{app_sec:IN_dataset}.
The Edge dataset of IN (referred to as E) is generated as described in Section \ref{sec:approach}.

\input{tex/41_stylized_datasets}

\input{tex/42_training_details}

\subsection{Stylization variants}
\textbf{Stylization variants}
In addition to enhancing the shape bias using the edge maps,
we further study the contribution of different factors of Stylized ImageNet (SIN) \citep{geirhos2018imagenettrained} to gain insights on its improved performance on corruptions. We break down SIN into different factors to understand their influence on corruption robustness. We segregate the factors that jointly generate the stylized images and the factors that are hypothesized to improve corruption robustness. These include i) shape bias of the network, ii) styles that are transferred from paintings and iii) statistics of natural images from IN. The \emph{role of shape bias} is studied using the Edge dataset (E) proposed in Section \ref{sec:approach}. Other variants study the role of the remaining factors and are explained below:

\emph{Role of stylization} We create Stylized Edges (SE, see Figure \ref{fig:teaser_dataset}) for which the styles from the paintings are transferred to the edge maps of Edge ImageNet20 (E). Here, we study the significance of stylization without the presence of the statistics (texture details) of natural images.

\emph{Role of out-of-distribution styles} SIN is generated by transferring the styles from out-of-distribution images, namely paintings. We create its variant called Intra-Stylized IN (I-SIN, see Figure \ref{fig:teaser_dataset}) for which in-distribution images from IN are chosen randomly to transfer the styles. We also generate Intra-Stylized Edges (I-SE) where the image styles of IN are transferred to the Edge dataset E.

\emph{Role of natural images statistics}
The above variants of E or SE test the role of shape and stylization without retaining texture cues of natural images. We create another variant called \emph{Superposition} (SE+IN, see Figure \ref{fig:teaser_dataset}) that interpolates images \(I_{\text{SE}}\) from SE with images \(I_{\text{IN}}\) from IN to embed the statistics (texture details) from natural images: $I_{\text{SE+IN}} := (1-\alpha)\cdot I_{\text{SE}} + \alpha\cdot I_{\text{IN}}$. We set $\alpha=0.5$.

These different stylized variants including E allow insights into the interplay between shape bias and corruption robustness.
For simplicity, we term the networks that are trained on a certain dataset using the name of that dataset. For example, network trained on Stylized Edge (SE) is referred to as SE. The evaluation of SIN and I-SIN reveals the significance of the choice of styles and evaluation of Edge (E) indicates the role of the shape bias for corruption robustness. SE explains the importance of stylization and finally SE+IN allows to understand the importance of natural image statistics that are preserved in SIN and I-SIN but are missing from SE. Table \ref{tab:finetune_corruptions} provides an overview of the input image compositions of different variants that are described above.

\section{Evaluation of shape bias}
\label{sec:evaluation}
\begin{table}[t]
\centering
\begin{tabular}{|l||l|l|l||l|l|l|}
\hline
 & \multicolumn{3}{c||}{\thead{shuffled image patches $4 \times 4$ acc(\%)}}  & \multicolumn{3}{c|}{\thead{shape based cue conflict \#400}}\\  \cline{2-7}
\thead{Network\\} &   \thead{No \\styling} & \thead{style \\blending} & \thead{style \\randomization} & \thead{No \\styling} & \thead{style \\blending} & \thead{style \\randomization}\\ \hline
\makecell{IN}   & \makecell{67.22}  & \makecell{51.34}  & \makecell{41.97} & \makecell{63}  & \makecell{82}  & \makecell{86}
\\ \hline
\makecell{SIN} & \makecell{38.46}  & \makecell{36.96}   & \makecell{34.95} & \makecell{144}     & \makecell{155} & \makecell{156}
\\ \hline
\makecell{E}  & \makecell{34.11}    & \makecell{33.95}   & \makecell{\textbf{28.43}} & \makecell{155}    & \makecell{166}  & \makecell{\textbf{193}}
\\ \hline

\end{tabular}
\caption{Comparison of different feature space style augmentation methods on $4 \times 4$ shuffled image patches and number of shape based predictions in texture-shape cue conflict images. Evaluation of shuffled patches is conducted on \(598\) correctly classified validation images by all the networks.}
	\vskip -0.15in

	\label{tab:stylization}
\end{table}

\begin{table}[t]
\centering
\begin{tabular}{|l||l|l|l||l|l|l|}
\hline
 & \multicolumn{3}{c||}{\thead{shuffled image patches acc(\%)}}  & \multicolumn{3}{c|}{\thead{texture-shape cue conflict results}}\\  \cline{2-7}
\thead{Network\\} &   \thead{\textbf{$2 \times 2$}}  & \thead{$4 \times 4$}  & \thead{$8 \times 8$} & \thead{shape \#400} & \thead{shape \#100} & \thead{texture \#100}\\ \hline
\makecell{IN}   & \makecell{78.57}  & \makecell{41.93}  & \makecell{31.21} & \makecell{86}  & \makecell{18}  & \makecell{20}
\\ \hline
\makecell{SIN} & \makecell{75.78}  & \makecell{35.56}   & \makecell{18.48} & \makecell{156}     & \makecell{32} & \makecell{\bf 2}
\\ \hline
\makecell{E}  & \makecell{73.29}    & \makecell{28.42}   & \makecell{11.18} & \makecell{193}    & \makecell{46}  & \makecell{15}
\\ \hline
\makecell{SE}  & \makecell{\bf 66.77}  & \makecell{28.73} & \makecell{12.89}  & \makecell{224}     &  \makecell{55} & \makecell{6}
\\ \hline
\makecell{E-SIN} & \makecell{71.12} & \makecell{\bf 23.76} & \makecell{\bf 10.25} & \makecell{\bf 234}   & \makecell{\bf 58}  & \makecell{6}
\\ \hline

\end{tabular}
\caption{Comparison of models trained on different datasets on shuffled image patches and number of texture-shape cue conflict predictions based on shape and texture labels. Evaluation of shuffled image patches is conducted on \(644\) validation images that are correctly classified by all the networks.}
	\vskip -0.15in

	\label{tab:shape_bias}
\end{table}

In this section, we evaluate different methods in terms of their shape bias using two different evaluation criteria - \emph{Shuffled image patches} and \emph{Texture-shape cue conflict} that are described below.

\textbf{Shuffled image patches:}
Following \citet{luo2019defective}, we manipulate images by perturbing the shape details while preserving the local texture of the objects. We divide an image into different patches of size $n\times n$ with $n\in \{2,4,8\}$ and randomly shuffle the patches as shown in Figure \ref{fig:shuffled_patches}. Larger $n$ corresponds to more distorted shapes. The performance of networks that rely more on shape is expected to deteriorate more strongly as the number of patches increases.
We conduct this evaluation only on the ImageNet20 validation images that were correctly classified by \emph{all} the networks that are selected for comparison.

\textbf{Texture-shape cue conflict:}
The cue conflict image dataset proposed by \citet{geirhos2018imagenettrained} consists of images where the shape of an object carries the texture of a different object. For example, the object \texttt{cat} holds the texture of \texttt{elephant} as shown in Figure \ref{fig:cue_conflict}. Each image in the dataset contains two class labels: labels with respect to shape and texture. The evaluation is carried out to test the network's bias towards shape or texture. Networks with strong shape bias will exhibit higher accuracy according to the shape label while networks with texture bias will have higher accuracy for texture-based label. The original dataset contains a total of \(1280\) cue conflict images designed for the evaluation of the networks trained on the entire ImageNet dataset. \(400\) of these images have classes (shape labels) present in ImageNet20. A subset of \(100\) instances (\(20\) instances from \(5\) different categories) from the selected images also has a texture label that belongs to ImageNet20 (see Figure \ref{fig:cue_conflict} bottom). The remaining \(300\) images with texture labels that do not belong to the classes of ImageNet20 are not considered for texture-based classification.

\textbf{Results}
The results in Table \ref{tab:stylization} compare \emph{style blending (SB)} \citep{nam2019reducing}, \emph{style randomization (SR)} (Section \ref{sec:approach}), and no styling in feature space for networks trained on IN, SIN and E. In terms of performance on $4\times 4$ shuffled patches, \emph{SB} performs worse than no styling, and \emph{SR} performs even worse than \emph{SB}. This indicates increasing shape bias from no styling over \emph{SB} to \emph{SR}. This finding is reinforced by an increasing number of images classified according to the shape label for texture-shape cue conflict images from no styling over \emph{SB} to \emph{SR}. Similarly, when comparing different training datasets, SIN results in stronger shape bias than IN, and E exhibits stronger shape bias than SIN.

In Table \ref{tab:shape_bias}, we compare additional networks, all with \emph{SR} enabled. Here, we again see a consistent trend of increasing shape bias from IN over SIN to E. Moreover, stylized edges (SE) further increase shape bias than E. Lastly, E-SIN improves shape bias even slightly beyond SE. In summary, we can see a clear increase in shape bias for the methods proposed in this paper over IN or SIN. Next, we investigate to which extent this also results in an increased corruption robustness.

\begin{comment}
\begin{figure}
	\begin{center}
		\includegraphics[width=0.75\linewidth, height=0.25\textheight]{eccv2020kit/images/results/shape_cnn_patch_shuffle_results.png}

		\caption{Results of ShapeCNN on shuffled image patches with different stylizations in feature space}
		\vskip -0.15in
		\label{fig:patch_shuffle_results}
	\end{center}
\end{figure}
\end{comment}
\section{Influence of shape bias on common corruptions}
\label{sec:robustness_corruptions}
\begin{figure}[t]
	\begin{center}
		\includegraphics[width=1.0\linewidth, height=0.25\textheight]{images/results/distortions_eval_v2.png}

		\caption{Classification accuracy of different networks on two corruptions across \(5\)  severity levels. Severity \(0\) represents accuracy on clean validation data of IN. Severity levels \(1\) - \(5\) follow the corruption parameters from \citet{hendrycks_benchmarking_2018} and represent increasingly strong corruptions.}
		\vskip -0.15in
		\label{fig:patch_shuffle_results}
	\end{center}
\end{figure}

\begin{table}[tb]
\centering
\begin{tabular}{|l||l|l|l||l|l|l|}
\hline
 & \multicolumn{3}{c||}{\thead{Input image composition}}  &          &     &  \\  \cline{2-4}
\thead{Network\\} &   \thead{Natural \\image} & \thead{Edge\\ map} & \thead{Style\\ transfer} & \thead{Shape\\ \#100} & \thead{Texture \\ \#100} & \thead{Mean corruption \\acc(\%)}\\ \hline
\makecell{IN}   & \makecell{\cmark}  & \makecell{\xmark}  & \makecell{\xmark} & \makecell{11}  & \makecell{39}  & \makecell{64.69}
\\ \hline
\makecell{SIN} & \makecell{\cmark}  & \makecell{\xmark}   & \makecell{\cmark} & \makecell{34}     & \makecell{2} & \makecell{77.64}
\\ \hline
\makecell{E}  & \makecell{\xmark}    & \makecell{\cmark}   & \makecell{\xmark} & \makecell{46}    & \makecell{15}  & \makecell{62.01}
\\ \hline
\makecell{SE}  & \makecell{\xmark}  & \makecell{\cmark} & \makecell{\cmark}  & \makecell{55}     &  \makecell{6} & \makecell{71.81}
\\ \hline
\makecell{E-SIN} & \makecell{\cmark} & \makecell{\cmark} & \makecell{\cmark} & \makecell{62}   & \makecell{5}  & \makecell{71.55}
\\ \hline
\makecell{SE+IN} & \makecell{\cmark} & \makecell{\cmark} & \makecell{\cmark} & \makecell{22}   & \makecell{13}  & \makecell{\textbf{78.96}}
\\ \hline

\end{tabular}
\caption{Mean corruption accuracy (mCA) and texture/shape results on texture-shape cue conflict dataset of different networks. mCA is the mean accuracy over \(15\) ImageNet-C corruption and severities ranging from \(1\) to \(5\). Networks trained with style transfer augmentation perform better than those without and network trained on superpositioned images (SE+IN) yield best mCA.}
	\vskip -0.15in

	\label{tab:finetune_corruptions}
\end{table}

We compare different networks in terms of their corruption robustness. Figure \ref{fig:patch_shuffle_results} shows the accuracy of different networks for two types of corruptions: Gaussian noise and frost (refer Figure \ref{fig:appendix_networks_corruptions} for all corruptions). Table \ref{tab:finetune_corruptions} presents the corruption accuracy averaged over 15 ImageNet-C corruptions along with shape and texture results on the texture-shape cue conflict dataset.
Generally, a CNN trained on IN performs poorly in terms of corruption robustness while SIN is relatively robust. On the other hand, E performs considerably worse than SIN and is not consistently better than IN despite having an even stronger shape bias than SIN. Networks SE and E-SIN further increase shape bias but still perform considerably worse than SIN in terms of corruption robustness. These results contradict the hypothesis that stronger shape bias results in increased corruption robustness.

The only method that slightly surpasses SIN in terms of corruption robustness is the superposition of SE with natural images (SE+IN). However, this method has a relatively small shape bias. A common theme of SIN and SE+IN is that both exhibit properties of a natural image but are strongly distorted by stylization (see Figure \ref{fig:teaser_dataset}). We hypothesize that these methods correspond to strong augmentation methods that stay close enough to the data manifold while inducing high diversity in appearance and thereby encourage learning robust representations, which need not necessarily be shape-based.
We extend these findings to larger datasets with 200 classes of ImageNet, deeper architectures like ResNet50, DenseNet121, MobileNetV2  and different normalization layers like BatchNorm in Section \ref{app_sec:extended experiments}.
Lastly, as can be seen from Figure \ref{fig:patch_shuffle_results}, intra-stylization is nearly as effective as stylization based on paintings, implying that style need not necessarily be out-of-distribution for being useful.
\section{On the adaptability of learned representations}
\label{sec:finetune_affine}
\begin{table}[tb]
\centering
\begin{tabular}{|l||l|l|l|l||l|l|l|}
\hline
 & \multicolumn{4}{c||}{\thead{Corruptions acc(\%)}} & & \multicolumn{2}{c|}{\thead{Cue Conflict}} \\
 \cline{2-5} \cline{7-8}

 \thead{Network}       &     \thead{Speckle\\ noise} & \thead{Gaussian\\ blur} & \thead{Frost} & \thead{Pixelate}  & \thead{SIN \\val acc(\%)}  &  \thead{shape\\ \#400} & \thead{texture \\ \#100}\\  \hline
\makecell{IN}                                                     & \makecell{61.28}         & \makecell{42.96}         & \makecell{66.62} & \makecell{78.54}        & \makecell{42.0}    & \makecell{63}  & \makecell{39}      \\ \hline
\makecell{IN (fine-tuned)} & \makecell{82.7}          & \makecell{77.3}          & \makecell{81.02} & \makecell{87.02} &    \makecell{68.0}   & \makecell{130}   & \makecell{13}    \\ \hline
\makecell{E}                                                        & \makecell{67.76}      & \makecell{44.48}         & \makecell{61.04} & \makecell{70.94}      &   \makecell{62.3}     & \makecell{193}  & \makecell{15}      \\ \hline
\makecell{E (fine-tuned)}    & \makecell{80.18}         & \makecell{71.74}         & \makecell{73.7}  & \makecell{74.78} & \makecell{72.4}    & \makecell{222}  & \makecell{9}              \\ \hline
\end{tabular}
\caption{Mean corruption accuracy, SIN and cue conflict results of networks with \& without additional fine-tuning of the affine parameters of normalization layers on the respective corruptions. Fine-tuned networks perform significantly better, despite only the normalization layers are updated.}
	\vskip -0.15in

	\label{tab:finetune_corruptions_sin}
\end{table}
As seen in the previous section, style augmentation on natural images is important for the network to be able to generalize to different domains such as common corruptions. We now study how easily a pre-trained network can be adapted to a different distribution such as corruptions. Importantly, this uses the ``unknown'' distortion during training; this experiment is not meant as a practical procedure for the ImageNet-C benchmark but rather for understanding internal representations of a network.

\cite{chang2019domain} showed that domain-specific affine parameters in normalization layers are essential when training a network on different input data distributions jointly. We conduct a similar experiment with the key difference that our network is already pre-trained on IN/E and only the affine parameters of normalization layers are fine-tuned to fit the distribution of the respective target domain. First, we fine-tune affine parameters of the network on several ImageNet-C corruptions separately and evaluate the mean corruption accuracy on the same corruption across different severity levels. As shown in Table \ref{tab:finetune_corruptions_sin} (left), performance on the corruptions can be greatly improved even with fixed convolutional parameters trained on IN/E by just tuning the affine parameters. Similarly, we also fine-tune the affine parameters of pre-trained CNN on SIN. Results in Table \ref{tab:finetune_corruptions_sin} (right) show not only an improvement on SIN validation accuracy but also improved shape-based classification results on texture-shape cue conflict images. These results suggest that the standard CNN encodes robust representations that can be leveraged when adapting affine parameters on a target domain.
\section{Conclusion}
We performed a systematic empirical evaluation of the hypothesis that enhanced shape bias of a neural network is predictive for increased corruption robustness. Our evidence suggests that this is not the case and increased shape bias is mostly an unrelated byproduct. Increased corruption robustness by image stylization is better explained as a strong form of augmentation which encourages robust representations regardless whether those are shape-based or based on other cues. We conclude that if corruption robustness is the main objective, one should not primarily focus on increasing the shape bias of learned representations.
Potential future research directions will focus on understanding whether shape-biased representations offer advantages in other domains than corruption robustness \citep{hendrycks2020many}. Moreover, one could try devising stronger augmentation procedures in image or feature space based on our findings. Lastly, gaining a better understanding of which kind of features (if not shape-based ones) are essential for corruption robustness is an important direction.

\section{Appendix}

\subsection{Details of the edge detector}
\label{app_sec:edge_dataset}
The edge details are extracted by the CNN-based edge detector using richer convolutional features (RCF) proposed in \cite{liu2017richer}, which produces edge maps as side outputs of different layers of the network.
The level of details of the edge maps gradually become coarser from the early layers to the final layers of the network.
We employ here a ResNet101-based RCF edge detector which was trained on the BSDS500 dataset \cite{arbelaez2010contour} and use the pre-trained network to extract edge maps from images.
The ResNet-101 architecture is composed of four stages, and we select the side output of the second stage to generate the edge maps, as we observed that these edge maps contain the essential shape details, as shown in Figure \ref{fig:edgemaps}.

\begin{figure}[h]
	\begin{center}
		\includegraphics[width=0.9\linewidth, height=0.16\textheight]{images/edgemaps_whitewashed.png}

		\caption{Illustration of a natural image, and edge maps extracted with Canny edge detector \cite{canny1986computational}, with RCF \cite{liu2017richer}, and with binarized edges from RCF.}
		\vskip -0.15in
		\label{fig:edgemaps}
	\end{center}
\end{figure}

\subsection{ImageNet20 dataset}
\label{app_sec:IN_dataset}
We use a subset of \(20\) classes from ImageNet dataset to study the influence of the shape bias on robustness towards corruptions. It comprises a total of 25784 training and 1000 validation images. This subset consists of animal classes (\texttt{african elephant}, \texttt{german shepherd}, \texttt{tabby cat}, \texttt{arabian camel}, \texttt{tailed frog}, \texttt{scorpion}), birds (\texttt{king penguin}, \texttt{albatross}), insects (\texttt{fly}, \texttt{sulphur butterfly}), man made objects (\texttt{tea pot}, \texttt{stop watch}, \texttt{teddy bear}, \texttt{fur coat}), automobile (\texttt{sports car}, \texttt{trolley bus}, \texttt{life boat}) and edible items (\texttt{mushroom}, \texttt{bell pepper}, \texttt{pretzel}).
\begin{comment}
\subsection{Training details}
\label{app_sec:training_details}
We employ a ResNet18 \citep{he2016deep} architecture with group normalization \citep{wu2018group} and weight standardization \citep{qiao2019weight} to conduct our experiments. Network is trained on the different dataset variants IN, SIN, E, I-SIN, I-SE and SE+IN described under \emph{Stylization variants} in Section \ref{sec:experiments}. IN and SIN \citep{geirhos2018imagenettrained} are considered as baseline for our experiments. All networks except IN are trained in two stages. The first stage begins with training the network on the respective dataset variant (e.g: E) for a total of 75 epochs starting with a learning rate of \(0.1\), which is dropped at the 60th epoch by a factor 10. In the second stage, the networks are then fine-tuned on the respective dataset along with IN (e.g: E \& IN) for another round of 75 epochs starting with a learning rate of \(0.01\), later reduced to \(0.001\) at the 60th epoch. On the other hand, the network on IN is trained for 100 epochs with a learning rate of \(0.1\), reduced to \(0.01\) and \(0.001\) at the 60th and 90th epochs, respectively. We use a batch size of \(128\) samples with the SGD optimizer and weight decay \(10^{-4}\).

During the fine-tuning stage, the networks receive an equal number of training samples from both datasets (e.g: 128 samples from E and 128 samples from IN in a mini-batch). Note that the data distribution of edge maps from the datasets E, SE and I-SE are different than the distribution of images from other datasets. Fine-tuning the network on inputs with different distributions results in degradation of the performance. In other words, the datasets E, SE and I-SE do not preserve natural image statistics and degrade task performance when finetuning along with clean images. Hence, we weigh the loss of training samples of edge maps from E, SE and I-SE when fine-tuning along with IN. The loss between training samples is weighted as follows: $Loss \ L$ = $L_{\text{IN}} + \lambda L_{\text{edgemaps}}$, with $\lambda=0.01$. Finetuning on style variants SIN, I-SIN that better preserve natural image statistics does not affect classification performance significantly, hence $\lambda$ is not used. Larger $\lambda$ preserves the shape bias but affects the clean accuracy while smaller $\lambda$ reduces shape bias of the network.

We freeze the first convolutional layer and the first normalization layer's affine parameters during fine tuning. We observed that freezing these two layers demonstrate more global shape bias than fine-tuning all the layers in the network. We show that E possesses more global shape details of the objects whereas SIN demonstrates little or no texture bias for decision making. Both these datasets are complementary to each other and further enhance shape-based representations when combined. We fine-tune the network that is pre-trained on E in the first stage of training with SIN and IN (termed as \textbf{E-SIN}) in the second stage and show that such setup further improves shape-based predictions.
\end{comment}
\subsection{Techniques for the evaluation of shape bias}
\begin{figure}[tb]
\centering
\begin{subfigure}[h]{\linewidth}
\includegraphics[width=\linewidth, height=0.14\textheight]{images/shuffled_image_patches.png}
\caption{The image is divided into $n \times n$ patches and the patches are randomly shuffled. The global object shape is increasingly perturbed with larger n.}\label{fig:shuffled_patches}
\end{subfigure}

\begin{subfigure}[b]{\linewidth}
\includegraphics[width=0.8\linewidth, height=0.14\textheight]{images/cue_conflict_images.png}
\caption{Images with conflicting shape and texture cues. The images are obtained by applying style transfer with a texture image as style source.}\label{fig:cue_conflict}
\end{subfigure}
\caption{Techniques to evaluate the shape bias of networks}
\label{fig:eval_shape_bias}
\end{figure}

Figure \ref{fig:eval_shape_bias} depicts the two techniques that are used in our work to evaluate the shape bias. Figure \ref{fig:shuffled_patches} points to the \textbf{shuffled image patches} technique that perturb the shape details by preserving the local texture. On the other hand, Figure \ref{fig:cue_conflict} show examples of \textbf{texture-shape cue conflict images} that test the network's bias towards shape or texture.

\subsection{Convolutional filters}

We show in Table \ref{tab:stylization} and Table \ref{tab:shape_bias} that the network E demonstrates stronger shape bias than IN and SIN. In Figure \ref{fig:conv1_filter}, we visualize the filters of first convolutional layer of networks IN, SIN and E to understand the behavior of these networks. As seen in the figure, filters of E strongly resembles the Gabor filters for edge detection compared to other networks. These results suggest that E extracts features that corresponds to the shape information in the form of edges and effectively based its decision on shape details. The filters of E are non-colored because E is trained on edge maps whereas IN and SIN are trained on natural and stylized images respectively.

\begin{figure}
\centering
\begin{subfigure}[h]{.3\linewidth}
\includegraphics[width=\linewidth]{images/conv1_filters/standard.png}
\caption{Standard CNN (IN)}\label{fig:conv1_filter_standard}
\end{subfigure}
\begin{subfigure}[h]{.3\linewidth}
\includegraphics[width=\linewidth]{images/conv1_filters/stylized.png}
\caption{Stylized CNN (SIN)}\label{fig:conv1_filter_stylized}
\end{subfigure}
\begin{subfigure}[h]{.3\linewidth}
\includegraphics[width=\linewidth]{images/conv1_filters/edge.png}
\caption{Edge CNN (E)}\label{fig:conv1_filter_edge}
\end{subfigure}
\caption{Filters of first convolutional layer of Standard CNN (IN), Stylized CNN (SIN), Edge CNN (E) (ours). The filters of E strongly resembles Gabor filters for edge detection than other two networks.}
\label{fig:conv1_filter}
\end{figure}

\subsection{Additional experimental results}
\label{app_sec:extended experiments}
\subsubsection{Patch shuffled results on the whole validation data}
We extend the results of \emph{shuffled image patches} in Table \ref{tab:stylization} and \ref{tab:shape_bias} on the whole validation data and are shown in Table \ref{tab_app:stylization} and \ref{tab_app:shape_bias} respectively.

\begin{table}[t]
\centering
\begin{tabular}{|l||l|l|l||}
\hline
 & \multicolumn{3}{c||}{\thead{shuffled image patches $4 \times 4$ (\%)}} \\  \cline{2-4}
\thead{Network\\} &   \thead{No styling} & \thead{style blending} & \thead{style randomization} \\ \hline
\makecell{IN}   & \makecell{57.8}  & \makecell{44.9}  & \makecell{36}
\\ \hline
\makecell{SIN} & \makecell{33.1}  & \makecell{32.2}   & \makecell{29.8}
\\ \hline
\makecell{E}  & \makecell{28.3}    & \makecell{28.4}   & \makecell{\bf 23.43}
\\ \hline
\end{tabular}
\caption{Comparison of different feature space style augmentation methods on $4 \times 4$ shuffled image patches on whole validation data.}
	\vskip -0.15in

	\label{tab_app:stylization}
\end{table}

\begin{table}[t]
\centering
\begin{tabular}{|l||l|l|l||}
\hline
 & \multicolumn{3}{c||}{\thead{shuffled image patches (\%)}}  \\  \cline{2-4}
\thead{Network\\} &   \thead{\textbf{$2 \times 2$}}  & \thead{$4 \times 4$}  & \thead{$8 \times 8$}\\ \hline
\makecell{IN}   & \makecell{66.7}  & \makecell{36}  & \makecell{26.7}
\\ \hline
\makecell{SIN} & \makecell{63.5}  & \makecell{29.8}   & \makecell{17.6}
\\ \hline
\makecell{E}  & \makecell{61.8}    & \makecell{23.4}   & \makecell{9.8}
\\ \hline
\makecell{SE}  & \makecell{\bf 57}  & \makecell{23.7} & \makecell{10}
\\ \hline
\makecell{E-SIN} & \makecell{59} & \makecell{\bf 20.1} & \makecell{\bf 9.1}
\\ \hline

\end{tabular}
\caption{Comparison of models trained on different datasets on shuffled image patches evaluated on whole validation data.}

	\label{tab_app:shape_bias}
\end{table}

\subsubsection{ResNet18 on ImageNet200}
We present an additional evaluation on ImageNet200 (200 TinyImageNet classes but all the images in full resolution from ImageNet) in Table \ref{app_tab:IN200}: E-SIN has higher shape bias but demonstrates lower mean Corruption Accuracy(mCA) than SIN. This reinforces our core finding of shape bias not implying corruption robustness on a larger dataset. Here, all models have clean validation accuracy of about 70\%.

\begin{table}[t]
\centering
\begin{tabular}{|l||l|l|l||l|l|}
\hline
 & \multicolumn{3}{c||}{\thead{shuffled image patches(\%)}}  &    & \\  \cline{2-4}
\thead{Network\\} &   \thead{\textbf{$2 \times 2$}}  & \thead{$4 \times 4$}  & \thead{$8 \times 8$} & \thead{Cue conflict \\ shape \#880} & \thead{Mean corruption \\ accuracy (\%)}\\ \hline
\makecell{IN}   & \makecell{78.6}  & \makecell{45.7}  & \makecell{15.6} & \makecell{90}  & \makecell{35.8}
\\ \hline
\makecell{SIN} & \makecell{61.7}  & \makecell{17.7}   & \makecell{3.7} & \makecell{273}     & \makecell{\bf 52.4}
\\ \hline
\makecell{E-SIN}  & \makecell{\bf 54.3}  & \makecell{\bf 10.6} & \makecell{\bf 1.8}  & \makecell{\bf 337}     &  \makecell{47.7}
\\ \hline
\end{tabular}
\caption{Shape based evaluation \& corruption accuracies with ResNet18 on 200 classes of ImageNet. Patch shuffled evaluation is conducted on 5474 correctly classified validation images by all the networks.}

	\label{app_tab:IN200}
\end{table}

\subsubsection{ResNet18 with BatchNorm on ImageNet20}
We provide results on ResNet18 wih BatchNorm in Table \ref{app_tab:resnet18_BN} showing that it leads to the same findings as the ones reported in the paper for Group Normalization + Weight Standardization: Both networks E and E-SIN exhibit more shape bias but lower mean Corruption Accuracy (mCA) than SIN. Superposition (SE+IN) has comparable mean corruption accuracy with SIN despite showing lower shape bias on patch shuffled and cue conflict evaluation. Here, all models below exhibit validation accuracies of 89\%-90\%. Note that the architecture ResNet18 wih BatchNorm does not include Group Normalization and Weight Standardization layers.

\begin{table}[t]
\centering
\begin{tabular}{|l||l|l|l||l|l|}
\hline
 & \multicolumn{3}{c||}{\thead{shuffled image patches(\%)}}  &    & \\  \cline{2-4}
\thead{Network\\} &   \thead{\textbf{$2 \times 2$}}  & \thead{$4 \times 4$}  & \thead{$8 \times 8$} & \thead{Cue conflict \\ shape \#400} & \thead{Mean corruption \\ accuracy (\%)}\\ \hline
\makecell{IN}   & \makecell{89.9}  & \makecell{72.9}  & \makecell{41.9} & \makecell{58}  & \makecell{57.8}
\\ \hline
\makecell{SIN} & \makecell{\bf 81.3}  & \makecell{47.7}   & \makecell{19.6} & \makecell{151}     & \makecell{\bf 72.9}
\\ \hline
\makecell{E}  & \makecell{\bf 81.3}    & \makecell{\bf 39}   & \makecell{\bf 9.5} & \makecell{128}    & \makecell{50.8}
\\ \hline
\makecell{E-SIN}  & \makecell{84.3}  & \makecell{41} & \makecell{11.2}  & \makecell{\bf 169}     &  \makecell{67.5}
\\ \hline
\makecell{SE+IN} & \makecell{86.1} & \makecell{60.8} & \makecell{33.5} & \makecell{91}   & \makecell{\bf 72}
\\ \hline

\end{tabular}
\caption{Shape based evaluation \& corruption accuracies of ResNet18 networks with BatchNorm. Patch shuffled evaluation is conducted on 775 correctly classified validation images by all the networks.}

	\label{app_tab:resnet18_BN}
\end{table}

\subsubsection{ResNet50 on ImageNet20}
We present our analysis on deeper architecuture like ResNet50 with Group Normalization + Weight Standardization on ImageNet20 in Table \ref{app_tab:resnet50}: E and E-SIN with higher shape bias have lower mean corruption accuracy than SIN, whereas SE+IN with lower shape bias reach similar accuracy as SIN. These results show that our findings apply to deeper architectures. Here, all models have 86\%-87\% clean validation accuracy.

\begin{table}[t]
\centering
\begin{tabular}{|l||l|l|l||l|l|}
\hline
 & \multicolumn{3}{c||}{\thead{shuffled image patches(\%)}}  &    & \\  \cline{2-4}
\thead{Network\\} &   \thead{\textbf{$2 \times 2$}}  & \thead{$4 \times 4$}  & \thead{$8 \times 8$} & \thead{Cue conflict \\ shape \#400} & \thead{Mean corruption \\ accuracy (\%)}\\ \hline
\makecell{IN}   & \makecell{88.8}  & \makecell{74.1}  & \makecell{50.7} & \makecell{65}  & \makecell{58}
\\ \hline
\makecell{SIN} & \makecell{83.2}  & \makecell{46}   & \makecell{20.5} & \makecell{141}     & \makecell{\bf 78.2}
\\ \hline
\makecell{E}  & \makecell{\bf 72.5}    & \makecell{34.5}   & \makecell{13.7} & \makecell{164}    & \makecell{52.2}
\\ \hline
\makecell{E-SIN}  & \makecell{73.8}  & \makecell{\bf 25.7} & \makecell{\bf 8.4}  & \makecell{\bf 209}     &  \makecell{72.3}
\\ \hline
\makecell{SE+IN} & \makecell{85.5} & \makecell{66.4} & \makecell{36.5} & \makecell{102}   & \makecell{\bf 78.1}
\\ \hline

\end{tabular}
\caption{Shape based evaluation \& corruption accuracies with ResNet50 architecture.}

	\label{app_tab:resnet50}
\end{table}

\subsubsection{DenseNet121 and MobileNetV2 on ImageNet20}
We extend our findings from ResNet to other architectures like DenseNet121 and MobileNetV2 on ImageNet20 with style randomization. Table \ref{app_tab:densenet121} and Table \ref{app_tab:mobilenetv2} present the shape-based evaluation and mean corruption accuracy (mCA) of DenseNet121 and MobileNetV2 on different settings, respectively. Similar to the results on ResNet, we find that E-SIN exhibits a higher shape bias than other settings but still has a lower mCA than SIN. On the other hand, Superposition (SE+IN) shows a lower shape bias with similar mCA to SIN. These results show that our conclusion is also valid using different neural architectures.

Note that these results also include additional intermediate settings E-IN and E+IN. The explanation of these two settings is as follows:
\begin{itemize}[noitemsep,topsep=0pt,parsep=0pt,partopsep=0pt, leftmargin=*]
	\item E-IN: This is a network training setting similar to E-SIN. In E-SIN, network is pretrained on Edge dataset (E) in the first stage of training for 75 epochs and later finetune on both the stylized images (SIN) and original images (IN) for another 75 epochs in the second stage (please refer Section \ref{sec:experiments} for training details). Similarly, E-IN also pretrains the network on Edge dataset (E) in the first stage but later finetune only on the original images (IN) in the second stage. This setting also shares similarity with network setting E, where network pretrains on edge maps during the first stage and later finetune on both edge maps and original images in the second stage of training.

	\item E+IN: Similar to Superposition (SE+IN), this dataset variant E+IN interpolates edge maps \(I_{\text{E}}\) from Edge dataset (E) with images \(I_{\text{IN}}\) from ImageNet20 (IN): $I_{\text{E+IN}} := (1-\alpha)\cdot I_{\text{E}} + \alpha\cdot I_{\text{IN}}$. We set $\alpha=0.5$. Similar to SE+IN, this setting also pretrains the network on E+IN images in the first stage of training and later finetunes on both E+IN and also on the original images (IN) in the second stage.

\end{itemize}

\begin{table}[t]
\centering
\begin{tabular}{|l||l|l|l||l|l|}
\hline
 & \multicolumn{3}{c||}{\thead{shuffled image patches(\%)}}  &    & \\  \cline{2-4}
\thead{Network\\} &   \thead{\textbf{$2 \times 2$}}  & \thead{$4 \times 4$}  & \thead{$8 \times 8$} & \thead{Cue conflict \\ shape \#400} & \thead{Mean corruption \\ accuracy (\%)}\\ \hline
\makecell{IN}   & \makecell{\bf 62.9}  & \makecell{37.9}  & \makecell{23.5} & \makecell{55}  & \makecell{62.7}
\\ \hline
\makecell{SIN} & \makecell{68.4}  & \makecell{42.5}   & \makecell{21.7} & \makecell{154}     & \makecell{\bf 80.3}
\\ \hline
\makecell{E}  & \makecell{65.2}    & \makecell{35.8}   & \makecell{14.0} & \makecell{167}    & \makecell{60.2}
\\ \hline
\makecell{E-SIN}  & \makecell{66.0}  & \makecell{35.4} & \makecell{\bf 12.0}  & \makecell{\bf 219}     &  \makecell{77.7}
\\ \hline
\makecell{SE+IN} & \makecell{68.3} & \makecell{51.7} & \makecell{32.0} & \makecell{118}   & \makecell{79.0}
\\ \hline
\makecell{E-IN} & \makecell{64.5} & \makecell{\bf 33.7} & \makecell{12.7} & \makecell{164}   & \makecell{62.4}
\\ \hline
\makecell{E+IN} & \makecell{69.0} & \makecell{49.4} & \makecell{31.1} & \makecell{69}   & \makecell{65.0}
\\ \hline

\end{tabular}
\caption{Shape based evaluation \& corruption accuracies with DenseNet121 architecture. All models exhibit clean validation accuracies of 88\%-91\%. Patch shuffled evaluation is conducted on 761 correctly classified validation images by all the networks. It can be observed from these results that the SE+IN have similar mCA to SIN despite having lower shape bias than both SIN and E-SIN.}

	\label{app_tab:densenet121}
\end{table}

\begin{table}[t]
\centering
\begin{tabular}{|l||l|l|l||l|l|}
\hline
 & \multicolumn{3}{c||}{\thead{shuffled image patches(\%)}}  &    & \\  \cline{2-4}
\thead{Network\\} &   \thead{\textbf{$2 \times 2$}}  & \thead{$4 \times 4$}  & \thead{$8 \times 8$} & \thead{Cue conflict \\ shape \#400} & \thead{Mean corruption \\ accuracy (\%)}\\ \hline
\makecell{IN}   & \makecell{53.2}  & \makecell{27.5}  & \makecell{21.8} & \makecell{90}  & \makecell{56.5}
\\ \hline
\makecell{SIN} & \makecell{50.6}  & \makecell{20.4}   & \makecell{9.0} & \makecell{189}     & \makecell{\bf 72.2}
\\ \hline
\makecell{E}  & \makecell{52.6}    & \makecell{18.7}   & \makecell{11.4} & \makecell{160}    & \makecell{56.0}
\\ \hline
\makecell{E-SIN}  & \makecell{\bf 49.1}  & \makecell{\bf 16.1} & \makecell{\bf 6.0}  & \makecell{\bf 212}     &  \makecell{69.0}
\\ \hline
\makecell{SE+IN} & \makecell{57.4} & \makecell{31.6} & \makecell{20.0} & \makecell{138}   & \makecell{70.0}
\\ \hline
\makecell{E-IN} & \makecell{52.4} & \makecell{19.0} & \makecell{9.0} & \makecell{148}   & \makecell{56.6}
\\ \hline
\makecell{E+IN} & \makecell{56.4} & \makecell{33.0} & \makecell{20.0} & \makecell{94}   & \makecell{60.3}
\\ \hline

\end{tabular}
\caption{Shape based evaluation \& corruption accuracies with MobileNetV2 architecture. All models exhibit clean validation accuracies of 86\%-88.5\%. Patch shuffled evaluation is conducted on 700 correctly classified validation images by all the networks. Here, SE+IN shows high corruption robustness close to the one of SIN despite having a lower shape bias than SIN and E-SIN. The reason for the gap between SIN and SE+IN corruption accuracy can be explained as follows: stylized dataset SIN is seen as strongly augmented dataset than the superposition SE+IN (notice perceptual differences in Figure 2) and unlike larger architectures like ResNet18/DenseNet121, the compact nature of the MobileNetV2 architecture is shown to benefit from such stronger data augmentation than the superposition.}

	\label{app_tab:mobilenetv2}
\end{table}

\subsubsection{Evaluation on edge maps of validation set}
We evaluate the performance of models trained under different training settings on edge map based validation set of ImageNet20 and the validation accuracy results are presented in Table \ref{tab_app:eval_edge_validation}.
Note that among different settings in our experiments, edge maps of training data are directly used in E throughout the training, also used for pretraining the network in E-SIN, and stylized edge maps of training data are used during training of Superposition (SE+IN) (please refer Section A.3 for training details). On the other hand, edge maps are not used in any way for training in IN and SIN. From the results of Table \ref{tab_app:eval_edge_validation}, we can observe that the setting E has the highest edge map-based validation accuracy among all others as the edge maps have been used in the entire training process.

\begin{table}[t]
\centering
\begin{tabular}{|l||l|l|l||}
\hline
 & \multicolumn{3}{c||}{\thead{Architectures}}  \\  \cline{2-4}
\thead{Network\\} &   \thead{\textbf{ResNet18 (\%)}}  & \thead{DenseNet121 (\%)}  & \thead{MobileNetV2 (\%)}\\ \hline
\makecell{IN}   & \makecell{18.5}  & \makecell{22.1}  & \makecell{31.8}
\\ \hline
\makecell{SIN} & \makecell{48.6}  & \makecell{51.2}   & \makecell{57.8}
\\ \hline
\makecell{E}  & \makecell{\bf 77.9}    & \makecell{\bf 80.1}   & \makecell{\bf 74.5}
\\ \hline
\makecell{E-SIN} & \makecell{72.4} & \makecell{71.9} & \makecell{67.2}
\\ \hline
\makecell{SE+IN}  & \makecell{47.8}  & \makecell{51.6} & \makecell{54.8}
\\ \hline
\end{tabular}
\caption{Evaluation of different networks on edge map based validation dataset. It can observed that the network E has the highest edge map-based validation accuracy as the edge maps have been used in the entire training process. Here, SE+IN has higher validation accuracy than IN, comparable to SIN. The reason SE+IN has higher validation accuracy is that the stylized edges are used during training of the SE+IN setting.}

	\label{tab_app:eval_edge_validation}
\end{table}

\subsection{Significance of interpolation parameter in superposition}
In Section \ref{sec:experiments} under \emph{Stylization variants}, we discussed about studying the \textbf{role of natural image statistics} by interpolating images \(I_{\text{SE}}\) from SE with images \(I_{\text{IN}}\) from IN: $I_{\text{SE+IN}} := (1-\alpha)\cdot I_{\text{SE}} + \alpha\cdot I_{\text{IN}}$. We show in Table \ref{tab:finetune_corruptions} that such setup (SE+IN) with $\alpha=0.5$ outperforms SIN despite having lower shape bias than all the other networks. In Figure \ref{fig:superposition_alpha}, we show the performance of SE+IN at different values of $\alpha$ on the mean corruption accuracy on all 15 ImageNet-C corruptions at different severity levels. Here, $\alpha=0$ corresponds to the network trained only on SE whereas $\alpha=1$ on IN. As shown in the figure, the two extreme set of values of $\alpha$ i.e., $\alpha$ being very smaller or very larger result drop in performance on corruptions. This implies that the images with balanced details of natural image statistics and style variations is essential for improved performance on corruptions.

\begin{figure}[t]
\centering
\includegraphics[width=0.8\linewidth]{images/results/superposition.png}
\caption{Mean corruption accuracy on ImageNet-C corruptions at different values of $\alpha$ in a stylized dataset SE+IN. Here $\alpha \in \{0,\, 0.05,\, 0.1,\, 0.15,\, 0.25,\, 0.4,\, 0.5,\, 0.6,\, 0.75,\, 0.85,\, 0.9,\, 0.95,\, 1\}$. The solid black line represents the performance of the baseline SIN.
}
\label{fig:superposition_alpha}
\end{figure}

\subsection{Performance of different networks on common corruptions}
We show in Section \ref{sec:robustness_corruptions} that networks E, SE, E-SIN performs poorly on corruptions despite having stronger shape bias than SIN. We also show that superposition of SE with natural images IN (SE+IN) slightly outperforms SIN even having lower shape bias respectively. In Figure \ref{fig:appendix_networks_corruptions}, we show that these results are consistent across all 15 ImageNet-C distortions at different severity levels.
\begin{figure}
\centering
\begin{subfigure}[h]{1.0\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_graphs/1.png}
\end{subfigure}

\begin{subfigure}[h]{1.0\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_graphs/2.png}
\end{subfigure}

\begin{subfigure}[h]{1.0\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_graphs/3.png}
\end{subfigure}

\begin{subfigure}[h]{1.0\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_graphs/4.png}
\end{subfigure}

\begin{subfigure}[h]{1.0\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_graphs/5.png}
\end{subfigure}
\caption{Performance of different networks on ImageNet-C corruptions at different severity levels}
\label{fig:appendix_networks_corruptions}
\end{figure}

\subsection{Finetuning affine parameters on different corruptions}
\label{sec:appendix_finetune_affine}
As mentioned in Section \ref{sec:finetune_affine}, affine parameters of normalization layers in pre-trained IN are fine-tuned on corruptions from ImageNet-C separately. Here, fine-tuning only the affine parameters of IN on a respective corruption greatly improves the mean corruption accuracy on the same corruption across different severity levels. The affine parameters are fine-tuned on \texttt{speckle noise}, \texttt{gaussian blur}, \texttt{snow}, \texttt{frost}, \texttt{fog}, \texttt{brightness}, \texttt{contrast}, \texttt{elastic transform}, \texttt{pixelate}, \texttt{jpeg compression} separately and resulting 10 different IN networks. Each of these fine-tuned networks are evaluated on the same corruption or similar category of corruptions. For e.g, a network fine-tuned on \texttt{frost} is evaluated only on \texttt{frost} and network fine-tuned on \texttt{speckle noise} is also evaluated on the other set of noises like \texttt{gaussian noise}, \texttt{shot noise}, \texttt{impulse noise}. Note that training data of ImageNet20 is augmented with respective corruptions to fine-tune the affine parameters. Each training sample in a mini-batch is augmented with the corresponding corruption at a randomly chosen severity level. The severity parameters that are already pre-defined for every severity level in ImageNet-C are used. A total of 50 epochs are used for fine-tuning the affine parameters on a corruption, starting with learning rate 0.01 and reduce it to 0.001 after 45 epochs. Performance of the networks that are fine-tuned and evaluated on the same or similar category of corruptions at different severity levels are presented in Figure \ref{fig:appendix_corruptions_finetuned}. As shown in the figure, adapting just the distribution on the learned representations from the standard ImageNet20 is sufficient to achieve high performance on respective distribution of corruptions.

\begin{figure}
\centering
\begin{subfigure}[h]{1.0\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_finetune/1.png}
\end{subfigure}

\begin{subfigure}[h]{1.0\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_finetune/2.png}
\end{subfigure}

\begin{subfigure}[h]{0.325\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_finetune/3_1.png}
\end{subfigure}
\begin{subfigure}[h]{0.325\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_finetune/3_2.png}
\end{subfigure}
\begin{subfigure}[h]{0.325\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_finetune/3_3.png}
\end{subfigure}

\begin{subfigure}[h]{0.325\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_finetune/4_1.png}
\end{subfigure}
\begin{subfigure}[h]{0.325\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_finetune/4_2.png}
\end{subfigure}
\begin{subfigure}[h]{0.325\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_finetune/4_3.png}
\end{subfigure}

\begin{subfigure}[h]{0.325\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_finetune/5_1.png}
\end{subfigure}
\begin{subfigure}[h]{0.325\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_finetune/5_2.png}
\end{subfigure}
\begin{subfigure}[h]{0.325\linewidth}
\includegraphics[width=\linewidth, height=0.19\textheight]{images/corruptions_finetune/5_3.png}
\end{subfigure}

\caption{Performance of standard network IN on ImageNet-C corruptions when finetuned the affine parameters on the same corruptions}
\label{fig:appendix_corruptions_finetuned}
\end{figure}

Computer Vision and Pattern Recognition (cs.CV)