Comparison of algorithms for determination of rotation measure and
  Faraday structure I. 1100 - 1400 MHz
  (abridged) We run a Faraday structure determination data challenge to
benchmark the currently available algorithms including Faraday synthesis
(previously called RM synthesis in the literature), wavelet, compressive
sampling and $QU$-fitting. The frequency set is similar to POSSUM/GALFACTS with
a 300 MHz bandwidth from 1.1 to 1.4 GHz. We define three figures of merit
motivated by the underlying science: a) an average RM weighted by polarized
intensity, RMwtd, b) the separation $\Delta\phi$ of two Faraday components and
c) the reduced chi-squared. Based on the current test data of signal to noise
ratio of about 32, we find that: (1) When only one Faraday thin component is
present, most methods perform as expected, with occasional failures where two
components are incorrectly found; (2) For two Faraday thin components,
QU-fitting routines perform the best, with errors close to the theoretical ones
for RMwtd, but with significantly higher errors for $\Delta\phi$. All other
methods including standard Faraday synthesis frequently identify only one
component when $\Delta\phi$ is below or near the width of the Faraday point
spread function; (3) No methods, as currently implemented, work well for
Faraday thick components due to the narrow bandwidth; (4) There exist
combinations of two Faraday components which produce a large range of
acceptable fits and hence large uncertainties in the derived single RMs; in
these cases, different RMs lead to the same Q, U behavior, so no method can
recover a unique input model.
