The ASKAP/EMU Source Finding Data Challenge
  The Evolutionary Map of the Universe (EMU) is a proposed radio continuum
survey of the Southern Hemisphere up to declination +30 deg., with the
Australian Square Kilometre Array Pathfinder (ASKAP). EMU will use an automated
source identification and measurement approach that is demonstrably optimal, to
maximise the reliability, utility and robustness of the resulting radio source
catalogues. As part of the process of achieving this aim, a "Data Challenge"
has been conducted, providing international teams the opportunity to test a
variety of source finders on a set of simulated images. The aim is to quantify
the accuracy of existing automated source finding and measurement approaches,
and to identify potential limitations. The Challenge attracted nine independent
teams, who tested eleven different source finding tools. In addition, the
Challenge initiators also tested the current ASKAPsoft source-finding tool to
establish how it could benefit from incorporating successful features of the
other tools. Here we present the results of the Data Challenge, identifying the
successes and limitations for this broad variety of the current generation of
radio source finding tools. As expected, most finders demonstrate completeness
levels close to 100% at 10sigma dropping to levels around 10% by 5sigma. The
reliability is typically close to 100% at 10sigma, with performance to lower
sensitivities varying greatly between finders. All finders demonstrate the
usual trade-off between completeness and reliability, whereby maintaining a
high completeness at low signal-to-noise comes at the expense of reduced
reliability, and vice-versa. We conclude with a series of recommendations for
improving the performance of the ASKAPsoft source-finding tool.
