A recent development in radio astronomy is to replace traditional dishes with many small antennas. The signals are combined to form one large, virtual telescope. The enormous data streams are cross-correlated to filter out noise. This is especially challenging, since the computational demands grow quadratically with the number of data streams. Moreover, the correlator is not only computationally intensive, but also very I O intensive. The LOFAR telescope, for instance, will produce over 100 terabytes per day. The future SKA telescope will even require in the order of exaflops, and petabits s of I O. A recent trend is to correlate in software instead of dedicated hardware, to increase flexibility and to reduce development efforts. We evaluate the correlator algorithm on multi-core CPUs and many-core architectures, such as NVIDIA and ATI GPUs, and the Cell B.E. The correlator is a streaming, real-time application, and is much more I O intensive than applications that are typically implemented on many-core hardware today. We compare with the LOFAR production correlator on an IBM Blue Gene P supercomputer. We investigate performance, power efficiency, and programmability. We identify several important architectural problems which cause architectures to perform suboptimally. Our findings are applicable to data-intensive applications in general. The processing power and memory bandwidth of current GPUs are highly imbalanced for correlation purposes. While the production correlator on the Blue Gene P achieves a superb 96 of the theoretical peak performance, this is only 16 on ATI GPUs, and 32 on NVIDIA GPUs. The Cell B.E. processor, in contrast, achieves an excellent 92 . We found that the Cell B.E. and NVIDIA GPUs are the most energy-efficient solutions, they run the correlator at least 4 times more energy efficiently than the Blue Gene P. The research presented is an important pathfinder for next-generation telescopes.
Another interesting architecture to implement correlators are FPGAs @cite_21 . LOFAR's on-station correlators are also implemented with FPGAs. Solutions with FPGAs combine good performance with flexibility. A disadvantage is that FPGAs are relatively difficult to program efficiently. Also, we want to run more than just the correlator on our hardware. LOFAR is the first of a new generation of software telescopes, and how the processing is done best is still the topic of research, both in astronomy and computer science. We perform the initial processing steps on FPGAs already, but find that this solution is not flexible enough for the rest of the pipeline. For LOFAR, currently twelve different processing pipelines are planned. For example, we would like to do the calibration of the instrument and pulsar detection online on the same hardware, before storing the data to disk. We even need to support multiple different observations simultaneously. All these issues together require enormous flexibility from the processing solution. Therefore, we restrict us to many-cores, and leave application-specific instructions and FPGAs as future work. Once the processing pipelines are fully understood, future instruments, such as the SKA, will likely use ASICs.
@cite_21
2156412335
This paper describes a correlator that is optimized for the Xilinx Virtex-4 SX FPGA, and its application in the SKAMP radio telescope at the Molonglo Radio Observatory. The digital backend of the SKAMP telescope consists of more than 800 Virtex-4 FPGAs. Correlation is performed between each and every pairing of antenna inputs, so the SKAMP telescope, with its 384 inputs, has approximately 74,000 antenna correlations; with 100 MHz of input bandwidth from each antenna this requires real-time processing of more than 7 tera complex multiply-accumulates per second. The correlation cell described takes advantage of the hard IP blocks found within the Virtex-4 FPGA to perform one 4+4-bit complex correlation per cycle at a clock rate exceeding 256 MHz. At the core of each cell is an efficient 4-bit signed complex multiplier, implemented using the 18-bit signed multiplier of the Virtex-4 DSP slice, and a short term accumulator, implemented using the adjacent Block RAM. Nearly 30,000 correlation cells are instantiated across 192 Virtex-4 SX35 devices in order to process all the data from the SKAMP telescope.