Radio Astronomy is a field that generates huge volumes of data. Stuart Weston, of the Institute for Radio Astronomy and Space Research at AUT, needed a better way to manage that data.
Stuart’s research is conducted using data from the AUT’s Warkworth Radio Astronomical Observatory. Collected from two antennae at the Observatory, the many terabytes of data must be processed to correlate the signals. Radio Astronomy research centres overseas use high performance computing (HPC) clusters to perform this function.
Such clusters must be purchased, housed, powered, monitored, upgraded and maintained at considerable expense – far out of reach for New Zealand scientists. The nearest existing cluster is in Pawsey, Western Australia but AUT wants to start processing the data here in New Zealand. So, Stuart used some standard servers provided by AUT over a regular data link. This method works, but can be slow and cumbersome.