Forget the needle, take the haystack Newgen Systems. Newgen Systems isn't exactly a household name. The Melbourne-based information technology company is a modest enterprise, virtually unknown outside the world of telecommunications and IT professionals. The company's publicity brochures and industry presentations are bland and not particularly informative. They describe Newgen as "a systems integration and network communications company" and a "niche player" providing "innovative solutions for complex networking and IT problems." Robert Perin, the Australian telecommunications engineer who founded and still owns the firm may be a little more forthcoming, describing Newgen as a "network analytics, monitoring and security organisation," but he still doesn't give much away. What Newgen lacks in public profile, however, it enjoys in strategic positioning. It sits in the shadows between the Australian government, Australia's big telecommunications and internet service providers, especially Telstra, and the supplier of some of the most advanced mass surveillance technology available in the global marketplace. Business has been good for Newgen. But that same, highly profitable position also puts the company deep in the burgeoning controversy about government surveillance and spying triggered by the dramatic disclosures of United States intelligence contractor Edward Snowden. Among the many revelations from Snowden's material, slowly trickling out through a range of media organisations worldwide, are the deep relationships between the intelligence and security agencies of the so-called "five-eyes" alliance of the United States, Britain, Australia, Canada and New Zealand, and major telecommunications and IT companies in all five countries. This has led some commentators to talk of the emergence of a "surveillance industrial complex" and there's no doubt that today's modern intelligence machinery could not achieve its all-pervasive reach, across the globe and into intimate details of individual lives, without the assistance of the companies that provide the foundations of the ongoing IT revolution. Spying is big business. But some of these connections have small beginnings and have often slipped under the radar. Newgen is a case in point. Until recently, you wouldn't find the name Newgen emblazoned on the front of any corporate office. For six years the company's principal place of business was Perin's home in suburban Doncaster. Only in May this year did the company acquire a modest front office in an office block on Burwood Road in Hawthorn East. Leaked Newgen documents show early company meetings between Perin and his close colleagues, fellow IT industry veterans Bill Crocaris and Brad Hill, were held at the "Nook" at the Matthew Flinders Hotel in Chadstone. Behind these modest beginnings was Newgen's alliance with the Silicon Valley-based company Gigamon, developer of some of the world's most advanced IT network monitoring technology. Gigamon was established in 2003 and began shipping its distinctive "orange boxes," highly sophisticated modules that switch and copy data flows within large telecommunication and computer networks, two years later. The company describes itself as "a world leader in Traffic Visibility Fabric solutions," enabling network managers to achieve "complete network traffic visibility" through "100 per cent packet capture" without impacting on network performance. The language describing Gigamon's products is often highly technical, if not impenetrable to non-IT specialists. Put simply, Gigamon's technology addresses a basic problem for modern network managers and intelligence agencies: how to look into vast torrents of data flowing through computer networks, data centres and along high-speed fibre-optical cables without impeding the performance of such networks. In short, how do you find a needle in a haystack? Or more to the point, how do you find a few bits of a needle scattered among many, many haystacks? The answer is to vacuum up all the haystacks, and do so without creating bottlenecks or other problems. To achieve this, data flows are mirrored, copied and then subjected to filtering and analysis using a wide variety of tools without impeding the efficiency of networks. Confidential Newgen documents describe the Gigamon technology as "a vacuum cleaner" that "sucks up unsynchronised and disaggregated data, filters and sorts it to recreate the original puzzle." This novel technology has a wide range of applications, including network security and management, but significantly includes telecommunications and internet data interception. Being one of the first to bring a new product into the marketplace is a huge advantage and the past decade has been a boom time for Gigamon. The company's success has been fuelled by the global explosion of telecommunications traffic and the rapidly growing number and scale of data centres. Growth has also been driven by what Gigamon describes as "unrelenting" government demands for "unrestricted access for lawful interception in all manners of digital communications." Gigamon now exports its hardware to more than 40 countries. The company has offices in the US, the UK, Russia, China, Hong Kong and Singapore as well as in Australia, where Newgen acts as its representative. Gigamon is coy about its clients but leaked Newgen documents show they include major US telecommunications and internet service providers, including AT&T, Sprint, Comcast and Time Warner Cable; computer and networking companies including Apple and eBay; and major financial institutions such as MasterCard and Merrill Lynch. Government agencies and defence contractors are also prominent among Gigamon's customers, which in the US include the Defence Department, the National Security Agency, the Defence Advanced Projects Research Agency and the Defence Information Systems Agency as well as aerospace giants Raytheon and Lockheed Martin; and telecommunications equipment supplier Harris. The Royal Canadian Mounted Police is another customer. Interestingly, Gigamon began to move into the Russian market in 2009 with a company spokesperson declaring "there is a bright future for Gigamon in the Russian Federation." The company hasn't revealed its Russian customer list, but at a trade show in the US in late 2011 Gigamon representatives gave a presentation in which they mentioned "they'd just done a huge install with Russia . . . allowing the government to monitor data of its citizens." When Gigamon listed on the New York Stock Exchange in June this year the company was valued at more than $680 million. Robert Perin won't go into the details of how Newgen became the sole supplier of Gigamon technology in Australia and New Zealand other than to say he benefited from "industry contacts." Whatever the precise connections, Newgen secured access to a highly innovative technology that was in demand as Australia's telecommunications sector was undergoing rapid expansion and change and as Australia's intelligence agencies were ramping up operations in the "war on terrorism." Early Newgen business plans show telecommunications and surveillance were the firm's top marketing priorities. Industry contacts helped and Telstra quickly emerged as Newgen's main customer with the first sales of Gigamon hardware taking place in early 2007. Although Telstra has purchased a variety of Gigamon systems, a key purpose is "lawful interception" to provide data to ASIO, the Australian Federal Police and state law enforcement agencies. In April 2010, Newgen submitted a proposal to Telstra's "Special Projects" group for the installation of Gigamon hardware at 24 metropolitan locations around Australia to meet "a government mandated regulatory requirement" for interception coverage as Telstra upgraded its network. An initial roll-out of new Gigamon systems for Telstra's top 10 exchanges was costed at $2.7 million and Telstra's purchases from Newgen in 2010 were worth more than $3.5 million. Vodafone Australia has also been a Newgen customer. However, Newgen documents suggest Vodafone's purchases of Gigamon hardware have been much less extensive than those of Telstra. Defence's published contract lists are uninformative, referring only to purchases from Newgen of "computer equipment and accessories," "communications devices" and "data access switches." Only in one case is there a public reference, apparently inadvertent, to the acquisition of "Gigamon hardware." However, the public record does show that Newgen's sales to Defence have been made to the Defence Intelligence and Security Group that includes the top secret Australian Signals Directorate, and by the Electronic Warfare Branch of the Defence Materiel Organisation. When interviewed by Fairfax Media, Perin said that he doesn't know what the Defence Department does with Newgen's products. "They interrogate us about the technology, but they don't tell us," he says. However, he also said that on one occasion Newgen had "tracked" the movement of a piece of Gigamon equipment and using Google Maps discovered it was being used at "a Defence facility in Western Australia." Moreover, notes written by another Newgen representative involved in training Defence personnel do refer to the Australian Signals Directorate's use of Gigamon equipment for "LI" [lawful interception] including in the context of "HC [high capacity fibre]-optic cables - o/s [overseas] links 10 Gbps [gigabits per second], 40 Gbps, 100 Gbps." Newgen is certainly fully aware of the surveillance application of its products for domestic telecommunications interception and data collection being done by Telstra and other companies. Indeed, one former Newgen employee told Fairfax Media that Gigamon systems supported telecommunications interception through the Telstra network that helped thwart the 2009 Holsworthy Barracks terrorist plot in which four Melbourne men allegedly linked with the Somali-based al-Shabaab terrorist group planned a mass killing of Australian Defence Force personnel. Police and prosecutors drew on what they described as "voluminous" telephone and text intercepts and internet data to build their case against the plotters. The former Newgen employee insists that the "very comprehensive" collection of information could not have been achieved without "complete access to all telecommunications data - to find the needle you have to take the entire haystack." In May last year, Gigamon congratulated Newgen for supporting its "leadership position" in the Australian market "with a strong emphasis on telcos." Both companies hope for bigger things. "With the history of accomplishment that we share with Newgen, we know they have the expertise, knowledge and support infrastructure to continue our expansion in the region," Gigamon's vice-president of sales in the Asia Pacific and Japan, David Sajoto, said. Having maintained a low profile, Newgen probably won't appreciate much public attention, and it should be emphasised that the company is a legitimate enterprise providing products that, in Australia, support lawful government intelligence operations. Whether such all-encompassing collection efforts are justifiable is, however, a very legitimate matter of public debate. It's also worth noting that Newgen and Gigamon are just a small part of a multibillion dollar a year trade in which private firms are selling spying tools and mass surveillance technologies to governments of all persuasions to scoop up millions of emails, text messages and phone calls. Research by Privacy International, an independent watchdog group focused on the proliferation of surveillance technology, has found more than 338 companies offering a total of 97 different technologies worldwide. Selling such equipment is perfectly legal and these companies say the new technologies are part of the fabric of modern IT systems and help governments defeat terrorism and crime. But human rights and privacy campaigners are concerned that oppressive regimes can use such technology to clamp down on critics and democracy advocates. For example, Libya's former leader Muammar Gaddafi is known to have used off-the-shelf surveillance equipment to clamp down on dissidents. Gigamon's sales to Russia and China may be another concern. Some privacy and human rights activists are calling for mass surveillance technology to be subject to the same strict export controls faced by arms manufacturers. Philip Dorling is a senior writer.