In his witty account of cluster computing, In Search of Clusters, author and IBM Distinguished Engineer Greg Pfister notes that software, not hardware, stands in the way of constructing high-availability, high-performance systems from inexpensive commodity components. Since the publication of Pfister's book seven years ago, high availability and scalability have become a requirement not only for large enterprises processing millions of transactions an hour, but equally for medium and small businesses.
Scalable and highly availabile systems traditionally required not only expensive hardware, but specially trained technical operators carefully watching over those systems, tuning them, and ensuring their smooth operation. Since smaller firms could not afford that expense, they had no choice but to compromise on system availability and choose less capable hardware and software for their information processing needs.
However, a quiet revolution is changing all that, making high-availability systems a reality even for the smallest enterprises. This article describes a high-availability, scalable clustering technique using Jini technology. As a tool for dynamic networking, Jini technology found a niche as middleware in support of highly available and scalable enterprise systems that operate on clusters built on commodity hardware. The architecture presented in this article, and developed by Boomi Software, uses Jini technology in that manner, and forms the core of that company's system integration server product.
A cluster is a collection of independent computers working together in support of a common task. A cluster's key benefit is that adding more nodes to a cluster increases the cluster's computing capability while also making the cluster more resilient to failure. Because a cluster can be constructed from very low-cost hardware, clustering emerged as a popular high-availability and scalability technique.
Each node in a cluster maintains its own CPU, memory, and often even storage. Node hardware can come in great diversity, from specialized blade computers to off-the-shelf commodity PCs. Each cluster node runs its own operating system copy. The interconnection network between the nodes can also assume a large variety, from 100Mb/s Ethernet to 10Gb Ethernet, or specialized cluster networks, such as Myrinet.
Several techniques help transform the collection of independent computers in a cluster to a more or less unified computing device. A favorite clustering technique aims to virtualize cluster resources. Virtualization abstracts out hardware and operating system resources from a single node to the level of a cluster. Consider, for example, a file system that could be virtualized by mounting a shared storage device identically on every cluster node. Thus, a file saved on the virtualized filesystem mounted at, say, /home/user, would be accessible on any cluster node with an identical file path.
Virtualized cluster resources, such as a distributed and networked file system, offer shared capabilities by means of services. Those services, in turn, correspond to processes. A file system daemon process, for instance, offers a file system service. Each cluster node running the appropriate network client can invoke the file system service across the network and make the file system available locally.
Common cluster services, such as a file system or even I/O device access, are often defined by means of a network communication protocol. For instance, the Network File System (NFS) defines a network mounting protocol in terms of message exchanges between the NFS daemon process and its clients.
While virtualization of cluster resources has traditionally been achieved via operating-system or protocol-specific means, a higher-level service virtualization layer often hides the details of a service's communication protocol, and instead exposes that service based on a conceptual service type. For instance, invoking the
ls command on a file system produces similar results whether the file system is mounted via the NFS daemon or is local to th