BACKGROUND

Many companies and other organizations operate computer networks that interconnect numerous computing systems to support their operations, such as with the computing systems being co-located (e.g., as part of a local network) or instead located in multiple distinct geographical locations (e.g., connected via one or more private or public intermediate networks). For example, data centers housing significant numbers of interconnected computing systems have become commonplace, such as private data centers that are operated by and on behalf of a single organization, and public data centers that are operated by entities as businesses to provide computing resources to customers. Some public data center operators provide network access, power, and secure installation facilities for hardware owned by various customers, while other public data center operators provide “full service” facilities that also include hardware resources made available for use by their customers.

The advent of virtualization technologies for commodity hardware has provided benefits with respect to managing large-scale computing resources for many customers with diverse needs, allowing various computing resources to be efficiently and securely shared by multiple customers. For example, virtualization technologies may allow a single physical virtualization host to be shared among multiple users by providing each user with one or more “guest” virtual machines hosted by the single virtualization host. Each such virtual machine may represent a software simulation acting as a distinct logical computing system that provides users with the illusion that they are the sole operators of a given hardware computing resource, while also providing application isolation and security among the various virtual machines. Instantiating several different virtual machines on the same host may also help increase the overall hardware utilization levels at a data center, leading to higher returns on investment.

A network-accessible service that provides virtualized computing functionality may have to manage hundreds of thousands, or even millions, of virtual machines concurrently. Some of the virtual machines, established for long-running client applications, may remain operational for weeks, months, or years, and may each consume a substantial amount of computing, memory and storage resources. Other virtual machines may be short-lived, e.g., lasting for just a few minutes or seconds to perform a specific task on behalf of a client, and may each consume a smaller amount of resources. The demand for different types of virtual machine may vary substantially over time. Designing a virtualization service that can efficiently handle large, dynamically changing mixes of virtual machines with widely differing functional and performance requirements remains a non-trivial technical challenge.

