TECHNICAL FIELD

The present disclosure generally relates to storage systems supporting a plurality of hosts and, more particularly, to dynamic allocation of storage resources in response to host requests.

BACKGROUND

Multi-device storage systems utilize multiple discrete data storage devices, generally disk drives (solid-state drives, hard disk drives, hybrid drives, tape drives, etc.) for storing large quantities of data. These multi-device storage systems are generally arranged in an array of drives interconnected by a common communication fabric and, in many cases, controlled by a storage controller, redundant array of independent disks (RAID) controller, or general controller, for coordinating storage and system activities across the array of drives. The data stored in the array may be stored according to a defined RAID level, a combination of RAID schemas, or other configurations for providing desired data redundancy, performance, and capacity utilization. In general, these data storage configurations may involve some combination of redundant copies (mirroring), data striping, and/or parity (calculation and storage), and may incorporate other data management, error correction, and data recovery processes, sometimes specific to the type of disk drives being used (e.g., solid-state drives versus hard disk drives).

There is an emerging trend in the storage industry to deploy disaggregated storage. Disaggregated storage brings significant cost savings via decoupling compute and storage node life cycles and allowing different nodes or subsystems to have different compute to storage ratios. In addition, disaggregated storage allows significant flexibility in migrating compute jobs from one physical server to another for availability and load balancing purposes.

Disaggregated storage has been implemented using a number of system architectures, including the passive Just-a-Bunch-of-Disks (JBOD) architecture, the traditional All-Flash Architecture (AFA), and Ethernet Attached Bunch of Flash (EBOF) disaggregated storage, which typically uses specialized chips from Mellanox or Kazan to translate commands from external NVMe-OF (Non-Volatile Memory Express over Fabrics) protocol to internal NVMe (NVM Express) protocol. These architectures may be configured to support various Quality of Service (QoS) metrics and requirements to support host applications, often supporting a plurality of host systems with different workload requirements.

The systems may be deployed in data centers to support cloud computing services, such as platform as a service (PaaS), infrastructure as a service (IaaS), and/or software as a service (SaaS). Data centers and their operators may offer defined (and sometime contractually guaranteed) QoS with responsive, on-demand provisioning of both hardware and software resources in multi-tenant systems. Various schemes for dynamic resource allocation may be used at different levels of the system hierarchies and roles. Prior resource allocation schemes may not provide optimal allocation of non-volatile memory resources among a plurality of hosts with differing workloads in a multi-tenant system.

Therefore, there still exists a need for storage systems with dynamic resource allocation configurations for back-end non-volatile memory resources.

SUMMARY

Various aspects for dynamic allocation of storage resources based on connection type and at connection time are described. More particularly, responsive to connection requests, a controller may determine desired ratios between connection types and resource pools and cap the connections allowed for a subset of each resource pool, such as processing cores and/or queue pairs. Connections exceeding the maximums for the resource subsets may be failed.

One general aspect includes a system that includes: a processor; a memory; a storage interface configured to communicate with a plurality of data storage devices and support a first connection type and a second connection type; and a resource allocation engine. The resource allocation engine is configured to: determine, for a new storage connection request, a connection type selected from the first connection type and the second connection type; determine, based on the connection type, an allocated number for a first connection type subset of a first pool of allocated storage resources; and establish, responsive to the new storage connection request, a new storage connection with the first connection type subset of the first pool of allocated storage resources.

Implementations may include one or more of the following features. The resource allocation engine may be further configured to: determine a maximum number of connections for the first connection type subset of the first pool of allocated storage resources; determine whether the new storage connection request exceeds the maximum number of connections for the first connection type subset of the first pool of allocated storage resources; and fail, responsive to the new storage connection request exceeding the maximum number of connections for the first connection type subset of the first pool of allocated storage resources, the new storage connection request. The resource allocation engine may be further configured to: determine, based on the connection type, an allocated number for a second connection type subset of the first pool of allocated storage resources; determine a maximum number of connections for the second connection type subset of the first pool of allocated storage resources; determine whether the new storage connection request exceeds the maximum number of connections for the second connection type subset of the first pool of allocated storage resources; and establish, responsive to the new storage connection request not exceeding the maximum number of connections for the second connection type subset of the first pool of allocated storage resources, the new storage connection with the first pool of allocated storage resources. The first connection type subset may correspond to the first connection type using a first connection protocol, the second connection type subset may correspond to the second connection type using a second connection protocol, the resource allocation engine may be further configured to determine an allocation ratio between a first connection type and a second connection type for the first pool of allocated resources; and the maximum number of connections of the first connection type subset and the maximum number of connections of the second connection type subset may correspond to the allocation ratio applied to the first pool of allocated resources. The first connection type may be a remote direct memory access connection for accessing the plurality of data storage devices and the second connection type is a transmission control protocol connection for accessing the plurality of data storage devices. The resource allocation engine may be further configured to: monitor read/write operations executed through the storage interface; determine a read/write ratio based on the monitored read/write operations; adjust, based on the read/write ratio, a resource ratio; and recalculate, based on the resource ratio, the maximum number of connections for the first connection type subset of the first pool of allocated storage resources. The processor may include a plurality of processor cores, the first pool of allocated resources may include assignments to the plurality of processor cores, and the first connection type subset of the first pool of allocated resources may correspond to a core ratio allocated to a first connection type. The resource allocation engine may be further configured to: determine, based on the connection type, an allocated number for a first connection type subset of a second pool of allocated storage resources; determine a maximum number of connections for the first connection type subset of the second pool of allocated storage resources; determine whether the new storage connection request exceeds the maximum number of connections for the first connection type subset of the second pool of allocated storage resources; and establish, responsive to the new storage connection request not exceeding the maximum number of connections for the first connection type subset of the second pool of allocated storage resources, a new storage connection with the second pool of allocated storage resources. The memory may support a plurality of queue pairs for managing command queues and storage queues for the plurality of data storage devices, the second pool of allocated resources may include assignments to the plurality of queue pairs, and the first connection type subset of the second pool of allocated resources may correspond to a queue pair ratio allocated to the first connection type. The resource allocation engine may be further configured to: determine the core ratio from a set of configuration parameters stored in the memory; and determine the queue pair ratio from the set of configuration parameters stores in the memory; and the core ratio is different from the queue pair ratio.

Another general aspect includes a computer-implemented method that includes: configuring a storage node for communication with a plurality of data storage devices, where the storage node controls at least one pool of storage resources for allocation to host storage connections; determining, for a new storage connection request, a connection type selected from a first connection type and a second connection type; determining, based on the connection type, a first connection type subset of a first pool of allocated storage resources; and establishing, responsive to the new storage connection request, a new storage connection with the first connection type subset of the first pool of allocated storage resources.

Implementations may include one or more of the following features. The computer-implemented method may include: determining a maximum number of connections for the first connection type subset of the first pool of allocated storage resources; determining whether the new storage connection request exceeds the maximum number of connections for the first connection type subset of the first pool of allocated storage resources; and failing, responsive to the new storage connection request exceeding the maximum number of connections for the first connection type subset of the first pool of allocated storage resources, the new storage connection request. The computer-implemented method may include: determining, based on the connection type, a second connection type subset of the first pool of allocated storage resources; determining a maximum number of connections for the second connection type subset of the first pool of allocated storage resources; determining whether the new storage connection request exceeds the maximum number of connections for the second connection type subset of the first pool of allocated storage resources; and establishing, responsive to the new storage connection request not exceeding the maximum number of connections for the second connection type subset of the first pool of allocated storage resources, the new storage connection with the first pool of allocated storage resources. The computer-implemented method may include determining an allocation ratio between a first connection type and a second connection type for the first pool of allocated resources, where: the first connection type subset corresponds to the first connection type using a first connection protocol; the second connection type subset corresponds to the second connection type using a second connection protocol; and the maximum number of connections of the first connection type subset and the maximum number of connections of the second connection type subset correspond to the allocation ratio applied to the first pool of allocated resources. The first connection type may be a remote direct memory access connection for accessing the plurality of data storage devices and the second connection type may be a transmission control protocol connection for accessing the plurality of data storage devices. The computer-implemented method may include assigning each host storage connection to a processor core of a plurality of processor cores, where the first pool of allocated resources includes assignments to the plurality of processor cores and the first connection type subset of the first pool of allocated resources corresponds to a core ratio allocated to the first connection type. The computer-implemented method may include: determining, based on the connection type, a first connection type subset of a second pool of allocated storage resources; determining a maximum number of connections for the first connection type subset of the second pool of allocated storage resources; determining whether the new storage connection request exceeds the maximum number of connections for the first connection type subset of the second pool of allocated storage resources; and establishing, responsive to the new storage connection request not exceeding the maximum number of connections for the first connection type subset of the second pool of allocated storage resources, the new storage connection with the second pool of allocated storage resources. The computer-implemented method may include assigning at least one queue pair from a plurality of queue pairs to each host connection, where: the second pool of allocated resources includes assignments of the plurality of queue pairs; the plurality of queue pairs manages command queues and storage queues for the plurality of data storage devices; and the first connection type subset of the second pool of allocated resources corresponds to a queue pair ratio allocated to the first connection type. The computer-implemented method may include: monitoring read/write operations executed through the storage interface; determining a read/write ratio based on the monitored read/write operations; adjusting, based on the read/write ratio, a resource ratio; and recalculating, based on the resource ratio, the maximum number of connections for the first connection type subset of the first pool of allocated storage resources.

Still another general aspect includes a storage system that includes: a processor; a memory; a plurality of storage devices; means for determining, for a new storage connection request, a connection type selected from a first connection type and a second connection type; means for determining, based on the connection type, a first connection type subset of a first pool of allocated storage resources; means for determining a maximum number of connections for the first connection type subset of the first pool of allocated storage resources; means for determining whether the new storage connection request exceeds the maximum number of connections for the first connection type subset of the first pool of allocated storage resources; and means for establishing, responsive to the new storage connection request not exceeding the maximum number of connections for the first connection type subset of the first pool of allocated storage resources, a new storage connection with the first pool of allocated storage resources.

The various embodiments advantageously apply the teachings of data storage devices and/or multi-device storage systems to improve the functionality of such computer systems. The various embodiments include operations to overcome or at least reduce the issues previously encountered in storage arrays and/or systems and, accordingly, are more reliable and/or efficient than other computing systems. That is, the various embodiments disclosed herein include hardware and/or software with functionality to improve shared access to non-volatile memory resources by host systems in multi-tenant storage systems, such as by using connection type to dynamically allocate resource pools in configurable ratios. Accordingly, the embodiments disclosed herein provide various improvements to storage networks and/or storage systems.

It should be understood that language used in the present disclosure has been principally selected for readability and instructional purposes, and not to limit the scope of the subject matter disclosed herein.

