t are required to enable hundreds of logical processors to handle millions of I/Os per second with microsecond response.
As an example, consider a common design pattern: a linked-list used as a work queue which is protected by a spinlock. The lock has to be takenwhenever work is added to the queue or removed from it. On a busy system this can result in almost continuous contention for the lock, which means that CPUs are tied up just waiting for access. Basic spinlocks don’t provide the concept of ‘first come first served’, so an individual CPU can get delayed for an extended period, holding up I/O requests indefinitely and affecting application response time. On a lightly loaded system, of course, there is less contention and the issues are simply not seen.
There are some well-known techniques that can be adopted, including using a queued spinlock which incorporates the concept of fairness and lock-free programming based around compare-and-set instructions. At first glance, these seem to provide elegant solutions to the problem, but none of them really hold up well under this level of load. Even a simple memory access can have a significant impact on performance if it misses cache or falls in the wrong cache-line and on a multi-socket system the cost of an access can also depend upon which CPU is making it.
Innovative Approaches for Performing Parallel IO Operations
DataCore has developed a number of innovative techniques to maximize performance, whilst still maintaining the required level of consistency. One of the key principles is that state is maintained in the domain with the lowest contention until there is a necessity to expose it more widely. The protection techniques within each domain are matched to the expected level of contention. This all has to be achieved whilst retaining global visibility to the state.
This may sound simple, but there’s a lot of complexity hidden in the detail; multithreading on highly parallel systems is far from trivial. In fact, it’s so difficult that the trend now is towards scaling applications by building them from independent, containerized, microservices … an environment just crying out for a new kind of storage stack, but that’s a story for another day!
DataCore’s most recently issued patent in this area is:
U.S. Patent Number 10,409,640 – Methods and Apparatus for Data Request Scheduling in Performing Parallel IO Operations.
It’s part of a set of related applications covering techniques for improving I/O performance on a multi-core system, including:
U.S. Patent Number 10,318,354 – Methods and Apparatus for Command List Processing in Performing Parallel IO Operations
What do these patents mean in practice? A software-defined storage product that delivers greater I/O throughput, with better and more deterministic latency, whilst using less resources. It’s technical innovations like these that cause nearly 80 percent of our customers to report 300-1000 percent storage performance improvements and that elevate DataCore from being just a provider of storage virtualization into the Authority on Software-Defined Storage.
Request a live demo or fully-functional free trial of the most advanced technology to accelerate performance, increase resource efficiency, and achieve zero-downtime availability for your storage systems.