BACKGROUND

Some systems may use variable deduplication in fixed block storage arrays with sector level deduplication for fixed blocks. Although the method may have increased dramatically the dedupe data reduction, it may require additional resources (e.g., memory and CPU) and as a result impacts the IOs per second (IOPS) performance of the array.

BRIEF SUMMARY OF DISCLOSURE

In one example implementation, a method, performed by one or more computing devices, may include but is not limited to identifying a plurality of blocks. At least one heuristic associated with at least a portion of the plurality of blocks may be determined. It may be determined whether at least the portion of the plurality of blocks is a candidate for deduplication based upon, at least in part, the at least one heuristic. At least the portion of the plurality of blocks may be deduplicated based upon, at least in part, the at least one heuristic.

One or more of the following example features may be included. The at least one heuristic may include a distance function associated with at least the portion of the plurality of blocks. The at least one heuristic may include a digital entropy value associated with at least the portion of the plurality of blocks resulting from the distance function. The distance function may include alphabet encoding. Additional data comparison on the candidate may be performed prior to deduplication. At least the portion of the plurality of blocks may be the candidate for deduplication when the digital entropy value associated with at least the portion of the plurality of blocks resulting from the distance function is less than a threshold. At least the portion of the plurality of blocks may not be the candidate for deduplication when the digital entropy value associated with at least the portion of the plurality of blocks resulting from the distance function is more than a threshold.

In another example implementation, a computing system may include one or more processors and one or more memories configured to perform operations that may include but are not limited to identifying a plurality of blocks. At least one heuristic associated with at least a portion of the plurality of blocks may be determined. It may be determined whether at least the portion of the plurality of blocks is a candidate for deduplication based upon, at least in part, the at least one heuristic. At least the portion of the plurality of blocks may be deduplicated based upon, at least in part, the at least one heuristic.

One or more of the following example features may be included. The at least one heuristic may include a distance function associated with at least the portion of the plurality of blocks. The at least one heuristic may include a digital entropy value associated with at least the portion of the plurality of blocks resulting from the distance function. The distance function may include alphabet encoding. Additional data comparison on the candidate may be performed prior to deduplication. At least the portion of the plurality of blocks may be the candidate for deduplication when the digital entropy value associated with at least the portion of the plurality of blocks resulting from the distance function is less than a threshold. At least the portion of the plurality of blocks may not be the candidate for deduplication when the digital entropy value associated with at least the portion of the plurality of blocks resulting from the distance function is more than a threshold.

In another example implementation, a computer program product may reside on a computer readable storage medium having a plurality of instructions stored thereon which, when executed across one or more processors, may cause at least a portion of the one or more processors to perform operations that may include but are not limited to identifying a plurality of blocks. At least one heuristic associated with at least a portion of the plurality of blocks may be determined. It may be determined whether at least the portion of the plurality of blocks is a candidate for deduplication based upon, at least in part, the at least one heuristic. At least the portion of the plurality of blocks may be deduplicated based upon, at least in part, the at least one heuristic.

One or more of the following example features may be included. The at least one heuristic may include a distance function associated with at least the portion of the plurality of blocks. The at least one heuristic may include a digital entropy value associated with at least the portion of the plurality of blocks resulting from the distance function. The distance function may include alphabet encoding. Additional data comparison on the candidate may be performed prior to deduplication. At least the portion of the plurality of blocks may be the candidate for deduplication when the digital entropy value associated with at least the portion of the plurality of blocks resulting from the distance function is less than a threshold. At least the portion of the plurality of blocks may not be the candidate for deduplication when the digital entropy value associated with at least the portion of the plurality of blocks resulting from the distance function is more than a threshold.

The details of one or more example implementations are set forth in the accompanying drawings and the description below. Other possible example features and/or possible example advantages will become apparent from the description, the drawings, and the claims. Some implementations may not have those possible example features and/or possible example advantages, and such possible example features and/or possible example advantages may not necessarily be required of some implementations.

