TECHNICAL FIELD

The present disclosure relates to computer technologies, and in particular, to the storage field.

BACKGROUND

Distributed storage modes such as cloud storage are increasingly popular. With explosive growth of information, a scale of data of a user in a cloud storage system becomes larger. An increasing data volume causes costs pressure on both a storage service provider and the user.

To improve data reliability, redundancy is performed on data using an erasure code (EC) technology in the cloud storage system. The redundant data is divided into a data fragment and a parity fragment (which are collectively referred to as a fragment). When a total quantity of damaged fragments does not exceed a threshold, for any quantity of fragments are damaged, a damaged fragment can be reconstructed based on an undamaged fragment. Further, in addition to the EC, a local reconstruction code (LRC) technology may be used. In the LRC technology, the parity fragment includes a local parity fragment and a global parity fragment. The local parity fragment may be used to recover a fragment when there is a small quantity of damaged fragments in a data center, and the global parity fragment may be used to recover data in an EC manner when there is a relatively large quantity of damaged fragments. The data fragment and the parity fragment may be distributed in a plurality of data centers. The local parity fragment protects data in the data center, and when a data center is faulty, the global parity fragment ensures that data can be recovered.

The parity fragment can improve data reliability. However, the parity fragment occupies a large amount of storage space. When an expensive high-speed storage medium is occupied, use costs of the user are greatly increased.

SUMMARY

According to a first aspect, the present disclosure provides an implementation of a fragment management method. The method may be applied to a distributed storage system, where the distributed storage system includes a computing node and at least one storage node, the storage node includes at least one storage medium, the distributed storage system includes a plurality of storage media, different fragments are stored in different storage media, both a data fragment and a first parity fragment are located in a first-level storage medium, and the method includes reading, by the computing node, metadata of the first parity fragment to obtain a first storage location in which the first parity fragment is located, selecting, by the computing node, a second storage location, where the second storage location is located in a second-level storage medium, a read speed of the second-level storage medium is lower than that of the first-level storage medium, and the second storage location has idle space, sending, by the computing node, a migration instruction to a storage node in which the first parity fragment is located to instruct the storage node in which the first parity fragment is located to send the first parity fragment to a storage node in which the second storage location is located, storing, by the storage node in which the second storage location is located, the first parity fragment in the second storage location, and instructing, by the computing node to update information about the second storage location to the metadata of the first parity fragment.

According to the method, a parity fragment of EC or a global parity fragment of LRC may be migrated from a high-speed storage medium with relatively high costs to a low-speed storage medium with relatively low costs, thereby reducing occupation of the high-speed storage medium.

In a first optional solution of the first aspect, the method further includes reading, by the computing node, metadata of a second parity fragment to obtain a third storage location in which the second parity fragment is located, selecting, by the computing node, a fourth storage location, where the fourth storage location is located in a third-level storage medium, a read/write speed of the third-level storage medium is higher than that of the second-level storage medium and is lower than that of the first-level storage medium, and the fourth storage location has idle space, sending, by the computing node, a migration instruction to a storage node in which the second parity fragment is located to instruct the storage node in which the second parity fragment is located to send the second parity fragment to a storage node in which the fourth storage location is located, storing, by the storage node in which the fourth storage location is located, the second parity fragment in the fourth storage location, and instructing, by the computing node, to update information about the fourth storage location to the metadata of the second parity fragment.

According to the method, a local parity fragment of LRC may be migrated from a high-speed storage medium with relatively high costs to a low-speed storage medium with relatively low costs, thereby reducing occupation of the high-speed storage medium. In addition, considering that usage of the local parity fragment of the LRC is higher than that of the global parity fragment of the LRC, a read/write speed of an in-migration storage medium of the local parity fragment of the LRC is higher than a read/write speed of an in-migration storage medium of the global parity fragment of the LRC.

Optionally, in a first optional solution of the first aspect, the data fragment, the first parity fragment, and the second parity fragment meet an LRC algorithm, where the first parity fragment is a global parity fragment in the LRC algorithm, and the second parity fragment is a local parity fragment in the LRC algorithm.

In a second optional solution of the first aspect, the method includes receiving, by the computing node, a data write request, dividing target data included in the data write request into data fragments, and generating a global parity fragment and a local parity fragment based on the data fragments and according to an LRC algorithm, where the global parity fragment is used to check a plurality of data fragments, and the local parity fragment is used to check some of the plurality of data fragments.

This solution describes a process of generating the data fragment, the local parity fragment, and the global parity fragment based on the LRC algorithm and a parity relationship between these fragments.

In a third optional solution of the first aspect, the data fragment and the first parity fragment meet an EC algorithm, and the method further includes receiving, by the computing node, a data write request, dividing target data included in the data write request into data fragments, and generating the first parity fragment based on the data fragments and according to the EC algorithm.

This solution describes a process of generating the data fragment and the parity fragment (which is also referred to as a “global parity fragment” in this application) based on the EC algorithm and a parity relationship between these fragments.

According to a second aspect, the present disclosure provides an embodiment of a computing node. The computing node includes a processor unit and a memory, the memory is configured to store a computer program, and by running the computer program, the processor unit is configured to read metadata of a first parity fragment, to obtain a first storage location in which the first parity fragment is located, select a second storage location, where the second storage location is located in a second-level storage medium, a read speed of the second-level storage medium is lower than that of the first-level storage medium, and the second storage location has idle space, send a migration instruction to a storage node in which the first parity fragment is located to instruct the storage node in which the first parity fragment is located to migrate the first parity fragment to the second storage location, and update information about the second storage location to the metadata of the first parity fragment after the migration is completed.

In a first possible implementation of the second aspect, the processor is further configured to read metadata of the second parity fragment to obtain a third storage location in which the second parity fragment is located, select a fourth storage location, where the fourth storage location is located in a third-level storage medium, a read speed of the third-level storage medium is higher than that of the second-level storage medium and is lower than that of the first-level storage medium, and the fourth storage location has idle space, send a migration instruction to a storage node in which the second parity fragment is located to instruct the storage node in which the second parity fragment is located to migrate the second parity fragment to the fourth storage location, and update information about the fourth storage location to the metadata of the first parity fragment after the migration is completed.

Optionally, in the first possible implementation of the second aspect, a data fragment, the first parity fragment, and the second parity fragment meet an LRC algorithm, where the first parity fragment is a global parity fragment in the LRC algorithm, and the second parity fragment is a local parity fragment in the LRC algorithm.

In a second possible implementation of the second aspect, the processor is further configured to receive a data write request, divide target data included in the data write request into data fragments, and generate the global parity fragment and the local parity fragment based on the data fragments and according to an LRC algorithm, where the global parity fragment is used to check a plurality of data fragments, and the local parity fragment is used to check some of the plurality of data fragments.

In a third possible implementation of the second aspect, a data fragment and the first parity fragment meet an EC algorithm, and the processor is further configured to receive a data write request, divide target data included in the data write request into data fragments, and generate the first parity fragment based on the data fragments and according to the EC algorithm.

In the second aspect and each possible implementation of the second aspect, a resolved calculation problem and a beneficial effect are similar to those in the embodiments corresponding to the first aspect, and therefore details are not described again.

According to a third aspect, the present disclosure provides an embodiment of a fragment management method, where the method includes receiving, by a computing node, a data unit using an interface, and generating a data fragment based on the data unit, generating a first parity fragment based on the data fragment, selecting storage space in a first-level storage medium as a data fragment storage location, selecting storage space in a second-level storage medium as a first parity fragment storage location, where a read/write speed of the second-level storage medium is lower than a read/write speed of the first-level storage medium, and sending the data fragment and the first parity fragment to the selected storage locations, to store the data fragment and the first parity fragment, where a write request of the data fragment includes the data fragment and the data fragment storage location, and a write request of the first parity fragment includes the first parity fragment and the first parity fragment storage location.

Compared with the solutions of the first/second aspect, in this implementation, after fragments are generated, the fragments are directly sent to different levels of storage media for storage. Therefore, no “migration” operation needs to be performed, and an effect is directly equivalent to the effect of the migration operation performed in the solutions of the first/second aspect. Therefore, efficiency of storing a fragment is further improved.

In a first possible implementation of the third aspect, the method further includes generating a second parity fragment based on the data fragment, where the data fragment, the first parity fragment, and the second parity fragment meet an LRC algorithm, the first parity fragment is a global parity fragment in the LRC algorithm, and the second parity fragment is a local parity fragment in the LRC algorithm, selecting storage space in a third-level storage medium as a second parity fragment storage location, where a read/write speed of the third-level storage medium is lower than the read/write speed of the first-level storage medium and is higher than or equal to the read/write speed of the second-level storage medium, and sending a data write request to the selected storage location to store the second parity fragment, where a write request of the second parity fragment includes the second parity fragment and the second parity fragment storage location.

This solution describes a relationship between the data fragment, the global parity fragment, and the local parity fragment based on the LRC algorithm.

According to a fourth aspect, the present disclosure provides a computing node. The computing node includes a processor unit and a memory, where the memory stores a computer program, and by running the computer program, the processor unit is configured to receive a data unit using an interface, and generate a data fragment based on the data unit, generate a first parity fragment based on the data fragment, select storage space in a first-level storage medium as a data fragment storage location, and select storage space in a second-level storage medium as a first parity fragment storage location, where a read/write speed of the second-level storage medium is lower than a read/write speed of the first-level storage medium, and send the data fragment and the first parity fragment to the selected storage locations to store the data fragment and the first parity fragment, where the data fragment storage location is storage space in the first-level storage medium, and the first parity fragment storage location is storage space in the second-level storage medium.

Optionally, in a solution of the fourth aspect, the processor unit is further configured to generate a second parity fragment based on the data fragment, where the data fragment, the first parity fragment, and the second parity fragment meet an LRC algorithm, the first parity fragment is a global parity fragment in the LRC algorithm, and the second parity fragment is a local parity fragment in the LRC algorithm, select storage space in a third-level storage medium as a second parity fragment storage location, where a read/write speed of the third-level storage medium is lower than the read/write speed of the first-level storage medium and is higher than or equal to the read/write speed of the second-level storage medium, and send the data fragment and the second parity fragment to the selected storage location, to store the second parity fragment, where the second parity fragment storage location is storage space in the third-level storage medium.

In the fourth aspect and the optional solutions of the fourth aspect, a beneficial effect and a resolved technical problem are the same as those in the third aspect and the optional solutions of the third aspect. Details are not described again.

According to a fifth aspect, the present disclosure further provides an embodiment of a storage medium. The storage medium may store program code, and by running the stored code, a computer/server/distributed storage system may perform the first aspect and the possible optional manners of the first aspect, or perform the third aspect and the possible optional manners of the third aspect.

According to a sixth aspect, the present disclosure further provides an embodiment of a fragment management apparatus. The fragment management apparatus may be software or hardware, the fragment management apparatus includes modules, and each module has a function corresponding to the foregoing method embodiment.

For example, an embodiment provides a fragment management apparatus, including a reading module, a location selection module, a migration module, and a metadata management module. The reading module is configured to read metadata of the first parity fragment to obtain a first storage location in which the first parity fragment is located. The location selection module is configured to select a second storage location, where the second storage location is located in a second-level storage medium, a read speed of the second-level storage medium is lower than that of the first-level storage medium, and the second storage location has idle space. The migration module is configured to send a migration instruction to a storage node in which the first parity fragment is located to instruct the storage node in which the first parity fragment is located to migrate the first parity fragment to the second storage location.

For another example, another embodiment also provides a fragment management apparatus, and the fragment management apparatus includes a fragment module, a location selection module, and a storage module. The fragment module is configured to receive a data unit, and generate a data fragment based on the data unit, and generate a first parity fragment based on the data fragment. The location selection module is configured to select storage space in a first-level storage medium as a data fragment storage location, and select storage space in a second-level storage medium as a first parity fragment storage location, where a read speed of the second-level storage medium is lower than a read speed of the first-level storage medium. The storage module is configured to send the data fragment and the first parity fragment to the selected storage locations, to store the data fragment and the first parity fragment, where a write request of the data fragment includes the data fragment and the data fragment storage location, and a write request of the first parity fragment includes the first parity fragment and the first parity fragment storage location. The metadata management module is configured to record a storage location of a fragment in metadata of the fragment.

