TECHNICAL FIELD

The present disclosure relates generally to storing data in remote object storages, and more specifically to replicating file systems in such remote object storages.

BACKGROUND

With the increasing move to cloud-based computing, various cloud providers have increased their offerings. In particular, as websites provide a vastly increasing amount of content including both user-generated and professionally produced content, the data storage requirements needed for cloud computing services have grown exponentially.

One particular requirement for website providing user-generated content is the ability to scale storage on demand. As a result, cloud service providers have developed several solutions for providing scalable cloud storage. One such solution is the remote object storage, examples of which include Amazon Simple Storage Service (S3), Google Cloud Platform, and Microsoft Azure. Some of these solutions further offer tiered storage, which allows for storing data based on anticipated demand in order to allow clients to reduce costs when the anticipated demand is lower. These tiers may include a high end tier intended for cases where high availability and frequent access are needed, a medium tier intended for cases where high availability is needed but frequent access may not be, and a low end tier intended for cases where high latency is acceptable such as when archiving data.

Because many enterprises increasingly rely on cloud service providers for storing their data, techniques which allow for efficiently replicating their data in the remote storages are needed. Further, since cloud service providers are third-party entities which utilize their own software, their services may not be compatible with enterprise systems by default and therefore may require software to bridge the gap.

It would therefore be advantageous to provide a solution that would overcome the challenges noted above.

SUMMARY

A summary of several example embodiments of the disclosure follows. This summary is provided for the convenience of the reader to provide a basic understanding of such embodiments and does not wholly define the breadth of the disclosure. This summary is not an extensive overview of all contemplated embodiments, and is intended to neither identify key or critical elements of all embodiments nor to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more embodiments in a simplified form as a prelude to the more detailed description that is presented later. For convenience, the term “some embodiments” or “certain embodiments” may be used herein to refer to a single embodiment or multiple embodiments of the disclosure.

Certain embodiments disclosed herein include a method for replicating a file system. The method comprises: copying at least a portion of the file system from a first storage, wherein the at least a portion of the file system includes underlying data and metadata, wherein the metadata includes pointers to portions of the underlying data and metadata defining a file system hierarchy of the file system; partitioning the copied data of the file system into a plurality of blobs, wherein the plurality of blobs includes a plurality of data blobs and a plurality of metadata blobs, wherein each data blob includes at least one portion of the underlying data, wherein the plurality of metadata blobs include a plurality of data content blobs and a plurality of name content blobs, the data content blobs having pointers to respective data blobs of the plurality of data blobs, wherein the name content blobs are created based on the metadata defining the file system hierarchy of the file system; generating a plurality of filter objects based on the copied data, wherein each filter object includes a list of metadata blobs, wherein the metadata blobs listed in each filter object include at least one of the plurality of data content blobs and at least one of the name content blobs; and storing the plurality of blobs and the plurality of filter objects in a second storage.

Certain embodiments disclosed herein also include a non-transitory computer readable medium having stored thereon causing a processing circuitry to execute a process, the process comprising: copying at least a portion of the file system from a first storage, wherein the at least a portion of the file system includes underlying data and metadata, wherein the metadata includes pointers to portions of the underlying data and metadata defining a file system hierarchy of the file system; partitioning the copied data of the file system into a plurality of blobs, wherein the plurality of blobs includes a plurality of data blobs and a plurality of metadata blobs, wherein each data blob includes at least one portion of the underlying data, wherein the plurality of metadata blobs include a plurality of data content blobs and a plurality of name content blobs, the data content blobs having pointers to respective data blobs of the plurality of data blobs, wherein the name content blobs are created based on the metadata defining the file system hierarchy of the file system; generating a plurality of filter objects based on the copied data, wherein each filter object includes a list of metadata blobs, wherein the metadata blobs listed in each filter object include at least one of the plurality of data content blobs and at least one of the name content blobs; and storing the plurality of blobs and the plurality of filter objects in a second storage.

Certain embodiments disclosed herein also include a system for replicating a file system. The system comprises: a processing circuitry; and a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to: copying at least a portion of the file system from a first storage, wherein the at least a portion of the file system includes underlying data and metadata, wherein the metadata includes pointers to portions of the underlying data and metadata defining a file system hierarchy of the file system; partitioning the copied data of the file system into a plurality of blobs, wherein the plurality of blobs includes a plurality of data blobs and a plurality of metadata blobs, wherein each data blob includes at least one portion of the underlying data, wherein the plurality of metadata blobs include a plurality of data content blobs and a plurality of name content blobs, the data content blobs having pointers to respective data blobs of the plurality of data blobs, wherein the name content blobs are created based on the metadata defining the file system hierarchy of the file system; generating a plurality of filter objects based on the copied data, wherein each filter object includes a list of metadata blobs, wherein the metadata blobs listed in each filter object include at least one of the plurality of data content blobs and at least one of the name content blobs; and storing the plurality of blobs and the plurality of filter objects in a second storage.

