1. Field of the Invention
The present invention relates to storage managers that provide data storage services to software applications. More particularly, the invention concerns the provision of filtering functions such as encryption, compression and other data conversions as part of storage manager operations.
2. Description of the Prior Art
By way of background, a storage manager is a system that acts as an intermediary between a software application (such as a backup/restore program or a web server) and a data storage resource (such as a tape drive, a disk drive, a storage subsystem, etc.). The storage manager, which could be integrated with the application program or implemented separately therefrom, provides an interface that accepts objects for storage and subsequently retrieves the objects upon request. Applications for which a storage manager has been used include the management of backup images of database installations, enterprise application data, individual workstations, web content, etc.
There is often a need for a storage manager to filter the data being written to or read from physical storage devices by compressing or encrypting the data. Existing storage managers that provide support for compression and/or encryption do so in one of two ways. Most commonly, such filtering is provided by algorithms that are embedded in the storage manager product itself. Less commonly, such filtering is supported by providing a programming hook that gives a storage manager user the option of writing their own algorithm(s). With this option, the user is also required to re-implement much of the functionality of the storage manager on their own.
Drawbacks of the first approach include:                The user is limited to the compression and/or encryption algorithms that are built into the storage manager product.        Some products support encryption but not compression and vice versa.        Some products support only weak encryption or poor compression.        The storage manager vendor may charge customers extra to enable the compression and/or encryption algorithms that are built in.        If a built-in algorithm is found to have a security flaw or a crippling bug, a customer cannot immediately swap in a different off-the-shelf algorithm to avoid exposure to the risk.        Storage manager customers must wait for the vendor to update the embedded algorithms with the latest technology when better algorithms are invented, even though the new technology may already exist in stand-alone off-the-shelf programs.        A vendor may not implement a particular compression or encryption algorithm that a customer desires.        
Drawbacks of the second approach include:                The storage manager programming hook places a burden on the customer to re-implement much of the functionality the storage manager otherwise provides. The user must typically write a program that can accept objects for storage, track the location of these objects, write and read them to/from physical storage devices, and retrieve them upon request based on whatever query protocol the storage manager requires, as well as write in the desired compression and/or encryption algorithms. In this solution, the storage manager essentially delegates all work to the user and does not provide any functionality of its own. The storage manager mostly acts as a hollow shell or “stub” that forwards all storage and retrieval requests to the user-written external program for handling. The storage manager itself merely assembles and disassembles buffers of information that pass between it and the application that is calling it, and provides stubs for the interface APIs (Application Programming Interfaces) but delegates most of the work to the user's program.        This approach provides very little support for compression and encryption. There is the programming hook but the customers are required to create the needed support at great additional expense and effort to themselves.        A customer who uses the programming hook but does not sufficiently test and debug their external program may find that their data has been corrupted by their own custom program, or that bugs in the program prevent the retrieval of storage objects at a critical time, such as when they need them to restore a down system.        If the event described in the preceding paragraph occurs, the storage manager vendor may find itself exposed to liability for the customer's own programming mistakes.        
Accordingly, a need exists for a storage manager filtering technique that overcomes the foregoing disadvantages. What is required is a solution that allows storage manager filters to be easily implemented without having to redesign the storage manager or duplicate its functionality in a custom program. It would be further desirable to provide the capability of implementing new and different filters. At present, the most common needs for storage manager filtering are compression and encryption. However, it is submitted that the possibilities are broader, and it may be advantageous in some circumstances to provide other data conversions, such as converting between English and metric units, or between different code pages or character sets like ASCII (American Standard Code for Information Interchange), EBCDIC (Extended Binary Coded Decimal Interchange Code), and Unicode. By way of example, this capability would be useful if backup data was generated by a first system in a first character format (e.g., a mainframe computer using EBCDIC character) and the data needed to be restored to a second system that used a second character format (e.g., a workstation using ASCII character encoding). Another area where storage manager filtering could be used is the generation of audit trails. Such a filter could be used to inspect the data being stored or retrieved and generate audit information for management purposes.