Backup or preservation of websites is often not considered until after a catastrophic event has occurred. In the face of complete website loss, “lazy” webmasters or concerned third parties may be able to recover some of their website from the Internet Archive. Other pages may also be salvaged from commercial search engine caches. We introduce the concept of “lazy preservation”- digital preservation performed as a result of the normal operations of the Web infrastructure (search engines and caches). We present Warrick, a tool to automate the process of website reconstruction from the Internet Archive, Google, MSN and Yahoo. Using Warrick, we have reconstructed 24 websites of varying sizes and composition to demonstrate the feasibility and limitations of website reconstruction from the public Web infrastructure. To measure Warrick’s window of opportunity, we have profiled the time required for new Web resources to enter and leave search engine caches.
Numerous systems have been built to archive individual websites and web pages. InfoMonitor archives the server-side components (e.g., CGI scripts and datafiles) and filesystem of a web server @cite_34 . It requires an administrator to configure the system and a separate server with adequate disk space to hold the archives. Other systems like TTApache @cite_27 and iPROXY @cite_16 archive requested pages from a web server but not the server-side components. TTApache is an Apache module which archives different versions of web resources as they are requested from a web server. Users can view archived content through specially formatted URLs. iPROXY is similar to TTApache except that it uses a proxy server and archives requested resources for the client from any number of web servers. A similar approach using a proxy server with a content management system for storing and accessing Web resources was proposed in @cite_21 . Commercial systems like Furl ( http: furl.net ) and Spurl.net ( http: spurl.net ) also allow users to archive selected web resources that they deem important.
@cite_27 @cite_34 @cite_21 @cite_16
2117044215 2024882820 2005783315 2091712605
This paper presents a transaction-time HTTP server, called TTApache that supports document versioning. A document often consists of a main file formatted in HTML or XML and several included files such as images and stylesheets. A change to any of the files associated with a document creates a new version of that document. To construct a document version history, snapshots of the document's files are obtained over time. Transaction times are associated with each file version to record the version's lifetime. The transaction time is the system time of the edit that created the version. Accounting for transaction time is essential to supporting audit queries that delve into past document versions and differential queries that pinpoint differences between two versions. TTApache performs automatic versioning when a document is read thereby removing the burden of versioning from document authors. Since some versions may be created but never read, TTApache distinguishes between known and assumed versions of a document. TTApache has a simple query language to retrieve desired versions. A browser can request a specific version, or the entire history of a document. Queries can also rewrite links and references to point to current or past versions. Over time, the version history of a document continually grows. To free space, some versions can be vacuumed. Vacuuming a version however changes the semantics of requests for that version. This paper presents several policies for vacuuming versions and strategies for accounting for vacuumed versions in queries. It is important to provide long-term preservation of digital data even when those data are stored in an unreliable system such as a filesystem, a legacy database, or even the World Wide Web. In this paper we focus on the problem of archiving the contents of a Web site without disrupting users who maintain the site. We propose an archival storage system, the InfoMonitor, in which a reliable archive is integrated with an unmodified existing store. Implementing such a system presents various challenges related to the mismatch of features between the components such as differences in naming and data manipulation operations. We examine each of these issues as well as solutions for the conflicts that arise. We also discuss our experience using the InfoMonitor to archive the Stanford Database Group's Web site. The growth of the World Wide Web holds great promise for universal online information access. New information is constantly being made available for users. However, the information accessible on the Web changes constantly. These changes may occur as modifications to both the content and the location of previously existing Web resources. As these changes occur, the accessibility to past versions of such Web resources is often lost. This paper presents an approach to provide content persistence of Web resources by organizing collections of historical Web resources in a distributed configuration management system to allow online, read-only access to the versioned resources. The Web contains so much information that it is almost beyond measure. How do users manage the useful information that they have seen while screening out the rest that doesn't interest them? Bookmarks help, but bookmarking a page doesn't guarantee that it will be available forever. Search engines are becoming more powerful, but they can't be customized based on the access history of individual users. This paper suggests that a better alternative to managing web information is through a middleware approach based on iPROXY, a programmable proxy server. iPROXY offers a suite of archiving, retrieval, and searching services. It can extend a URL to include commands that archive and retrieve pages. Its modular architecture allows users to plug in new features without having to change existing browsers or servers. Once installed on a network, iPROXY can be accessed by users using different browsers and devices. Internet service providers who offer customers iPROXY will be free to develop new services without having to wait for the dominant browsers to be updated.