Change Detection and Notification of Webpages: A Survey <s> Categories of Change Detection and Notification <s> Digital documents are likely to have problems associated with the persistence of links, especially when dealing with references to external resources. People keep track of various webpages of their interest using distributed digital collections and without possession of these documents; the curator cannot control how they change. In the current context, managing these distributed digital collections and getting notifications about various changes have become a significant challenge. In this paper, we address the architectural aspects of change detection systems and present optimized change detection architecture, including a web service and a browser plugin, along with an email notification service. We have performed an experimental study on our hybrid architecture for change detection in a distributed digital collection. The proposed method introduces a preliminary framework that can serve as a useful tool to mitigate the impact of unexpected change in documents stored in decentralized collections in the future. <s> BIB001 </s> Change Detection and Notification of Webpages: A Survey <s> Categories of Change Detection and Notification <s> The Internet at present has become vast and dynamic with the ever increasing number of web pages. These web pages change when more content is added to them. With the availability of change detection and notification systems, keeping track of the changes occurring in web pages has become more simple and straightforward. However, most of these change detection and notification systems work based on predefined crawling schedules with static time intervals. This can become inefficient if there are no relevant changes being made to the web pages, resulting in the wastage of both temporal and computational resources. If the web pages are not crawled frequently, some of the important changes may be missed and there may be delays in notifying the subscribed users. This paper proposes a methodology to detect the frequency of change in web pages to optimize server-side scheduling of change detection and notification systems. The proposed method is based on a dynamic detection process, where the crawling schedule will be adjusted accordingly in order to result in a more efficient server-based scheduler to detect changes in web pages. <s> BIB002
Based on the architecture involved, change detection can be segregated into two main sub domains. The first branch is "server-side change detection" and the other is "clientside change detection" BIB001 ]. Server-side change detection systems use servers which poll webpages, track changes and notify them to users. The client-side change detection systems make the client-side infrastructure poll the webpages and track changes on their own. CDN systems obtain webpage versions by crawling them and save the data to version repositories. This data is saved in an unstructured manner, mostly in the format of documents with tags, in order to allow easy storage and retrieval. Then changes are detected by comparing a previously saved version with the latest version of a particular webpage using similarity computations. Majority of the change detection mechanisms convert the data of a saved version in to a "XML-like" format where an element represents an "open-close HTML tag". Certain CDN systems allow the user to define a threshold of change, and the user gets notified about the changes, only if the change exceeds this threshold. Majority of the CDN systems operate on predefined monitoring schedules, which are specified by the user or determined by the system itself. Detected changes are visualized using many methods. A common means of visualizing text changes is by showing the newly added content in green color and the deleted text in red color (often with a strikethrough formatting) [Wachete.com 2014] . Another prominent factor discussed on CDN systems is their crawling schedules. Most of the currently available CDN systems crawl webpages under predefine schedules BIB002 ]. However, webpages can update at different time schedules (certain webpages may be frequently updated, whereas some webpages may get updated rarely), thus how often they change can vary from page to page. Hence, CDN systems require mechanisms for estimating the change frequency to create efficient checking schedules for webpages. This will reduce the number of page checks, where no changes were detected and maximize the probability of achieving optimum resource utilization.