A review of quality evaluation of digital libraries based on users ' perceptions <s> Criteria correlation <s> The Perseus Digital Library, (PDL) is one of the primary digital resources for the humanities. Under continuous development since 1987, the project has included an ongoing evaluation component that aims to understand the effects of access to digitized source materials in the humanities. A summary of the PDL genesis and current status is given and the multifaceted and longitudinal evaluation effort is described. A brief synthesis of results is provided and reflections on the evaluation along with recommendations for DL evaluation are given. Introduction Digital libraries marry the missions, techniques, and cultures of physical libraries with the capabilities and cultures of computing and telecommunications. Evaluating digital libraries is a bit like judging how successful is a marriage. A lot depends on how successful the partners are as individuals as well as the emergent conditions made possible by the union. All three entities —the two individuals and the gestalt union--are of course influenced by their context as well. The difficulties arise from the complexity of mutually self-adapting systems interacting in a rich environment. Metrics for success for component parts of a complex system may be distinct from the metrics for success of the marriage (e.g., success for an individual partner is typically necessary but not sufficient to insure success for the marriage). Digital libraries (DLs) are extensions and augmentations of physical libraries (Marchionini & Fox, 1999). As extensions, we might evaluate the individual partners using existing techniques and metrics. Assessing the impacts of libraries on patron's lives and the larger social milieu are the ultimate goals of evaluation but the practical difficulties of assessing such complex and varied impacts causes us to measure the effectiveness and efficiencies of library operations and services as surrogates for these impacts. Metrics such as circulation, collection size and growth rate, patron visits, reference questions answered, patron satisfaction, and financial stability may be used to assess physical library performance in this regard. Clearly, these are metrics that may be points of departure for evaluating digital libraries, but they are not sufficient to characterize the new, rapidly emerging entity. Evaluation criteria for digital technologies can also be useful. For example, metrics such as response time, storage capacity, transfer rate, user satisfaction, and cost per operation may be useful in assessing technological components but may not be sufficient to characterize DL performance, let alone impact. As extensions of physical libraries and digital technologies, these metrics are good starting points but we must look further to consider the effects of DLs as augmentations that provide new services, products, and capabilities. <s> BIB001 </s> A review of quality evaluation of digital libraries based on users ' perceptions <s> Criteria correlation <s> This paper reviews how usability has been defined in the context of the digital library, what methods have been applied and their applicability, and proposes an evaluation model and a suite of instruments for evaluating usability for academic digital libraries. The model examines effectiveness, efficiency, satisfaction, and learnability. It is found that there exists an interlocking relationship among effectiveness, efficiency, and satisfaction. It also examines how learnability interacts with these three attributes. <s> BIB002 </s> A review of quality evaluation of digital libraries based on users ' perceptions <s> Criteria correlation <s> Advances in the publishing world have emerged new models of digital library development. Open access publishing modes are expanding their presence and realize the digital library idea in various means. While user-centered evaluation of digital libraries has drawn considerable attention during the last years, these systems are currently viewed from the publishing, economic and scientometric perspectives. The present study explores the concepts of usefulness and usability in the evaluation of an e-print archive. The results demonstrate that several attributes of usefulness, such as the level and the relevance of information, and usability, such as easiness of use and learnability, as well as functionalities commonly met in these systems, affect user interaction and satisfaction. <s> BIB003
In order to minimize the number of criteria that have to be taken into account to evaluate a DL, several authors have analysed the possible correlation among the criteria. According to Tsakonas et al.  , there is a correlation between usefulness and usability. In addition, Jeng BIB002 , Marchionini BIB001 and Tsakonas and Papatheodorou BIB003 have identified the following intra-usability and intra-usefulness relations, which are summarized in Table 2 . Using Analysis of Variance (ANOVA), Jeng BIB002 concludes that there exist interlocking relationships among effectiveness, efficiency and satisfaction. Marchionini BIB001 reports that there is a positive correlation between system interface and learning impact. In addition, he notes that there is a lack of correlation between demographics and learning impact. Tsakonas and Papatheodorou BIB003 report the following correlations: 1. Usability criteria: between (i) ease of use and navigation, (ii) ease of use and learnability, (iii) navigation and aesthetics, and (iv) terminology and learnability. 2. Usefulness criteria: between (i) reliability and format, (ii) reliability and level of information, and (iii) coverage and level of information.