A review of quality evaluation of digital libraries based on users ' perceptions <s> Introduction <s> While there were many efforts in the research and practices of digital libraries, evaluation was not a conspicuous activity. It is well recognized that digital library evaluation is a complex and difficult undertaking. Challenges facing digital library evaluation are enumerated. A conceptual framework for evaluation is suggested. A review of evaluation efforts in research and practice concentrates on derivation of criteria used in evaluation. Essential requirements for evaluation are stated. Discussed are constructs, context, and criteria of digital libraries: What should we evaluate? For what purpose do we evaluate? Who should evaluate? At what level do we evaluate? Upon what criteria do we evaluate? In addition, included are suggestions for adaptation of criteria from related activities. The article is considered as a part of the evolution of concepts for digital library evaluation. <s> BIB001 </s> A review of quality evaluation of digital libraries based on users ' perceptions <s> Introduction <s> A review of prior, relevant literature is an essential feature of any academic project. An effective review creates a firm foundation for advancing knowledge. It facilitates theory development, closes areas where a plethora of research exists, and uncovers areas where research is needed. <s> BIB002 </s> A review of quality evaluation of digital libraries based on users ' perceptions <s> Introduction <s> Widespread use of the Internet has resulted in digital libraries that are increasingly used by diverse communities of users for diverse purposes and in which sharing and collaboration have become important social elements. As such libraries become commonplace, as their contents and services become more varied, and as their patrons become more experienced with computer technology, users will expect more sophisticated services from these libraries. A simple search function, normally an integral part of any digital library, increasingly leads to user frustration as user needs become more complex and as the volume of managed information increases. Proactive digital libraries, where the library evolves from being passive and untailored, are seen as offering great potential for addressing and overcoming these issues and include techniques such as personalisation and recommender systems. In this paper, following on from the DELOS/NSF Working Group on Personalisation and Recommender Systems for Digital Libraries, which met and reported during 2003, we present some background material on the scope of personalisation and recommender systems in digital libraries. We then outline the working group’s vision for the evolution of digital libraries and the role that personalisation and recommender systems will play, and we present a series of research challenges and specific recommendations and research priorities for the field. <s> BIB003 </s> A review of quality evaluation of digital libraries based on users ' perceptions <s> Introduction <s> Social tagging, as a recent approach for creating metadata, has caught the attention of library and information science researchers. Many researchers recommend incorporating social tagging into the library environment and combining folksonomies with formal classification. However, some researchers are concerned with the quality issues of social annotation because of its uncontrolled nature. In this study, we compare social tags created by users from the LibraryThing website with the subject terms assigned by experts according to the Library of Congress Subject Headings (LCSH). The purpose of this study is to examine the difference and connections between social tags and expert-assigned subject terms and further explore the feasibility and obstacles of implementing social tagging in library systems. The results of our study show that it is possible to use social tags to improve the accessibility of library collections. However, the existence of non-subject-related tags may impede the application of social tagging in traditional library cataloguing systems. <s> BIB004 </s> A review of quality evaluation of digital libraries based on users ' perceptions <s> Introduction <s> In the context of information literacy studies, the research put forward in this paper is focused on the individual and subjective levels of motivation and self-efficacy, together with the sources of learning used by a population of translation students from three Spanish universities. The data were collected by means of the self-evaluating and specific IL-HUMASS survey, and are analysed using a mixed (quantitative and qualitative) case study methodology. From the 26 variables of the survey, the most and least consistent and important ones with regard to motivation and self-efficacy are detected. In general, central tendency scores regarding motivation and self-efficacy increase as respondents advance academically. The most striking opportunities for improvement emerged in relation to the following issues: using bibliographic reference managers, knowing the laws on the use of information, using database managers and communicating in other languages. The surveyed population prefer autonomous learning, with the library being a much underused source of learning. <s> BIB005
A digital library (DL) is a collection of information that has associated services delivered to user communities using a variety of technologies BIB003 . In general, DLs are the logical extension of physical libraries BIB004  BIB005  in an electronic information society. Such extensions offer new levels of access to broader audiences of users   . The use of DLs has grown significantly in the past two decades  . At the end of the 1980s, DLs were barely a part of the landscape of librarianship, information science or computer science. A decade later, by the end of the 1990s, research, practical developments and general interest in DLs had exploded globally. The accelerated growth of numerous and highly varied efforts related to DLs has continued unabated in the 2000s BIB001  . Since the 1990s, the internet and the web have become the primary platform for libraries to build and deliver information resources, services and instructions. Library users are now offered a variety of resources with different forms of interactivity, and with different levels of media richness. They can obtain research data and publications as needed, without the massive investment of capital and infrastructure to house vast physical collections. Information seeking in DLs has become an indispensable tool in 2. DL designers exploit their knowledge of the application semantic domain in order to define, customize and maintain the DL so that it is aligned with the information and functional needs of its potential DL end-users. 3. DL system administrators select the software components needed to construct the DL system. Their choice of elements reflects the expectations that DL end-users and DL designers have for the DL, as well as the requirements that the available resources impose on the definition of the DL. BIB005 . DL applications developers develop the software components that will be used as constituents of the DL systems, to ensure that the appropriate levels and types of functionality are available. According to the user classification proposed by DELOS, we restrict our survey to the quality evaluation of DLs from the content consumer's point of view. Taking it into account, the aim of this review is to answer the following research questions (RQs): • RQ1: What criteria are used to evaluate the quality of DLs in a user-centred fashion? This question motivates the following subquestions: • Do all criteria have the same importance in the evaluation? • Is there any correlation among the criteria? • RQ2: How are those criteria measured? This question motivates the following subquestions: • Are the measures quantitative or qualitative? • What instruments are used to elicit users' opinions? • How are the measurements analysed? • RQ3: What are the challenges to be faced in the future? Sections 3-5 attempt to answer questions RQ1, RQ2 and RQ3 respectively. To do so, as recommended by Webster et al. BIB002 , we have used both manual and automated methods to make a selection of candidate papers in leading journals, conferences and other related events. Table 1 classifies the primary studies  we have reviewed according to the year and type of publication. Of the 41 papers included in the review, 26 were published in journals, 8 in conferences, 4 in workshops, 2 in books, and 1 as a technical report.