An evaluation of UK credit ratings services
Describes a novel method for evaluating UK-based credit ratings services. Using a combination of timeliness, coverage, ease of use and accuracy of information, based upon a sample of 100 records added to the databases of six commercial companies’ information databases in autumn 2005, a framework for evaluation was developed and applied. The following services were evaluated: Creditsafe UK; Dun & Bradstreet; Equifax; Experian; Graydon; and ICC. Based upon the results of the evaluations, the services were graded overall with Experian coming out best. Equifax and ICC were next best, with Credit-safe after that. Dun and Bradstreet and Graydon performed the worst. However, changing the weightings of the various criteria would have produced a different outcome. Detailed explanations of the tests that were applied and the performance of each service are provided.
The evaluation of WWW search engines
The literature of the evaluation of Internet search engines is reviewed. Although there have been many studies, there has been little consistency in the way such studies have been carried out. This problem is exacerbated by the fact that recall is virtually impossible to calculate in the fast changing Internet environment, and therefore the traditional Cranfield type of evaluation is not usually possible. A variety of alternative evaluation methods has been suggested to overcome this difficulty. The authors recommend that a standardised set of tools is developed for the evaluation of web search engines so that, in future, comparisons can be made between search engines more effectively, and that variations in performance of any given search engine over time can be tracked. The paper itself does not provide such a standard set of tools, but it investigates the issues and makes preliminary recommendations of the types of tools needed.
Metadata, recall, and abstracts: can abstracts ever be reliable indicators of document value?
Abstracts from seven Internet subject trees (Euroferret, Excite, Infoseek, Lycos Top 5%, Magellan, WebCrawler, Yahoo!), five Internet subject gateways (ADAM, EEVL, NetFirst, OMNI, SOSIG), and three online databases (ERIC, ISI, LISA) were examined for their subject content, treatment of various enriching features, physical properties such as overall length, and their readability. Considerable differences were measured, and consistent similarities among abstracts from each type of source were demonstrated. Internet subject tree abstracts were generally the shortest, and online database abstracts the longest. Subject tree and online database abstracts were the most informative, but the level of coverage of document features such as tables, bibliographies, and geographical constraints was disappointingly poor. On balance, the Internet gateways appeared to be providing the most satisfactory abstracts. The authors discuss the continuing rle in networked information retrieval of abstracts and their functional analogues such as metadata.