youtube and flickr are examples of large - scale web portals that allow users to annotate multimedia resources by adding textual notes and comments to images or videos .figure [ fig : flickr_example ] shows an image from the flickr commons collection contributed by the library of congress .flickr users added several annotations to specific image segments , one of them telling us that this picture shows a cathedral in bergen .annotations describe resources with additional information , which is valuable to other users , who are searching and browsing resource collections .they are also important for underlying information systems , which can exploit the high - level descriptive semantics of annotations in combination with automatically extracted low - level features , such as image size and color , to implement search and retrieval over multimedia resources .taking the previous example , users who are searching for `` bergen '' or even `` bergen cathedral '' will now find this particular image in flickr , because some user provided this descriptive information in textual form .annotations are also becoming an increasingly important component in scholarly cyber - infrastructures ( cf . ) , which are often realized as web systems . therefore, a web - based annotation model should fulfill several requirements . in the age of video blogging and real - time sharing of geo - located images ,the notion of solely textual annotations has become obsolete .instead , _multimedia _ web resources should be annotatable and also be able to be annotated onto other resources .users often discuss multiple segments of a resource , or multiple resources , in a single annotation and thus the model should support multiple targets .an annotation framework should also follow the linked open data guidelines  to promote annotation sharing between systems . in order to avoid inaccurate or incorrect annotations, it must take the ephemeral nature of web resources into account .annotations on the web have many facets : a simple example could be a textual note or a tag ( cf . , ) annotating an image or video .things become more complex when a particular paragraph in an html document annotates a segment ( cf .,  ) in an online video or when someone draws polygon shapes on tiled high - resolution image sets .if we further extend the annotation concept , we could easily regard a large portion of twitter tweets as annotations on web resources . therefore , in a generic and web - centric conception , we regard an annotation as an association created between one _ body _ resource and other _ target _ resources , where the body must be somehow _ about _ the target .annotea  already defines a specification for publishing annotations on the web but has several shortcomings : ( i ) it was designed for the annotation of web pages and provides only limited means to address segments in multimedia objects , ( ii ) if clients want to access annotations they need to be aware of the annotea - specific protocol , and ( iii ) annotea annotations do not take into account that web resources are very likely to have different states over time .  throughout the yearsseveral annotea extensions have been developed to deal with these and other shortcomings : koivunnen  introduced additional types of annotations , such as _ bookmark _ and _ topic_. schroeter and hunter  proposed to express segments in media - objects by using _resources in combination with formalized or standardized descriptions to represent the context , such as svg or complex datatypes taken from the mpeg-7 standard . based on that work , haslhofer et al .  introduce the notion of _ annotation profiles _ as containers for content- and annotation - type specific annotea extensions and suggested that annotations should be dereferencable resources on the web , which follow the linked data guidelines .however , these extensions were developed separately from each other and inherit some of the above - mentioned annotea shortcomings .  in this articlewe describe the _ _ open annotation model _ _ , which is currently being developed in an international collaboration .it applies a web- and resource - centric view on annotations and defines a modular architecture , which has a simple base line model in its core .the model also provides means to address segments in multimedia resources either by encoding segment information using the media fragment uri specification or by introducing custom segment constraints for more complex annotation use cases . by allowing fixity and timestamp information on the resources involved in an annotation, it also takes into account the ephemeral nature of web resources .pulling together the functionalities provided by various , partly independent annotea extensions is a major goal of this effort .this article extends our work previously published in  by the following : it explains the design rationale that lead to the specification of the current annotation model and also the technical aspects of the model in more detail .it also contains an updated and extended related work section .in this section we outline the design rationale that drives the specification of the open annotation model . we give examples illustrating common requirements we found in several real - world annotation use cases ( e.g. , ) and describe the reasons behind our design decisions . from each design decision we derived a set of guiding design principles , which are reflected in the current open annotation model design .textual notes or tags on images , as in the flickr example in figure [ fig : flickr_example ] , occur frequently on the web and are the simplest examples for web annotations . in open annotation model terms , these are _ annotations _ that have a textual _ body _ and an image resource as _target_. however , in many scenarios the prevailing view that annotation bodies are textual is insufficient .figure [ fig : hubble_example ] shows an annotation in which the body is not a textual note , but a video , which itself is an addressable web resource identified by a uri . in order to cover such cases , we must abstract from purely textual annotation bodies and model them as resources that can be of any media type .  from a conceptual point of view ,an annotation is an association between two resources , the _ body _ and the _ target_. however , in most cases this association needs to be qualified and addressable in some way . indicating the creator and creation date of a resource , or replying to existing annotations , are frequently occurring scenarios that require an annotation to be expressed as a first - class entity .web annotations are then instances of an annotation and relate together the body and target resources that are involved in the annotation association . with this approachwe follow a common design pattern , which is also known as qualified relation in linked data  or association class in uml .in contrast to existing tagging models ( see  ) the annotation _creator _ is not a mandatory core model entity .we believe that adopters of the open annotation model already have user models in place or rely on open web identities ( e.g. , a google or facebook account ) and are most likely not willing to map their user models against another user model , which is specified as part of an annotation model . however , we encourage adopters to relate annotations with existing web resources , which identify users .  from these considerationswe derive the following guiding principles :  * all core entities ( annotation , body , target ) must be resources .* annotations must allow for both body and target of any media type . *annotations , bodies , and targets can have different authorship .our introductory flickr example illustrates how annotations can target specific segments in web resources . in figure[ fig : hubble_example ] we gave an example that involves multimedia resources : a video as body , and an image as target .there is strong dependency between these requirements because a resource s media type affects the way segments need to be addressed .annotating an area in an image requires a different segment representation than those that target segments in the spatial and temporal dimensions of a video .similarly , addressing text segments in a pdf document differs from addressing text in plain text or html documents .since we abstract from purely textual annotation bodies , we must take into account that both the body and the target of an annotation can be resource segments .we could , for instance , refine our previous example by saying that some sequence within the video annotates a certain image segment .the problem of addressing media segments , also known as fragment identification , is well known and will be explained in more detail in section [ subsec : rel_work_media_segments ] .since the open annotation model will be implemented in web environments and all resources involved in an annotation are web resources identified by uris , it should reuse the fragment identification mechanisms that are already defined as part of the architecture of the world wide web  and extensions thereof , such as fragment construction rules for specific media types .however , many annotation use cases require more complex segment representations such as polygon regions in images , which can not be expressed with available standards . therefore , our guiding principles with respect to addressing segments in multimedia resources are :  * annotations must support resource segment addressing both on body and target resources . *preferably this should be done with ( media ) fragment uris , but extensibility must be provided for cases in which the use of uris for segment addressing is not possible .previously we argued that all core entities of the open annotation model must be first class web resources , which are identified by uris , if possible http uris .the great benefit of this approach is that existing technologies and solutions that can be applied for web resources ( e.g. , mime - types , fragments , access control , etc . ) also work for resources involved in a web annotation without the necessity to include these aspects in an annotation specification .however , there is one big problem we inherit from the web architecture and which is severe in the context of annotations : uri - addressable web resources are ephemeral , which means that the representations obtained by dereferencing their uris may change over time .the annotation example in figure [ fig : cnn_example ] shows a twitter tweet annotating the cnn web site and illustrates the ephemerality problem : the tweet refers to the cnn page at a certain point in time and might be misinterpreted when the cnn main web site changes .measures should be provided that can help in avoiding misinterpretations of annotations , including the expression of timestamps and fixity information for body and target resources .the focus of the open annotation model is on sharing annotations on scholarly resources and therefore the model should have sufficient richness of expression to satisfy scholars needs .however , in order to maximize the likelihood of adoption , the model should also be an interoperability framework readily applicable in other domains .one possibility to achieve this is to follow a modular and extensible modeling approach , with a generally applicable baseline model and domain- or media - type specific extensions .an annotation client should implement at least the baseline model and , if possible , provide fallback behavior for annotations that contain domain - specific extensions the client might not be aware of .  in order to increase the likelihood of adoption , and in alignment with the goal of sharing annotations , no client - server protocol for publishing , updating , or deleting annotationswill be specified .rather , the specifications will take a perspective whereby clients publish annotations to the web and make them discoverable using common web approaches .such an approach does not require a preferred annotation server for a client , yet it does not preclude one either .  our guiding principles to achieve annotation interoperability and widespread adoption are :  * the open annotation model should have a simple but expressive baseline model defining top - level classes / entities and properties / relationships .* the baseline model should be extensible .* annotation protocols are out of scope .the open annotation data model draws from various extensions of annotea to form a cohesive whole . the web architecture and linked data guidelinesare foundational principles , resulting in a specification that can be applied to annotate any set of web resources . at the time of this writing , the specification , which is available at http://www.openannotation.org/spec/beta/ , is still under development . in the following ,we describe the major technical building blocks of the open annotation data model and use some simple examples to illustrate how to apply the model in practice . for further examples , which also cover different use cases , such as the annotation of medieval manuscripts , we refer to the online documentation .  following its predecessors , the open annotation model , shown in figure [ fig : oac_data_model ] ,has three primary classes of resources . in all cases below, the ` oac ` namespace prefix expands to http://www.openannotation.org / ns/.    * the ` oac : body ` of the annotation ( node b-1 ) : this resource is the comment , metadata or other information that is created about another resource .the body can be any web resource , of any media format , available at any uri .the model allows for either one body per annotation , or an annotation without any body , but not annotations with multiple bodies . * the ` oac : target ` of the annotation ( node t-1 ) : this is the resource that the body is about . like the body, it can be any uri identified resource .the model allows for one or more targets per annotation . *the ` oac : annotation ` ( node a-1 ) : this resource is an rdf document , identified by an http uri , that describes at least the body and target resources involved in the annotation as well as any additional properties and relationships ( e.g. , ` dcterms : creator ` ) .dereferencing an annotation s http uri returns a serialization in a permissible rdf format .an annotation is a web resource , which is identified by an http uri and returns an rdf document when being dereferenced . as with any rdf data , additional properties and relationshipscan be associated with any of the resources .it is recommended that an annotation has a timestamp of when the annotation relationship was created ( ` dcterms : created ` ) and a reference to the agent that created it ( ` dcterms : creator ` ) .resources referenced by additional relationships may themselves have additional properties and relationships .figure [ fig : oac_data_model_add_prop ] gives an example of recommended and other possible ( e.g. , ` dc : title ` ) properties and relationships that can be added to the open annotation baseline model .the set of properties and relationship in this example is by no means exhaustive .properties and relationships from other vocabularies may also be used .it is also important to note that the creator and created timestamp of each of the three resource types above may be different .an annotation might refer to an annotation body created by a third party , perhaps from before the open annotation specification was published , and a target created by yet another party .similarly , there may be additional subclasses of ` oac : annotation ` that further specify restrictions on the meaning of the annotation , such as a _ reply _ ; an annotation where the single target is itself an annotation .this allows chaining of annotations into a threaded discussion model .if the body of an annotation is identified by a dereferencable http uri , as it is the case in twitter , various blogging platforms , or google docs , it can easily be referenced from an annotation . if a client can not create uris for an annotation body , for instance because it is an offline client , it can assign a unique non - resolvable uri ( called a urn ) as the identifier for the body node .this approach can still be reconciled with the linked data principles as servers that publish such annotations can assign http uris they control to the bodies , and express equivalence between the http uri and the urn .the open annotation model allows the inclusion of information directly in the annotation document by adding the representation of a resource inline within the rdf document using the _ content in rdf _specification  from the w3c .the example annotation in figure [ fig : oac_data_model_inline_body_example ] shows how to express this : the representation is the object of the ` cnt : chars ` predicate , and its character encoding the object of the ` cnt : characterencoding ` predicate .further classes from this specification include base64 encoded resources and xml encoded resources .most of the use cases , which have been explored before specifying the model , involved comments that were about a segment of a resource , rather than the entire resource identified by a uri .the data model allows two different methods of identifying and describing the region of interest of a resource ; either using a _ fragment uri _ , or a more expressive _ constraint _ resource .it is clearly recommended to use fragment uris whenever possible , because this method relies on normative specifications , which brings interoperability with other applications .only if there are no appropriate uri fragment specifications available , the creators should define their own constraints .a fragment uri normally identifies a part of a resource , and the method for constructing and interpreting these uris is dependent on the media type of the resource . in general ,fragment uris are created by appending a fragment that describes the section of interest to the uri of the full resource , separated by a # character ( see ) .there are two main sources for existing fragment uri specifications , which can both identify and describe how to discover a segment of interest within a resource .the first is the set of mime type specification rfcs from the ietf .this includes x / html ( rfc 2854/3236 ) , xml ( rfc 3023 ) , pdf ( rfc 3778 ) and plain text ( rfc 5147 ) .the second is w3c media uris specification  , which is defined at a broader level to cover images , video and audio resources , regardless of the exact format .the following examples show how to apply these specifications to define media - type specific uris that describe a certain resource segment :  * http://www.example.net/foo.html#namedsection identifies the section named as `` namedsection '' in an html document . *http://www.example.net/foo.pdf#page=10&viewrect=20,100,50,60 identifies a rectangle starting at 20 pixels in from the left , and 100 down from the top , with a width of 50 and a height of 60 in a pdf document . *http://www.example.net/foo.png#xywh=160,120,320,240 identifies a 320 by 240 box , starting at x=160 and y=120 in an image . *[ http://www.example.net/foo.mpg#t=npt:10,20 ] identifies a sequence starting just before the 10th second , and ending just before the 20th in a video .we recommend that when a definition exists for how to construct a fragment uri for a particular document format , and such a fragment would accurately describe the section of interest for an annotation , then this technique should be used .it is recommended to also use ` dcterms : ispartof ` with the full resource as the object , in order to make the annotation more easily discoverable .figure [ fig : media_segment_uri ] shows an example in which a tweet annotates a rectangular section in an image , which in turn is identified and described by a media fragment uri .there are many situations when segments can not be described with fragment uris , but it is still desirable to be able to annotate a segment of a resource .for example , a non - rectangular section of an image , or a segment of a resource with a format or media type that is not covered by either fragment specification , such as a 3-dimensional model or a dataset . to handle these situations, we introduce an ` oac : constraint ` resource that describes the segment of interest using an appropriate standard , and a ` oac : constrainedtarget ` resource that identifies the segment of interest .this _ constrained target _ is the object of the ` oac : hastarget ` predicate of the ` oac : annotation ` , and subsequently ` oac : constrains ` the full target resource .figure [ fig : oac_model_constraint_target ] shows how constrained targets extend the open annotation baseline model .the nature of the constraint description will be dependent on the type of the resource for which the segment is being conveyed .it is then up to the annotation client to interpret the segment description with respect to the full resource .figure [ fig : oac_model_svg_constraint_example ] shows an example in which an area within an image is described by an svg path element .the document containing the svg specification is identified by a dereferencable http uri and a specialized type ` oac : svgconstraint ` in combination with a ` dc : format ` property informs the client about the type of constraint it needs to deal with .    alternatively , it is also possible to include the constraint information inline within the annotation document using the same technique as used for including the body .the ` oac : constraint ` is given a urn ( normally a urn : uuid ) and then the constraint information is included as the value of the ` cnt : chars ` property .the requirements for doing this are the same as for including the ` oac : body ` inline within the annotation document . for more complex use cases it also possible to express constraints in rdf and to apply constraints also on the body of an annotation .one goal of the ongoing open annotation demonstrator activities is to collect real - world constraint definitions from various use cases and to specify them in the context of the oa model .we hope that this also serves as as feedback loop for possible enhancements or additional uri fragment specifications .it must be stressed that different agents may create the _ annotation _ , _ body _ and _ target _ at different times .for example , alice might create an annotation saying that bob s youtube video annotates carol s flickr photo .also , being regular web resources , the body and target are likely to have different representations over time. some annotations may apply irrespective of representation , while others may pertain to specific representations . in order to provide the ability to accurately interpret annotations past their publication ,the open annotation model introduces three ways to express temporal context . the manner in which these three types of annotations use the ` oac : when ` property , which has a datetime as its value , distinguishes them .a _ timeless annotation _ applies irrespective of the evolving representations of body and target ; it can be considered as if the annotation references the semantics of the resources .for example , an annotation with a body that says `` this is the front page of cnn '' remains accurate as representations of the target http://cnn.com/ change over time .timeless annotations do nt make use of the ` oac : when ` property .a _ uniform time annotation _ has a single point in time at which all the resources involved in the annotation should be considered .this type of annotation has the ` oac : when ` property attached to the ` oac : annotation ` .for example , if alice recurrently publishes a tweet that comments on a story on the live cnn home page , an annotation that has the cartoon as body and the cnn home page as target would need to be handled as a uniform time annotation in order to provide the ability to match up correct representations of body and target .figure [ fig : uniform_time_example ] shows how uniform time annotations can be represented using the open annotation model .a _ varied time annotation _ has a body and target that need to be considered at different moments in time .this type of annotation uses the ` oac : when ` property attached to an ` oac : webtimeconstraint ` node , which is a specialization of ` oac : constraint ` , for both body and target .if , in the aforementioned example , alice would have the habit to publish a cartoon at http://example.org/cartoon when the mocked article is no longer on the home page , but still use http://cnn.com as the target of her annotation , the varied time annotation approach would have to be used .this temporal information can be used to recreate the annotation as it was intended by reconstructing it with the time - appropriate body and target(s ) .previous versions of web resources exist in archives such as the internet archive , or within content management systems such as mediawiki s article history , however they are divorced from their original uri .memento  , which is a framework that proposes a simple extension of http in order to connect the original and archived resources , can be applied for recreating annotations .it leverages existing http capabilities in order to support accessing resource versions through the use of the uri of a resource and a datetime as the indicator of the required version . in the framework ,a server that host versions of a given resource exposes a timegate , which acts as a gateway to the past for a given web resource . in order to facilitate access to a version of that resource , the timegate supports http content negotiation in the datetime dimension .several mechanisms support discovery of timegates , including http links that point from a resource to its timegate(s )  .in this section we give an overview of existing work in the area of web annotations . after summarizing general works about annotations and annotation interoperability, we analyze the features of existing web annotation models and compare them with those of the open annotation model .annotations have a long research history , and unsurprisingly the research perspectives and interpretations of what an _ annotation _ is supposed to be vary widely .agosti et al . provide a comprehensive study on the contours and complexity of annotations . a representative discussion on how annotations can be used in various scholarly disciplines is given by bradley  .he describes how annotations can support interpretation development by collecting notes , classifying resources , and identifying novel relationships between resources .the different forms and functions that annotations can take are analyzed by marshall  .she distinguishes between _ formal _ and _ informal _ annotations , whereby formal annotations follow structural standards and informal ones are unstructured . furthermore , marshall divides into _ implicit _ annotations that are intended for sharing and _ explicit _ annotations of personal nature , often interpretable only by the original creator .further divisions defined by marshall with regard to the function of an annotation include _ permanent _ vs. _ transient _ , _ annotation as writing _ vs._ annotation as reading _ ,_ extensive _ vs. _ intensive _ , _ published _ vs. _ private _ and_ institutional _ vs. _ workgroup _ vs. _ individual_. the difference between personal and public annotations in a digital environment is further investigated in a study by marshall and brush  .they derive design implications for annotation systems , e.g. regarding find and filtering requirements , and user interface strategies for processing and sharing annotations .a taxonomy of annotation types and marking symbols used by readers of scholarly documents is presented by qayyum  .his taxonomy is derived from the results of a user study conducted with students reading research articles in a private as well as a collaborative digital setting .a related recent effort is presented by blustein et al . . in their field study , conducted over the course of three years, they identify six purposes for scholarly annotation : _ interpretation _ , _ problem - working _ , _ tracing progress _ , _ procedural annotations _, _ place marking and aiding memory _ and _ incidental markings_.   the idea of publishing user annotations on the web is not new .annotea  was specified more than a decade ago and defines a data model and protocol for uploading , downloading , and modifying annotations . since the web has changed over time and nowalso comprises non - document web resources , the annotea model soon became insufficient for many annotation use cases , as we explained at the beginning of this article .annotea extensions , such as co - annotea  or lemo  , were developed to deal with the annotea shortcomings and to take into account emerging architectural styles , such as restful web services , or linked open data .recent web annotation model specification efforts include the m3o ontology  , the annotation ontology  , and the open annotation model , which we presented in this article .    in figure[ fig : existing_ann_models_ft_analysis ] we present the results of a feature comparison we performed across the previously mentioned models .the models are timely ordered by their publication year , and the feature selection is based on the requirements and use case descriptions we found in the model documentations . although we can not generalize from this representative set of annotation models and features , we can observe hat annotation models have continually been adapted to emerging standards , needs , and architectures : linked open data is increasingly adopted for publishing and sharing annotations , multimedia resources can now be annotated also by other multimedia resources , extensibility has become a key requirement , security and provenance are being outsourced to other models , and standard segment identification mechanisms are being combined with custom solutions ( context , fragment , selectors ) to capture complex domain - specific needs .the annotation ontology , which is an open ontology for the annotation of scientific documents on the web , is technically very similar to the open annotation model .however , they differ e.g. , in terms of how fragments are being expressed : by representing constraints and constraint targets as first - class resources the open annotation model supports direct addressing of fragments , thus enabling use cases where different users annotate the same fragment , or search scenarios where annotations are retrieved by fragment . furthermore , the open annotation model supports structured annotation bodies and allows to overlay semantic statements pertaining to one or more annotation targets , which offers potentially more flexibility , e.g. , for use cases of entity and entity - relation extraction in scientific literature .  a related strand of research concerns models for _ social tagging _ of web resources .hunter  describes tags as `` _ _ a subclass of annotations that comprise simple , unstructured labels or keywords assigned to digital resources to describe and classify the digital resource _ _ '' .a comparison of tagging ontologies is presented by kim et al .they survey the state of the art in tagging models and identify three building blocks common to existing tagging models : _ taggers _ , the _ tags _ themselves and the _ resources _ being tagged .semantic annotations features can also be found in multimedia metadata frameworks such as mpeg-7 and multimedia metadata ontologies such as comm  or the recent _ _ w3c ontology for media resources _ _ specification , which provides a core metadata vocabulary for media resources on the web .it defines two metadata properties that can be used for the textual description of a media resource ( fragment ) or for relating rdf files or named graphs to a media resource .other ontologies were designed to embed annotations directly into the multimedia content representation .the m3o ontology  , for instance , allows the integration of annotations with smil and svg documents . on the contrary, the open annotation model is more in line with the previously discussed web annotation models .it treats annotations as first class web resources , which can exist independently from the content or metadata representations of media objects .this design choice is motivated by a set of scholarly use cases , which require that multimedia content objects can be annotated any time after the content production and metadata extraction process .early related work on the issue of describing segments in multimedia resources can be traced back to research on linking in hypermedia documents ( cf . ) . for describing segments using a non - uri based mechanism one can use mpeg-7 shape descriptors ( cf . ) or terms defined in a dedicated multimedia ontology .svg  and mpeg-21  introduced xpointer - based uri fragment definitions for linking to segments in multimedia resources .the temporal uri specification  addresses a temporal segment in a time - based media resource through a defined url query parameter ( t= ) .youtube supports similar direct linking to a particular point in time in a video using a fragment uri .the media fragments uri specification  is a w3c working draft that introduces a standard , uri - based approach for addressing temporal , spatial and track sub - parts of any non - textual media content , thus making audiovisual media segments first class citizens on the web  .   the ephemeral nature of web resources and methods to deal with that problem have been studied from the early years of the web on .phelps and wilensky  proposed to decorate hyperlinks with lexical signatures to re - find disappeared web resources .recent works include klein et al . , who proposed to compute lexical signatures from the link neighborhood of a web page , and morishima et al . who describe a method to fix broken links when link targets have moved .the problem has also been realized in the linked data context and solutions like dsnotify  were proposed to re - find resources by their representations .we apply a generic and web - centric conception to the various facets annotations can have and regard an annotation as association created between one _ body _ resource and other _ target _ resources , where the body must be somehow _ about _ the target .this conception lead to the specification of the open annotation model , which originates from activities in the open annotation collaboration and aims at building an interoperable environment for publishing annotations on the web .  at the time of this writing ,the open annotation model is still in beta stage and is currently implemented in several demonstration projects covering use cases ranging from annotating medieval manuscripts , over annotating online maps , to annotating online video segments . as part of these demonstration projects client apis are being developed for various programming environments to ease model adoption for developers .we expect to obtain user and developer feedback from these projects , which should further refine the open annotation model .pursuing better integration of the proposed segment identification approach with the w3c media fragment uri specification is on our research agenda .however , this requires an extension mechanism in that specification , which is not within the scope of the responsible working group at the moment .also the question of how to model multiple annotation targets and how to interpret this information correctly , is currently being discussed .finally , as a first outcome of the demonstration projects , we observed that a common set of use cases is annotating data with other data , rather than with information intended for human consumption .this raises the question of how to model _ data annotations _ in an interoperable way .  as a final result, we expect a data model , which provides an interoperable method of expressing annotations such that they can easily be shared between platforms , with sufficient richness of expression to satisfy also complex annotation scenarios .the work has partly been supported by the european commission as part of the econtentplus program ( europeanaconnect ) and by a marie curie international outgoing fellowship within the 7th europeana community framework program .the development of oac is funded by the andrew w. mellon foundation .richard arndt , raphal troncy , steffen staab , lynda hardman , and miroslav vacura .comm : designing a well - founded multimedia ontology for the web . in _ the semantic web , 6th international semantic web conference , 2nd asian semantic web conference , iswc 2007 + aswc 2007 _ , pages 3043 , busan , korea , november 2007 .james blustein , david rowe , and ann - barbara graff . making sense in the margins : a field study of annotation .in _ proceedings of the 15th international conference on theory and practice of digital libraries : research and advanced technology for digital libraries _ , tpdl11 , pages 252259 , berlin , heidelberg , 2011 .springer - verlag .bernhard haslhofer , rainer simon , robert sanderson , and herbert van de sompel . the open annotation collaboration ( oac ) model . in _ workshop on multimedia on the web 2011 _ , graz , austria , september 2011 .michael hausenblas , raphal troncy , yves raimond , and tobia brger . interlinking multimedia: how to apply linked data principles to multimedia fragments . in_ www 2009 workshop : linked data on the web ( ldow2009 ) _ , madrid , spain , 2009 .  tom heath and christian bizer . .synthesis lectures on the semantic web : theory and technology , 1:1 , 1 - 136 .morgan & claypool , 1st edition edition , 2011 .available at : http://linkeddatabook.com / editions/1.0/.     jos kahan and marja - ritta koivunen .annotea : an open rdf infrastructure for shared web annotations . in _ proceedings of the 10th international conference on world wide web _ , www 01 , pages 623632 , new york , ny , usa , 2001 .hak l. kim , simon scerri , john g. breslin , stefan decker , and hong g. kim . .dcmi 08 : proceedings of the 2008 international conference on dublin core and metadata applications _ , pages 128137 .dublin core metadata initiative , 2008 .martin klein , jeb ware , and michael l. nelson .rediscovering missing web pages using link neighborhood lexical signatures . in _ proceeding of the 11th annual international acm / ieee joint conference on digital libraries _ , jcdl 11 , pages 137140 , new york , ny , usa , 2011 .marja - riitta koivunen. semantic authoring by tagging with annotea social bookmarks and topics . in _ in the 5th international semantic web conference ( iswc2006 ) - 1st semantic authoring and annotation workshop ( saaw2006 ) _ , 2006 .catherine c. marshall .the future of annotation in a digital ( paper ) world . in harum and twidale ,editors , _ successes and failures of digital libraries _ , pages 97117 .urbana - champaign : university of illinois , 2000 .available at : http://www.csdl.tamu.edu/~marshall/uiucpaper-complete.pdf .catherine c. marshall and a. j. bernheim brush . exploring the relationship between personal and public annotations . in _ proceedings of the 2004 joint acm / ieee conference on digital libraries _ , pages 349357 , 2004 .atsuyuki morishima , akiyoshi nakamizo , toshinari iida , shigeo sugimoto , and hiroyuki kitagawa .bringing your dead links back to life : a comprehensive approach and lessons learned . in _ proceedings of the 20th acm conference on hypertext and hypermedia _ , ht 09 , pages 1524 , new york , ny , usa , 2009 .acm .carsten saathoff and ansgar scherp . unlocking the semantics of multimedia presentations in the web with the multimedia metadata ontology . in _ proceedings of the 19th international conference on world wide web _ , www 10 , pages 831840 , new york , ny , usa , 2010 .robert sanderson , benjamin albritton , rafael schwemmer , and herbert van de sompel .sharedcanvas : a collaborative model for medieval manuscript layout dissemination . in _ proceeding of the 11th annual international acm / ieee joint conference on digital libraries _ , jcdl 11 , pages 175184 , new york , ny , usa , 2011 .robert sanderson and herbert van de sompel .making web annotations persistent over time . in _ proceedings of the 10th annual joint conference on digital libraries _ , jcdl 10 , pages 110 , new york , ny , usa , 2010 .ronald schroeter , jane hunter , and andrew newman .annotating relationships between multiple mixed - media digital objects by extending annotea . in enrico franconi ,michael kifer , and wolfgang may , editors , _ eswc _ , volume 4519 .springer , 2007 .rainer simon , bernhard haslhofer , werner robitza , and elaheh momeni .semantically augmented annotations in digitized map collections . in _ proceeding of the 11th annual international acm / ieee joint conference on digital libraries _ , jcdl 11 , pages 199202 , new york , ny , usa , 2011 .davy van deursen , raphal troncy , erik mannens , silvia pfeiffer , yves lafon , and rik van de walle .implementing the media fragments uri specification . in _ proceedings of the 19th international conference on world wide web _ , www 10 , pages 13611364 , new york , ny , usa , 2010 .karin verspoor , judith cohn , cliff joslyn , sue mniszewski , andreas rechtsteiner , luis rocha , and tiago simas .protein annotation as term categorization in the gene ontology using word proximity networks ., 6(1 ) , 2005 .

Abstract: many web portals allow users to associate additional information with existing multimedia resources such as images , audio , and video . however , these portals are usually closed systems and user - generated annotations are almost always kept locked up and remain inaccessible to the web of data . we believe that an important step to take is the integration of multimedia annotations and the linked data principles . we present the current state of the open annotation model , explain our design rationale , and describe how the model can represent user annotations on multimedia web resources . applying this model in web portals and devices , which support user annotations , should allow clients to easily publish and consume , thus exchange annotations on multimedia web resources via common web standards .   example.eps gsave newpath 20 20 moveto 20 220 lineto 220 220 lineto 220 20 lineto closepath 2 setlinewidth gsave .4 setgray fill grestore stroke grestore


to improve the precision of retrieval output , especially within the very few ( e.g , 5 or 10 ) highest - ranked documents that are returned , a number of researchers have considered a _structural ing _ strategy .the idea is to re - rankthe top documents that some initial search engine produces , where the re - ordering utilizes information about inter - document relationships within that set .promising results have been previously obtained by using document _ centrality _ within the initially retrieved list to perform structural ing , on the premise that if the quality of this list is reasonable to begin with , then the documents that are most related to most of the documents on the list are likely to be the most relevant ones .in particular , in our prior work we adapted _ pagerank _ which , due to the success of google , is surely the most well - established algorithm for defining and computing centralitywithin a directed graph to the task of re - rankingnon - hyperlinked document sets .the arguably most well - known alternative to pagerank is kleinberg s _ hits _ algorithm . the major conceptual wayin which hitsdiffers from pagerank is that it defines two different types of centralitems : each node is assigned both a _ hub _ and an _ authority _ score as opposed to a single pagerank score . in the web setting , in which hitswas originally proposed , good hubs correspond roughly to high - quality resource lists or collections of pointers , whereas good authorities correspond to the high - quality resources themselves ; thus , distinguishing between two differing but interdependent types of webpages is quite appropriate .our previous study applied hits to non - web documents .we found that its performance was comparable to or better than that of algorithms that do not involve structural ing ; however , hits was not as effective as pagerank .  do these results imply that pagerank is better than hitsfor structural ingof non - web documents ?not necessarily , because there may exist graph - construction methods that are more suitable for hits . note that the only entities considered in our previous study were documents .if we could introduce entities distinct from documents but enjoying a mutually reinforcing relationship with them , then we might better satisfy the spirit of the hubs - versus - authorities distinction , and thus derive stronger results utilizing hits .a crucial insight of the present paper is that document _clusters _ appear extremely well - suited to play this complementary role .the intuition is that : ( a ) given those clusters that are `` most representative '' of the user s information need ,the documents within those clusters are likely to be relevant ; and ( b ) the `` most representative '' clusters should be those that contain many relevant documents . this apparently circular reasoning is strongly reminiscent of the inter - related hubs and authorities concepts underlying hits .also , clusters have long been considered a promising source of information .the well - known _ cluster hypothesis _ encapsulates the intuition that clusters can reveal groups of relevant documents ; in practice , the potential utility of clustering for this purpose has been demonstrated for both the case wherein clusters were created in a query - independent fashion and the re - rankingsetting .  in this paper ,we show through an array of experiments that consideration of the mutual reinforcement of clusters and documents in determining centralitycan lead to highly effective algorithms for re - rankingan initially retrieved list .specifically , our experimental results show that the centrality - induction methods that we previously studied solely in the context of document - only graphs result in much better re - rankingperformance if implemented over bipartite graphs of documents ( on one side ) and clusters ( on the other side ) .for example , ranking _ documents _ by their `` authoritativeness '' as computed by hits upon these cluster - document graphs yields better performance than that of a previously proposed pagerank implementation applied to document - only graphs .interestingly , we also find that _ cluster _authority scores can be used to identify clusters containing a large percentage of relevant documents .since we are focused on the structural ingparadigm , our algorithms are applied not to the entire corpus , but to a subset ( henceforth ) , defined as the top documents retrieved in response to the query by a given initial retrieval engine .some of our algorithms also take into account a set of _ clusters _ of the documents in .we use to refer generically to whichever set of entities either or  is used by a given algorithm .  the basic idea behind the algorithms we consider is to determine centralitywithin a _ relevance - flow graph _ ,defined as a directed graph with non - negative weights on the edges in which  * the nodes are the elements of , and * the weight on an edge between node and is based on the strength of evidence for s relevance that would follow from an assertion that is relevant .  by construction , then , any measure of the centralityof should measure the accumulation of evidence for its relevance according to the set of interconnections among the entities in .such information can then optionally be subjected to additional processing , such as integration with information on each item s similarity to the query , to produce a final re - rankingof .[ [ conventions - regarding - graphs ] ] conventions regarding graphs + + + + + + + + + + + + + + + + + + + + + + + + + + + +  the types of relevance - flow graphswe consider can all be represented as weighted directed graphs of the form , where is a finite non - empty set of nodes and is a non - negative edge - weight function . note that thus our graphs technically have edges between all ordered pairs of nodes ( self - loops included ) ; however , edges with zero edge - weight are conceptually equivalent to missing edges . for clarity ,we write instead of .the hitsalgorithm for computing centralitycan be motivated as follows .let be the input graph , and let be a node in .first , suppose we somehow knew the _ hub score _ of each node , where `` hubness '' is the extent to which the nodes that points to are `` good '' in some sense .then , s _ authority score _ would be a natural measure of how `` good '' is , since a node that is `` strongly '' pointed to by high - quality hubs ( which , by definition , tend to point to `` good '' nodes ) receives a high score .but where do we get the hub score for a given node ?a natural choice is to use the extent to which `` strongly '' points to highly authoritative nodes : clearly , equations [ eq : auth ] and [ eq : hub ] are mutually recursive .however , the iterative hits algorithm provably converges to ( non - identically - zero , non - negative ) score functions and that satisfy the above pair of equations .figure [ fig : hits - icon ] depicts the `` iconic '' case in which the input graph is _ one - way bipartite _ ,that is , can be partitioned into non - empty sets and such that only edges in can receive positive weight , and , .it is the case that for every and for every ; in this sense , the left - hand nodes are `` pure '' hubs and the right - hand nodes are `` pure '' authorities .  note that in the end , we need to produce a _ single _centralityscore for each node . for experimental simplicity, we consider only two possibilities in this paper using as the final centralityscore , or using instead although combining the hub and authority scores is also an interesting possibility .recall that the fundamental operation in our structural ingparadigm is to compute the centralityof entities ( with)in a set .one possibility is to define as , the documents in the initially retrieved set ; we refer generically to any relevance - flow graphinduced under this choice as a * _ document - to - document _ * graph . butnote that for non - web documents , it may not be obvious _ a priori _ what kinds of documents are hubs and what kinds are authorities .  alternatively , we can define as , where consists of clusters of the documents in . on a purely formal level, doing so allows us to map the hubs / authorities duality discussed above onto the documents / clusters duality , as follows .recalling our discussion of the `` iconic '' case of one - way bipartitegraphs , we can create * _ document - as - authority _ * graphs simply by choosing and , so that necessarily clusters serve the role of ( pure ) hubs and documents serve the role of ( pure ) authorities .contrariwise , we can create * _ document - as - hub _ * graphs by setting and .but the advantages of incorporating cluster - based information are not just formal .the well - known _ cluster hypothesis _ encapsulates the intuition that clusters can reveal groups of relevant documents ; in practice , the potential utility of clustering for this purpose has been demonstrated a number of times , whether the clusters were created in a query - independent fashion , or from the initially most - highly - ranked documents for some query ( i.e. , in the re - rankingsetting ) . since centralclusters are , supposedly , those that accrue the most evidence for relevance , documents that are strongly identified with such clusters should themselves be judged highly relevant . but identifying such clusters is facilitated by knowledge of which documents are most likely to be relevant exactly the mutual reinforcement property that hitswas designed to leverage .we will compare the results of using the hitsalgorithm against those derived using pagerank instead .this is a natural comparison because pagerank is the most well - known centrality - induction algorithm utilized for ranking documents , and because in earlier work , pagerank performed quite well as a tool for structural ingof non - web documents , at least when applied to document - to - documentgraphs .one can think of pagerank as a version of hitsin which the hub / authority distinction has been collapsed .thus , writing `` pr '' for both and , we _ conceptually _ have the ( single ) equation however , in practice , we incorporate brin and page s smoothing scheme together with a correction for nodes with no positive - weight edges emanating from them : \cdot { { \sf pr}\xspace}({u } ) \nonumber \\  & + &  \sum_{{u}\in { v } : { out({u } ) } = 0 } \frac{1}{|{v}|}\cdot { { \sf pr}\xspace}({u } ) \label{eq : pagerank}\end{aligned}\ ] ] where , and is the damping factor . , not 1 .conceptually , the role of the second summation in equation [ eq : pagerank ] is to set for these no - outflow nodes . ]equation [ eq : pagerank ] is recursive , but there are iterative algorithms that provably converge to the unique positive solution satisfying the sum - normalization constraint  .moreover , a ( non - trivial ) _ closed - form _ and quite easily computed solution exists for one - way bipartitegraphs :  if is one - way bipartite , then is an affine transformation ( with respect to positive constants ) of , and therefore equivalent for ranking purposes to , the unique positive sum - normalized solution to equation [ eq : pagerank ] .[ thm : pagerankbip ]  ( proof omitted due to space constraints . )interestingly , this result shows that while one might have thought that clusters and documents would `` compete '' for pagerank score when placed within the same graph , in our document - as - authorityand document - as - hubgraphs this is not the case .earlier work also considered scoring a node by its _ influx _ , .this can be viewed as either a non - recursive version of equation [ eq : pagerank - concept ] , or as an un - normalized analog of equation [ eq : bip - pagerank ] .clearly , we can rank documents by their scores as computed by any of the functions introduced above .but when we operate on document - as - authorityor document - as - hubgraphs , centralityscores for the clusters are also produced .these can be used to derive alternative means for ranking documents .we follow liu and croft s approach : first , rank the documents within ( or most strongly associated to ) each cluster according to the initial retrieval engine s scores ; then , derive the final list by concatenating the within - cluster lists in order of decreasing cluster score , discarding repeats .such an approach would be successful if cluster centralityis strongly correlated with the property of containing a large percentage of relevant documents .[ [ ranking - algorithms ] ] ranking algorithms + + + + + + + + + + + + + + + + + +  since we have two possible ranking paradigms , we adopt the following * algorithm naming conventions*. names consist of a hyphen - separated prefix and suffix .the prefix ( `` doc '' or `` clust '' ) indicates whether documents were ranked directly by their centralityscores , or indirectly through the concatenation process outlined above in which it is the _clusters _ centralityscores that were employed .the suffix ( `` auth '' , `` hub '' , `` pr '' , or `` influx '' ) indicates which score function ( , , ( or ) , or influx ) was used to measure centrality . for a given re - rankingalgorithm, we indicate the graph upon which it was run in brackets , e.g. , `` doc - auth[ '' .the potential merits of _ query - dependent clustering _ , that is , clustering the documents retrieved in response to a query , have long been recognized , especially in interactive retrieval settings .however , automatically detecting clusters that contain many relevant documents remains a very hard task .section [ sec : clustauth ] presents results for detecting such clusters using centrality - based cluster ranking .recently , there has been a growing body of work on graph - based modeling for different language - processing tasks wherein links are induced by inter - entity textual similarities .examples include document , text summarization , sentence retrieval , and document representation .in contrast to our methods , links connect entities of the same type , and clusters of entities are not modeled within the graphs .  while ideas similar to ours by virtue of leveraging the mutual reinforcement of entities of different types , or using bipartite graphs of such entities for clustering ( rather than using clusters ) , are abundant ( e.g. , ) , we focus here on exploiting mutual reinforcement in ad hoc retrieval .random walks ( with early stopping ) over bipartite graphs of terms and documents were used for query expansion , but in contrast to our work , no stationary solution was sought . a similar `` short chain '' approach utilizing bipartite graphs of clusters and documents for ranking an entire corpuswas recently proposed , thereby constituting the work most resembling ours .however , again , a stationary distribution was not sought .also , _ query drift _ prevention mechanisms were required to obtain good performance ; in our re - rankingsetting , we need not employ such mechanisms .most aspects of the evaluation framework described below are adopted from our previous experiments with non - cluster - based structural ing so as to facilitate direct comparison .section 4.1 of provides a more detailed justification of the experimental design . the main conceptual changes . for citation purposes ,the numbers reported in the current paper should be used . ] here are : a slightly larger parameter search - space for the `` out - degree '' parameter ( called the `` ancestry '' parameter in ) ; and , of course , the incorporation of clusters .  [ cols="<,^,^,^,^,^,^,^,^,^ " , ]   [ [ authorities - versus - hubs ] ] authorities versus hubs + + + + + + + + + + + + + + + + + + + + + + +  so far , we have only considered utilizing the authority scores that the hitsalgorithm produces .the chart below shows the effect of ranking entities by hub scores instead . specifically , the `` documents ? ''column compares ( i.e. , ranking documents by authoritativeness ) to ( i.e. , ranking documents by hubness ) ; similarly , the `` clusters ? ''column compares to .each entry depicts , in descending order of performance ( except for the one indicated tie ) as one moves left to right , those centralityscoring functions that lead to an improvement over the initial ranking : stands for `` authority '' and for `` hub '' .cases in which the improvement is significant are marked with a ` * ' .ll|*2l +  +  + & & & + & prec & & + ap & prec & & + & mrr & & + & prec & & + trec8 & prec & & + & mrr & & + & prec & & ( tie ) + wsj & prec & & + & mrr & & +  we see that in many cases , hub - based re - rankingdoes yield better performance than the initial ranking .but authority - based re - rankingappears to be an even better choice overall .[ [ hitson - pagerank - style - graphs ] ] hitson pagerank - style graphs + + + + + + + + + + + + + + + + + + + + + + + + + + + +  consider our comparison of doc - auth[]against doc - pagerank [ ] . as the notation suggests , this corresponds to running hitsand pagerankon the same graph , .but an alternative interpretation is that non - smoothed ( or no - random - jump ) pagerank , as expressed by equation ( [ eq : pagerank - concept ] ) , is applied to a _different _ version of wherein the original edge weights have been smoothed as follows : }}{({{u}{\to}{v } } ) } } { \stackrel{def}{=}}\frac{1 - { \lambda}}{|{v}| } + { \lambda}\frac{{{{\mathit wt}}{({{u}{\to}{v}})}}}{{out({u } ) } } \label{eq : reweight}\ ] ] ( we ignore nodes with no positive - weight out - edges to simplify discussion , and omit the superscripts for clarity ).  how does hitsperform on document - to - document graphs that are `` truly equivalent '' , in the sense of employing the above edge - weighting regime , to those that pagerankis applied to ?one reason this is an interesting question is that hitsassigns scores of zero to nodes that are not in the graph s largest connected component ( with respect to positive - weight edges , considered to be bi - directional ) .notice that the original graph may have several connected components , whereas utilizing }} ] edge weights .however , we can not discount the possibility that the performance differences might be due simply to the inclusion of the extra interpolation - parameter .moreover , in all but one case , the improved results were still below those for doc - pagerank[](and always lagged behind those of doc - auth [ ] ) .interestingly , the situation is qualitatively different if we consider graphs instead . in brief , we applied a smoothing scheme analogous to that described above , but only to edges leading from a left - hand node ( cluster ) to a right - hand node ( document ) '' in equation ( [ eq : reweight ] ) must be changed to the number of right - hand nodes . ] ; we thus preserved the one - way bipartitestructure . only in two of the nine evaluation settings did this change cause an increase in performance of doc - auth[]over the results attained under the original edge - weighting scheme , despite the fact that the re - weighting involves an extra free parameter .thus , while we have already demonstrated in previous sections of this paper that information about document - cluster similarity relationships is very valuable , the results just mentioned suggest that such information is more useful in `` raw '' form .[ [ re - anchoring - to - the - query ] ] re - anchoring to the query + + + + + + + + + + + + + + + + + + + + + + + + +  in previous work , we showed that pagerankcentralityscores induced over document - based graphs can be used as a multiplicative weight on document query - likelihood terms , the intent being to cope with cases in which centralityin and relevance are not strongly correlated . indeed , employing this technique on the ap , trec8 , and wsj corpora , prec increases from , and , to , and respectively .the same modification could be applied to the -based algorithms , although it is not particularly well - motivated in the hitscase . while pagerankscores correspond to a stationary distribution that could be loosely interpreted as a prior , in which case multiplicative combination with query likelihood is sensible , it is not usual to assign a probabilistic interpretation to hub or authority scores .nonetheless , for the sake of comparison completeness , we applied this idea to the doc - auth[]algorithm , yielding the following performance changes : from , , and to , and respectively .these results are still as good as and for two corpora better than those for pagerankas a multiplicative weight on query likelihood .thus , it may be the case that centralityscores induced over a document - based graph are more effective as a multiplicative bias on query - likelihood than as direct representations of relevance in ( see also ) ; but , modulo the caveat above , it seems that when centralityis induced over cluster - based one - way bipartitegraphs , the correlation with relevance is much stronger , and hence this kind of centralityserves as a better `` bias '' on query - likelihood .we have shown that leveraging the mutually reinforcing relationship between clusters and documents to determine centralityis very beneficial not only for directly finding relevant documents in an initially retrieved list , but also for finding clusters of documents from this list that contain a high number of relevant documents .specifically , we demonstrated the superiority of cluster - document bipartite graphs to document - only graphs as the input to centrality - induction algorithms .our method for finding `` authoritative '' documents ( or clusters ) using hits over these bipartite graphs results in state - of - the - art performance for document ( and cluster ) re - ranking .  * acknowledgments *  we thank eric breck , claire cardie , oren etzioni , jon kleinberg , art munson , filip radlinski , ves stoyanov , justin wick and the anonymous reviewers for valuable comments .this paper is based upon work supported in part by the national science foundation under grant no .iis-0329064 and an alfred p. sloan research fellowship .any opinions , findings , and conclusions or recommendations expressed are those of the authors and do not necessarily reflect the views or official policies , either expressed or implied , of any sponsoring institutions , the u.s .government , or any other entity .d. r. cutting , d. r. karger , j. o. pedersen , and j. w. tukey .catter / gather : a cluster - based approach to browsing large document collections . in _15th annual international sigir _ , pages 318329 , denmark , june 1992 .a. griffiths , h. c. luckhurst , and p. willett .using interdocument similarity information in document retrieval systems ., 37(1):311 , 1986 .reprinted in karen sparck jones and peter willett , eds ., _ readings in information retrieval _ , morgan kaufmann , pp . 365373 , 1997 .j. kleinberg .authoritative sources in a hyperlinked environment . in _ proceedings of the 9th acm - siam symposium on discrete algorithms ( soda ) _ , pages 668677 , 1998 .extended version in _ journal of the acm , 46:604632 , 1999 ._       a. leuski .evaluating document clustering for interactive information retrieval . in _ proceedings of the tenth international conference on information and knowledge managment ( cikm ) _ , pages 3340 , 2001 .a. leuski and j. allan .evaluating a visual navigation system for a digital library . in _ proceedings of the second european conference on research and advanced technology for digital libraries ( ecdl ) _ , pages 535554 , 1998 .j. otterbacher , g. erkan , and d. r. radev .using random walks for question - focused sentence retrieval . in _ proceedings of human language technology conference and conference on empirical methods in natural language processing ( hlt / emnlp ) _ , pages 915922 , 2005 .

Abstract: we present an approach to improving the precision of an initial document ranking wherein we utilize cluster information within a graph - based framework . the main idea is to perform re - ranking based on centrality within bipartite graphs of documents ( on one side ) and clusters ( on the other side ) , on the premise that these are mutually reinforcing entities . links between entities are created via consideration of language models induced from them .   we find that our cluster - document graphs give rise to much better retrieval performance than previously proposed document - only graphs do . for example , authority - based re - ranking of documents via a hits - style cluster - based approach outperforms a previously - proposed pagerank - inspired algorithm applied to solely - document graphs . moreover , we also show that computing authority scores for clusters constitutes an effective method for identifying clusters containing a large percentage of relevant documents .