senseval 2nd call for participants and papers * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * second call for participation * * wsd evaluation exercise * * ( senseval ) * * * * workshop announcement * * " senseval and the lexicography loop " * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * sponsored by acl siglex and euralex there are now many automatic word sense disambiguation ( wsd ) programs but it is currently very hard to determine which are better , which worse , and where the strengths and weaknesses of each lie . there is widespread agreement that the field urgently needs an evaluation framework . under the auspices of acl siglex and euralex , a pilot will take place in the course of 1998 . as in arpa evaluation exercises , the framework comprises : 1 ) definition of task and scoring metric 2 ) preparation of a set of manually tagged correct answers 3 ) a dry run , with sample data distributed to participants 4 ) distribution of test data to participants ; participants sense-tag and return ; taggings scored against correct answers 5 ) workshop to discuss results , lessons learned , way forward we shall be undertaking evaluation for at least english , french , italian and spanish . the workshop will be held at herstmonceux castle , sussex , uk on sept 2 - 4 1998 if you have a working wsd program ( or will have one by summer 1998 ) , and would like to subject it to objective , quantitative evaluation , or if you have skills or resources that you would like to contribute to the exercise , first look at http : / / www . itri . bton . ac . uk / events / senseval / exp-interest . html and then mail your expression of interest to senseval-coord @ itri . bton . ac . uk timetable = = = = = = = = = as soon as possible : expressions of interest to be registered by end of : april 98 : 'd ry run ' data samples distributed to participants june 98 : test data distributed to participants , and , one week later , returned with taggings july 98 : correct answers and scores made available sept 98 : workshop the workshop will comprise : ( 1 ) reports / papers on the manual tagging and other aspects of the evaluation exercise ( 2 ) reports / papers from participants on the design and performance of their system ( 3 ) other related research papers ( 4 ) working sessions on the way ahead for wsd evaluation the deadline for " other related research papers " is 16th april 1998 * preference will be given to papers discussing wsd , with particular attention to evaluation issues * maximum submission length : 6 pages * first page to include title , abstract , and author 's name ( s ) and contact details * electronic submission of postscript documents permitted but must be supported by hard copy to arrive not later than 23rd april ( in case of printing problems ) . email : senseval-submissions @ itri . bton . ac . uk hard copy : senseval submissions itri university of brighton lewes road brighton bn2 4gj , uk full details available at http : / / www . itri . bton . ac . uk / events / senseval / adam kilgarriff senseval co-ordinator
