BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Automatic Evaluation of Linguistic Quality in Multi-Document Summa
 rization - Helen Yannakoudakis
DTSTART:20110127T120000Z
DTEND:20110127T130000Z
UID:TALK29486@talks.cam.ac.uk
CONTACT:Marek Rei
DESCRIPTION:Helen will present the following paper:\n\nAutomatic Evaluatio
 n of Linguistic Quality in Multi-Document Summarization. ACL2010. Emily Pi
 tler\, Annie Louis\, Ani Nenkova.\n\nhttp://www.cis.upenn.edu/~epitler/pap
 ers/ACL10Sum.pdf\n\nTo date\, few attempts have been made to develop and v
 alidate methods for automatic evaluation of linguistic quality in text sum
 marization. We present the first systematic assessment of several diverse 
 classes of metrics designed to capture various aspects of well-written tex
 t. We train and test linguistic quality models on consecutive years of NIS
 T evaluation data in order to show the generality of results. For grammati
 cality\, the best results come from a set of syntactic features. Focus\, c
 oherence and referential clarity are best evaluated by a class of features
  measuring local coherence on the basis of cosine similarity between sente
 nces\, coreference information\, and summarization specific features. Our 
 best results are 90% accuracy for pairwise comparisons of competing system
 s over a test set of several inputs and 70% for ranking summaries of a spe
 cific input.
LOCATION:GS15\, Computer Laboratory
END:VEVENT
END:VCALENDAR