Using Derivation Trees for Informative Treebank Inter-Annotator Agreement Evaluation
Seth Kulick, Ann Bies, Justin Mott, Mohamed Maamouri, Beatrice Santorini and Anthony Kroch
This paper discusses the extension of a system developed for automatic
treebank annotation inconsistencies over an entire corpus to the particular
case of evaluation of
This system makes for a more informative IAA evaluation than other systems
because it pinpoints the
inconsistencies and groups them by their structural types.
We evaluate the system on two corpora - (1) a corpus of
English web text, and (2) a corpus of Modern British English.
Back to Papers Accepted