On "deep evaluation" for individual
computational grammars and for
cross-framework comparison
Lars Hellan
Abstract
A rather difficult point in grammar engineering evaluation is how to
test and compare for analytic adequacy. A test design for 'deep'
grammars is here proposed, where a parse is considered valid only if
the assignment of syntactic and semantic structures that it displays
obey certain conditions. The set of grammatical sentences in the test
suite is construed as leaf types in a construction ontology, where the
top types introduce the discriminants according to which constructions
are categorized. These discriminants conform to notions shared across
linguistic frameworks, and the validity conditions are defined within a
well-known space of analytic parameters. One may envisage that with
such a design, a meeting point can emerge for comparing frameworks with
regard to agreed-upon aspects of linguistic content, and individual
grammars with regard to their analytic aims and actual achievements
relative to the aims.
Proceedings of GEAF07; CSLI Publications On-line
Proceedings TOC
Proceedings as a single large pdf file