资源描述:
《Test Suites for Quality Evaluation of NLP Products》由会员上传分享,免费在线阅读,更多相关内容在行业资料-天天文库。
1、TestSuitesforQualityEvaluationofNLPProductsFrederikFouvryandLornaBalkanDepartmentofLanguageandLinguistics,UniversityofEssexWivenhoePark,ColchesterEssexCO43SQ,UK(fouvry
2、balka)@essex.ac.ukAbstractandshowshowthetestsuitecanbeusedinprac-ticetoprovideareliableassessmentmethodofthe
3、linguisticcapabilitiesofasystem.TestsuitesareausefulevaluationtoolfordevelopersandusersofNLPprod-ucts.Thepapergivesanoverviewofthe2DesignofTestSuitestsnlpdesignandmethodologyandde-scribeshowthetsnlpdataandmethod-Bytestsuiteismeantamoreorlesssystematicologycanbeusedinpracticet
4、oprovidecollectionofspeciallyconstructedlinguisticex-areliableassessmentmethodofthelin-pressions(testitems,e.g.sentences),perhapswithguisticcapabilitiesofNLPproducts.associatedannotationsanddescriptions.Mostex-istingpubliclyavailabletestsuiteshavebeenwrit-tenforspecicsystems
5、orsimplycontainnumbersofinterestingexamples".Theyhaveonlylim-itedorveryspecicannotations.Fortestsuites1Introductiontobereusable
6、sincetherearefewpubliclyavail-Testsuiteshavelongbeenrecognizedasausefulabletestsuitesaround,andhencealotofworkisevaluationtoolforNLPproducts,espec
7、iallyforduplicatedbywritingsuitesforspecicsystem
8、thedeveloper.Untilrecently,however,therehavetheremustbeannotationstoprovideaccessviabeenfewpubliclyavailabletestsuitesorguidelinesmanydierentsearchkeys.fortheirconstructionanduse.TheEU-fundedThisrequirementleadsustowardsalarg
9、e,sys-projecttsnlp(TestSuitesforNaturalLanguagetematicallyconstructedtestsuitethatcanbe1Processing,LRE62-089),haslledthatgapbyappliedtoawidevarietyofapplications.TheprovidingamethodologyfortheconstructionofHPtestsuite(Flickingeretal.87)wentsometestsuites,asubstantialmultilin
10、gualtestsuitewaytowardsthatgoal.However,itsusabilityand(over4,500testitemsineachofitsthreelan-extendibilityisrestrictedbythelackofsupport-guages,French,GermanandEnglish),mountedingdocumentationandannotations.TheDiToonadatabase,andanumberoftoolsforusetestsuite(Nerbonneetal.91)
11、pioneeredmoreex-andconstruction.Thetestdataaregeneralpur-tensiveanno