资源描述:
《Test Suites for NLP》由会员上传分享,免费在线阅读,更多相关内容在行业资料-天天文库。
1、TestSuitesforNLPLornaBalkan,DouglasArnold,FrederikFouvryDepartmentofLanguageandLinguisticsUniversityofEssexWivenhoeParkColchester,CO43SQ,UK.balka@essex.ac.ukdoug@essex.ac.ukfouvry@essex.ac.uk1IntroductionInrecentyearstherehasbeenadrivetoproducebetterresourcesfortheevaluationofNLPsys
2、tems.TheEUfundedLREprojectTSNLP(TestSuitesforNatural1LanguageProcessing)shouldbeseeninthiscontext.Bytestsuiteismeantamoreorlesssystematiccollectionofspeciallyconstructedlinguisticexpressions(testitems,e.g.sentences),perhapswithassociatedannotationsanddescriptions.Theprojectwasmotiva
3、tedbythefactthattestsuiteshavelongbeenacceptedintheNLPcommunityasausefultoolfordiagnosticevaluation,andbythefactthattherearefewpubliclyavailabletestsuites.Moreover,mostoftheexistingtestsuiteshavebeenwrittenforspecicsystemsorsimplycontainnumbersofinterestingexamples".Thisdoesnotmee
4、tthedemandforlarge,systematic,welldocumentedandannotatedcollectionsoflinguisticmaterial,requiredbyagrowingnumberofNLPapplications.Afurtherproblemfortheprospectivetestsuiteuseristhattherearefewguidelinesfortheconstructionoftestsuites.TheTSNLPprojectthushastwoprincipleaims:(1)toprovid
5、eguidelinesfortestsuiteconstructionand(2)toconstructsubstantialtestfragments.Thispaperdiscussesproblemsassociatedwithboththedesignandconstructionoftestmaterial,issuessuchastheidenticationoflinguisticphenomenaandsub-phenomenatobetested,thechoiceofthevocabularyusedintestdata,howoneca
6、nderiveill-formeddataandtowhatextenttheinteractionbetweendierentphenomenashouldbetested.Wealsolookatsomeofthemethodsdevelopedwithintheprojectforautomatingtestsuiteconstruction,namelyatestsuitegenerationtoolandanautomaticlexiconreplacementtool.1WewouldliketothankourpartnersintheTSNL
7、Pproject:DominiqueEstival,KirstenFalkedalandSabineLehmannatISSCO,Geneva,EvaDauphinandVeronikaLuxandSylvieRegnier-ProstatAerospatiale,Paris,andJudithKlein,KlausNetterandStephanOepenatDFKIinSaarbrucken.1OneofthemaincharacteristicsofTSNLPisthatitisamultilingualprojectthataimstoproducep
8、aralleltestdatainth