资源描述：
《NEAL-MONTGOMERY NLP SYSTEM EVALUATION METHODOLOGY》由会员上传分享，免费在线阅读，更多相关内容在行业资料-天天文库。
1、NEAL-MONTGOMERYNLPSYSTEMEVALUATIONMETHODOLOGYSharonM.WalterRomeLaboratoryRL/C3CAGriffissAFB,NY13441-5700walter@aivax.rl.af.milABSTRACTfeature.IllustrativelanguagepatternsandsamplesentencesthenguidethehumanevaluatortotheformulationofanOnwhatbasisaretheinputproc
2、essingcapabilitiesofNaturalinputthatteststhefeatureontheNLPsystemwithintheLanguagesoftwarejudged?Thatis,whatarethecapabilitiestosystem'snativedomain.bedescribedandmeasured,andwhatarethestandardsagainstwhichwemeasurethem?RomeLaboratoryiscurrentlyBasedonclearand
3、specificevaluationcriteriafortestitemsupportinganefforttodevelopaconciseterminologyforinputs,NLPsystemresponsesarescoredasfollows:describingthelinguisticprocessingcapabilitiesofNaturalLanguageSystems,andauniformmethodologyforS:Thesystemsuccessfullymetthestated
4、criteriaandappropriatelyapplyingtheterminology.Thismethodologyisdemonstratedunderstandingwithrespecttothefeatureundermeanttoproducequantitative,objectiveprofilesofNLsystemcapabilitieswithoutrequiringsystemadaptationtoanewtesttest.domainortextcorpus.Theeffortpr
5、oposestodeveloparepeatableprocedurethatproducesconsistentresultsforC:Thesystemrespondedinawaythatwascorrectindependentevaluators.(thatis,correctlyansweredthequestionposed),butthecriteriawerenotmet.1.INTRODUCTIONP:ThesystemrespondedinawaythatwasonlyAnappreciabl
6、edrawbacktocurrentcorpus-based(eg.,partiallycorrect.[BBN;1988],[Flickinger,etal;1987],[Hendrix,etal;1976],[Malhotra;1975])andtask-based(eg.,F:Thesystemrespondedinawaythatwasincorrect,["Proceedings";1991])methodologiesforevaluatingfailingtomeetthecriteria.Natur
7、alLanguageProcessingSystemsistherequirementN:Thesystemwasunabletoaccepttheinputorformfortransportationofthesystemtoatestdomain.Thearesponse(forexample,thesystemvocabularylacksexpenseandtimeconsumptionaresizableand,astheportappropriatewordstocompleteatestinpu0.
8、maybeminimalorincomplete,theevaluationmaybebasedonademonstrationoflessthanthefullpotentialofthesystem.Further,currentevaluationmethodologiesdoEachlinguisticfeatureistestedbymoretha
