模式识别的经典教材.pdf

《模式识别的经典教材.pdf》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、Lecture8:TheKNearestNeighborRule(k-NNR)gIntroductiongk-NNRinactiongk-NNRasalazyalgorithmgCharacteristicsofthek-NNRclassifiergOptimizingstoragerequirementsgFeatureweightinggImprovingthenearestneighborsearchIntroductiontoPatternRecognition1RicardoGutierrez-OsunaWrightStateUniversityI

2、ntroductiongTheKNearestNeighborRule(k-NNR)isaveryintuitivemethodthatclassifiesunlabeledexamplesbasedontheirsimilaritywithexamplesinthetrainingsetDnForagivenunlabeledexamplexuÎÂ,findthek“closest”labeledexamplesinthetrainingdatasetandassignxtotheclassthatappearsmostufrequentlywithint

3、hek-subsetgThek-NNRonlyrequiresnAnintegerknAsetoflabeledexamples(trainingdata)nAmetrictomeasure“closeness”w1w2gExamplenIntheexamplebelowwehavethreeclassesandthegoalistofindaclasslabelforthexuunknownexamplexunInthiscaseweusetheEuclideandistanceandavalueofk=5neighborsw3nOfthe5closest

4、neighbors,4belongtow1and1belongstow,soxuisassignedtow,the31predominantclassIntroductiontoPatternRecognition2RicardoGutierrez-OsunaWrightStateUniversityk-NNRinaction:example1gWehavegenerateddatafora2-dimensional3-classproblem,wheretheclass-conditionaldensitiesaremulti-modal,andnon-l

5、inearlyseparable,asillustratedinthefiguregWeusedthek-NNRwithnk=fivenMetric=EuclideandistancegTheresultingdecisionboundariesanddecisionregionsareshownbelowIntroductiontoPatternRecognition3RicardoGutierrez-OsunaWrightStateUniversityk-NNRinaction:example2gWehavegenerateddatafora2-dime

6、nsional3-classproblem,wheretheclass-conditionaldensitiesareunimodal,andaredistributedinringsaroundacommonmean.Theseclassesarealsonon-linearlyseparable,asillustratedinthefiguregWeusedthek-NNRwithnk=fivenMetric=EuclideandistancegTheresultingdecisionboundariesanddecisionregionsareshow

7、nbelowIntroductiontoPatternRecognition4RicardoGutierrez-OsunaWrightStateUniversityk-NNR,alazy(machinelearning)algorithmgK-NNRisconsideredalazylearningalgorithm[Aha]nDefersdataprocessinguntilitreceivesarequesttoclassifyanunlabelledexamplenRepliestoarequestforinformationbycombiningit

8、sstoredtrainingdatanDiscar