资源描述：
《Efficient Learning of Sparse Representations with an Energy-Based Model》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、EfficientLearningofSparseRepresentationswithanEnergy-BasedModelMarc'AurelioRanzato,ChristopherPoultney,SumitChopra,andYannLeCunCourantInstituteofMathematicalSciencesNewYorkUniversity,NewYork,NY10003{ranzato,crispy,sumit,yann}@cs.nyu.eduAbstractWedescr
2、ibeanovelunsupervisedmethodforlearningsparse,overcompletefea-tures.Themodelusesalinearencoder,andalineardecoderprecededbyaspar-sifyingnon-linearitythatturnsacodevectorintoaquasi-binarysparsecodevec-tor.Givenaninput,theoptimalcodeminimizesthedistanceb
3、etweentheoutputofthedecoderandtheinputpatchwhilebeingassimilaraspossibletotheen-coderoutput.Learningproceedsinatwo-phaseEM-likefashion:(1)computetheminimum-energycodevector,(2)adjusttheparametersoftheencoderandde-codersoastodecreasetheenergy.Themodel
4、produces“strokedetectors”whentrainedonhandwrittennumerals,andGabor-likefilterswhentrainedonnaturalimagepatches.Inferenceandlearningareveryfast,requiringnopreprocessing,andnoexpensivesampling.Usingtheproposedunsupervisedmethodtoinitializethefirstlayerof
5、aconvolutionalnetwork,weachievedanerrorrateslightlylowerthanthebestreportedresultontheMNISTdataset.Finally,anextensionofthemethodisdescribedtolearntopographicalfiltermaps.1IntroductionUnsupervisedlearningmethodsareoftenusedtoproducepre-processorsandfe
6、atureextractorsforimageanalysissystems.PopularmethodssuchasWaveletdecomposition,PCA,Kernel-PCA,Non-NegativeMatrixFactorization[1],andICAproducecompactrepresentationswithsomewhatuncor-related(orindependent)components.Mostmethodsproducerepresentationst
7、hateitherpreserveorreducethedimensionalityoftheinput.However,severalrecentworkshaveadvocatedtheuseofsparse-overcompleterepresentationsforimages,inwhichthedimensionofthefeaturevectorislargerthanthedimensionoftheinput,butonlyasmallnumberofcomponentsare
8、non-zeroforanyoneimage[2,3].Sparse-overcompleterepresentationspresentseveralpotentialadvantages.Usinghigh-dimensionalrepresentationsincreasesthelikelihoodthatimagecategorieswillbeeasily(possiblylinearly)separable.Sparserepresentationscanprovideasimpl
