[dataguru.cn]Understanding Deep Convolutional Networks

《[dataguru.cn]Understanding Deep Convolutional Networks》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、UnderstandingDeepConvolutionalNetworksStephaneMallatEcoleNormaleSuperieure,CNRS,PSL45rued'Ulm,75005Paris,FranceToappearinPhilosophicalTransactionsAin2016AbstractDeepconvolutionalnetworksprovidestateoftheartclassicationsandregressionsresultsovermanyhigh-dimensional

2、problems.Wereviewtheirarchitecture,whichscattersdatawithacascadeoflinearlterweightsandnon-linearities.Amathematicalframeworkisintroducedtoanalyzetheirproperties.Computationsofinvariantsinvolvemultiscalecontractions,thelinearizationofhierarchicalsymmetries,andsparsese

3、parations.Applicationsarediscussed.x1IntroductionSupervisedlearningisahigh-dimensionalinterpolationproblem.Weapproximateafunctionf(x)fromqtrainingsamplesfxi;f(xi)g,wherexisadatavectorofveryhighdimensiond.Thisdimensionisofteniqlargerthan106,forimagesorotherlargesizesi

4、gnals.Deepconvolutionalneuralnetworkshaverecentlyobtainedremarkableexperimentalresults[21].Theygivestateoftheartperformancesforimageclassicationwiththousandsofcomplexclasses[19],speechrecognition[17],bio-medicalapplications[22],naturallanguageunderstanding[30],andinm

5、anyotherdomains.Theyarealsostudiedasneuro-physiologicalmodelsofvision[4].Multilayerneuralnetworksarecomputationallearningarchitectureswhichpropagatetheinputdataacrossasequenceoflinearoperatorsandsimplenon-linearities.Thepropertiesofshallownetworks,withonehiddenlayer,a

6、rewellunderstoodasdecompositionsinfamiliesofridgefunctions[10].However,theseapproachesdonotextendtonetworkswithmorelayers.Deepconvolutionalneuralnetworks,introducedbyarXiv:1601.04920v1[stat.ML]19Jan2016LeCun[20],areimplementedwithlinearconvolutionsfollowedbynon-linear

7、ities,overtypicallymorethan5layers.Thesecomplexprogrammablemachines,denedbypotentiallybillionsoflterweights,bringustoadierentmathematicalworld.Manyresearchershavepointedoutthatdeepconvolutionnetworksarecomputingprogressivelymorepowerfulinvariantsasdepthincreases[4,

8、21],butrelationswithnetworksweightsandnon-linearitiesarecomplex.Thispaperaimsatclarifyingimportantprincipleswhichgovernthepr