资源描述：
《Neural Turing Machines》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、NeuralTuringMachinesAlexGravesgravesa@google.comGregWaynegregwayne@google.comIvoDanihelkadanihelka@google.comGoogleDeepMind,London,UKAbstractWeextendthecapabilitiesofneuralnetworksbycouplingthemtoexternalmemoryre-sources,whichtheycaninteractwithbyattentionalprocesses.Thecombinedsyst
2、emisanalogoustoaTuringMachineorVonNeumannarchitecturebutisdifferentiableend-to-end,allowingittobeefficientlytrainedwithgradientdescent.Preliminaryresultsdemon-stratethatNeuralTuringMachinescaninfersimplealgorithmssuchascopying,sorting,andassociativerecallfrominputandoutputexamples.1I
3、ntroductionComputerprogramsmakeuseofthreefundamentalmechanisms:elementaryoperations(e.g.,arithmeticoperations),logicalflowcontrol(branching),andexternalmemory,whicharXiv:1410.5401v2[cs.NE]10Dec2014canbewrittentoandreadfrominthecourseofcomputation(VonNeumann,1945).De-spiteitswide-rang
4、ingsuccessinmodellingcomplicateddata,modernmachinelearninghaslargelyneglectedtheuseoflogicalflowcontrolandexternalmemory.Recurrentneuralnetworks(RNNs)standoutfromothermachinelearningmethodsfortheirabilitytolearnandcarryoutcomplicatedtransformationsofdataoverextendedperiodsoftime.More
5、over,itisknownthatRNNsareTuring-Complete(SiegelmannandSontag,1995),andthereforehavethecapacitytosimulatearbitraryprocedures,ifproperlywired.Yetwhatispossibleinprincipleisnotalwayswhatissimpleinpractice.Wethereforeenrichthecapabilitiesofstandardrecurrentnetworkstosimplifythesolutiono
6、falgorithmictasks.Thisenrichmentisprimarilyviaalarge,addressablememory,so,byanalogytoTuring’senrichmentoffinite-statemachinesbyaninfinitememorytape,we1dubourdevicea“NeuralTuringMachine”(NTM).UnlikeaTuringmachine,anNTMisadifferentiablecomputerthatcanbetrainedbygradientdescent,yieldinga
7、practicalmechanismforlearningprograms.Inhumancognition,theprocessthatsharesthemostsimilaritytoalgorithmicoperationisknownas“workingmemory.”Whilethemechanismsofworkingmemoryremainsome-whatobscureatthelevelofneurophysiology,theverbaldefinitionisunderstoodtomeanacapacityforshort-termsto
8、rageofinformationan
