10_11_recursive_network

《10_11_recursive_network》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、NeuralnetworksNaturallanguageprocessing-recursivenetwork2FROMWORDSTOPHRASESTopics:wordphraserepresentations•We’veseenhowtolearnrepresentationsforsinglewords•Howcouldwelearnrepresentationsforphrasesofarbitrarylength?‣canwemodelrelationshipsbetweenwordsandmultiwordexpressions-

2、ex.:‘‘consider’’≈‘‘takeintoaccount’’‣canweextractarepresentationoffullsentencesthatpreservessomeofitssemanticmeaning-ex.:‘‘wordrepresentations‘‘wetrainedwordwerelearnedfroma≈representationsoncorpus’’atextdataset’‘toformawhole.Weintroduceamax-margin3structurepredictionarchite

3、cturebasedonre-cursiveneuralnetworksthatcansuccessfullyRECURSIVENEURALNETWORKrecoversuchstructurebothincomplexsceneimagesaswellassentences.Thesamealgo-Topics:recursiveneuralnetwork(RNN)rithmcanbeusedbothtoprovideacompeti-•Idea:recursivelymergepairsofword/phraserepresentation

4、stivesyntacticparserfornaturallanguagesen-tencesfromthePennTreebankandtoout-performalternativeapproachesforsemanticscenesegmentation,annotationandclassifi-cation.Forsegmentationandannotationouralgorithmobtainsanewlevelofstate-of-the-wordrepresentations{artperformanceontheStan

5、fordbackgrounddataset(78.1%).Thefeaturesfromtheim-ageparsetreeoutperformGistdescriptorsfor•Weneed2thingsFigure1.Illustrationofourrecursiveneuralnetworkar-Socher,Lin,NgandManning,2011sceneclassificationby4%.chitecturewhichparsesimagesandnaturallanguagesen-‣amodelthatmergespair

6、sofrepresentationstences.Segmentfeaturesandwordindices(orange)are‣amodelthatdeterminesthetreestructurefirstmappedintosemanticfeaturespace(blue)andthen1.Introductionrecursivelymergedbythesameneuralnetworkuntiltheyrepresenttheentireimageorsentence.BothmappingsandRecursivestruct

7、ureiscommonlyfoundindifferentmergingsarelearned.modalities,asshowninFig.1.Thesyntacticrulescanberecursivelysplitintosmallercarregionsdepict-ofnaturallanguageareknowntoberecursive,withingpartssuchastiresandwindowsandthesepartscannounphrasescontainingrelativeclausesthatthem-occ

8、urinothercontextssuchasbeneathairplanesorinselvescontainnounphrases,e.g.,..