资源描述：
《2016-Adversarial Training Methods for Semi-Supervised Text Classification》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、UnderreviewasaconferencepaperatICLR2017ADVERSARIALTRAININGMETHODSFORSEMI-SUPERVISEDTEXTCLASSIFICATIONTakeruMiyato1,2,AndrewMDai2,IanGoodfellow3takeru.miyato@gmail.com,adai@google.com,ian@openai.com1KyotoUniversity,2GoogleBrainand3OpenAIABSTRACTAdversarialtrainingpro
2、videsameansofregularizingsupervisedlearningalgo-rithmswhilevirtualadversarialtrainingisabletoextendsupervisedlearningal-gorithmstothesemi-supervisedsetting.However,bothmethodsrequiremakingsmallperturbationstonumerousentriesoftheinputvector,whichisinappropri-ateforsp
3、arsehigh-dimensionalinputssuchasone-hotwordrepresentations.Weextendadversarialandvirtualadversarialtrainingtothetextdomainbyapplyingperturbationstothewordembeddingsinarecurrentneuralnetworkratherthantotheoriginalinputitself.Theproposedmethodachievesstateoftheartresu
4、ltsonmultiplebenchmarksemi-supervisedandpurelysupervisedtasks.Weprovidevisualizationsandanalysisshowingthatthelearnedwordembeddingshaveim-provedinqualityandthatwhiletraining,themodelislesspronetooverfitting.1INTRODUCTIONAdversarialexamplesareexamplesthatarecreatedbym
5、akingsmallperturbationstotheinputde-signedtosignificantlyincreasethelossincurredbyamachinelearningmodel(Szegedyetal.,2014;Goodfellowetal.,2015).Severalmodels,includingstateoftheartconvolutionalneuralnetworks,lacktheabilitytoclassifyadversarialexamplescorrectly,someti
6、mesevenwhentheadversarialperturbationisconstrainedtobesosmallthatahumanobservercannotperceiveit.Adversarialtrainingistheprocessoftrainingamodeltocorrectlyclassifybothunmodifiedexamplesandad-versarialexamples.Itimprovesnotonlyrobustnesstoadversarialexamples,butalsogen
7、eralizationperformancefororiginalexamples.Adversarialtrainingrequirestheuseoflabelswhentrainingmodelsthatuseasupervisedcost,becausethelabelappearsinthecostfunctionthattheadversarialperturbationisdesignedtomaximize.Virtualadversarialtraining(Miyatoetal.,2016)extendst
8、heideaofadversarialtrainingtothesemi-supervisedregimeandunlabeledexamples.Thisisdonebyregularizingthemodelsothatgivenanexample,themodelwil
