2005-A framework for learning predictive structures from multiple tasks and unlabeled data

《2005-A framework for learning predictive structures from multiple tasks and unlabeled data》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、JournalofMachineLearningResearch6(2005)1817–1853Submitted5/05;Revised8/05;Published11/05AFrameworkforLearningPredictiveStructuresfromMultipleTasksandUnlabeledDataRieKubotaAndorie1@us.ibm.comIBMT.J.WatsonResearchCenterYorktownHeights,NY10598,U.S.A.TongZhangtzhang@yahoo-inc.c

2、omYahooResearchNewYork,NY,U.S.A.Editor:PeterBartlettAbstractOneofthemostimportantissuesinmachinelearningiswhetheronecanimprovetheperformanceofasupervisedlearningalgorithmbyincludingunlabeleddata.Methodsthatusebothlabeledandunlabeleddataaregenerallyreferredtoassemi-supervise

3、dlearning.Althoughanumberofsuchmethodsareproposed,atthecurrentstage,westilldon’thaveacompleteunderstandingoftheireffectiveness.Thispaperinvestigatesacloselyrelatedproblem,whichleadstoanovelapproachtosemi-supervisedlearning.Specificallyweconsiderlearningpredictivestructuresonh

4、ypothesisspaces(thatis,whatkindofclassifiershavegoodpredictivepower)frommultiplelearningtasks.Wepresentageneralframeworkinwhichthestructurallearningproblemcanbeformulatedandanalyzedtheoretically,andrelateittolearningwithunlabeleddata.Underthisframework,algorithmsforstructura

5、llearningwillbeproposed,andcomputationalissueswillbeinvestigated.Experimentswillbegiventodemonstratetheeffectivenessoftheproposedalgorithmsinthesemi-supervisedlearningsetting.1.IntroductionInmachinelearningapplications,onecanoftenfindalargeamountofunlabeleddatawithoutdifficulty

6、,whilelabeleddataarecostlytoobtain.Thereforeanaturalquestioniswhetherwecanuseunlabeleddatatobuildamoreaccurateclassifier,giventhesameamountoflabeleddata.Thisproblemisoftenreferredtoassemi-supervisedlearning.Ingeneral,semi-supervisedlearningalgorithmsusebothlabeledandunlabele

7、ddatatotrainaclassifier.Althoughanumberofmethodshavebeenproposed,theireffectivenessisnotalwaysclear.Forexample,Vapnikintroducedthenotionoftransductiveinference(Vapnik,1998),whichmayberegardedasanapproachtosemi-supervisedlearning.Al-thoughsomesuccesshasbeenreported(e.g.,seeJoa

8、chims,1999),therehasalsobeencriticismpointingoutthatthismethodmaynotbehavewellunde