资源描述：
《Maximum Likelihood Estimation (MLE)》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、MaximumLikelihoodEstimation(MLE)1SpecifyingaModelTypically,weareinterestedinestimatingparametricmodelsoftheformyi»f(µ;yi)(1)whereµisavectorofparametersandfissomespeci¯cfunctionalform(probabilitydensityormassfunction).1Notethatthissetupisquitegeneralsincethespeci¯cfunctionalfor
2、m,f,providesanalmostunlimitedchoiceofspeci¯cmodels.Forexample,wemightuseoneofthefollowingdistributions:²PoissonDistributione¡¸¸yiyi»f(¸;yi)=(2)yi!Asyoucansee,wehaveonlyoneparametertoestimate:¸.IntermsofEq.1,µ=¸.²BinomialDistributiony»f(¼;y)=N!¼yi(1¡¼)N¡yi(3)iiyi!(N¡yi)!orµ¶y»f
3、(¼;y)=N¼yi(1¡¼)N¡yi(4)iiyiAgain,wehaveonlyoneparametertoestimate:¼.IntermsofEq.1,µ=¼.²NormalDistribution21¡(yi¡¹i)yi»fN(µ;yi)=pe2¾2(5)2¼¾2whereµ=¹;¾2and¹=g((¯;xi).Asyoucansee,wehavetwoparameterstoestimate:¯and¾2.IntermsofEq.1,µ=¯;¾2.Obviouslythechoiceofdistributionwilldependon
4、yourtheory.1Probabilitymassfunctionsapplytodiscreterandomvariables,whereasprobabilitydensityfunctionsapplytocontinuousrandomvariables.12IntuitionThefollowingexampleprovidessomeintuitionaboutmaximumlikelihoodestimation.Supposeourdependentvariablefollowsanormaldistribution:y»N(¹
5、;¾2)(6)iThus,wehave:E[y]=¹(7)var(y)=¾2(8)Ingeneral,wewillhavesomeobservationsonYandwewanttoestimate¹and¾2fromthosedata.Theidea,aswewillsee,ofmaximumlikelihoodisto¯ndtheestimateoftheparameter(s)thatmaximizestheprobabilityofobservingthedatathatwehave.Supposethatwehavethefollowin
6、g¯veobservationsonY:Y=f54;53;49;61;58g(9)Intuitively,wemightwonderabouttheoddsofgettingthese¯vedatapointsiftheywerefromanormaldistributionwith¹=100.Theanswerhereisthatitisnotverylikely{allofthedatapointsarealongwayfrom100.Butwhataretheoddsofgettingthe¯vedatapointsfromanormaldi
7、stributionwith¹=55.Nowthisseemsmuchmorereasonable.Maximumlikelihoodestimationisjustasystematicwayofsearchingfortheparametervaluesofourchosendistributionthatmaximizetheprobabilityofobservingthedatathatweobserve.Before,welookattheprocessofmaximumlikelihoodestimationindetail,wene
8、edtogooversomepreliminaries¯rst.3Preliminaries3.1FundamentalP
