资源描述：
《Notes on Information Theory and StatisticsNotes on Information Theory and Statistics》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、NotesonInformationTheoryandStatisticsbyImreCsiszar1RenyiInstituteofMathematics,HungarianAcademyofSciencesPOB127,H-1364Budapest,Hungaryemail:csiszar@renyi.huandPaulC.ShieldsProfessorEmeritusofMathematics,UniversityofToledoemail:paul.shields@utoledo.eduPrefaceThesenotesareconce
2、rnedwithapplicationsofinformationtheorycon-ceptsinstatistics.TheyoriginatedaslecturesgivenbyImreCsiszarattheUniversityofMarylandin1989withlateradditionsandcorrectionsbyCsiszarandPaulShields.Attentionisrestrictedto nitealphabetmodels.Thisexcludessomecele-bratedapplicationssuch
3、astheinformationtheoreticproofofthedichotomytheoremforGaussianmeasures,orofSanov'stheoreminageneralsetting,butconsiderablysimpli esthemathematicsandadmitscombinatorialtech-niques.Evenwithinthe nitealphabetsetting,noe ortsweremadeatcompleteness.Rather,sometypicaltopicswereselect
4、ed,accordingtotheauthors'researchinterests.Inallofthem,theinformationmeasureknownasinformationdivergence(I-divergence)orKullback-Leiblerdistanceorrel-ativeentropyplaysabasicrole.Severalofthesetopicsinvolve"information1SupportedbytheHungarianNationalFoundationforScienti cResearc
5、h,GrantsT32323andTS40719.1geometry",thatis,resultsofageometric
avorwithI-divergenceintheroleofsquaredEuclideandistance.InSection1,acombinatorialtechniqueofmajorimportanceininforma-tiontheoryisappliedtolargedeviationandhypothesistestingproblems.TheconceptofI-projectionsisaddress
6、edinSections2and3,withappli-cationstomaximumlikelihoodestimationinexponentialfamiliesand,inparticular,totheanalysisofcontingencytables.Iterativealgorithmsbasedoninformationgeometry,tocomputeI-projectionsandmaximumlikelihoodestimates,areanalysedinSection4.Thestatisticalprinciple
7、ofminimumdescriptionlength(MDL)ismotivatedbyideasinthetheoryofuniversalcoding,thetheoreticalbackgroundforecientdatacompression.Sections5and6aredevotedtothelatter.Here,again,amajorroleisplayedbycon-ceptswithageometric
avorthatwecallI-radiusandI-centroid.Finally,theMDLprinciplei
8、saddressedinSection7,basedontheuniversalcodingresults.
