资源描述：
《Basics of Applied Stochastic Processes_Serfozo》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、Chapter1MarkovChainsAsequenceofrandomvariablesX0,X1,...withvaluesinacountablesetSisaMarkovchainifatanytimen,thefuturestates(orvalues)Xn+1,Xn+2,...dependonthehistoryX0,...,XnonlythroughthepresentstateXn.Markovchainsarefundamentalstochasticprocessesthathavemanydiverseapplica-tions.Thisisbecause
2、aMarkovchainrepresentsanydynamicalsystemwhosestatessatisfytherecursionXn=f(Xn−1,Yn),n≥1,whereY1,Y2...areindependentandidenticallydistributed(i.i.d.)andfisadeterministicfunc-tion.Thatis,thenewstateXnissimplyafunctionofthelaststateandanauxiliaryrandomvariable.Suchsystemdynamicsaretypicalofthose
3、forqueuelengthsincallcenters,stressesonmaterials,waitingtimesinproduc-tionandservicefacilities,inventoriesinsupplychains,parallel-processingsoftware,waterlevelsindams,insurancefunds,stockprices,etc.ThischapterbeginsbydescribingthebasicstructureofaMarkovchainandhowitssingle-steptransitionproba
4、bilitiesdetermineitsevolution.Forin-stance,whatistheprobabilityofreachingacertainstate,andhowlongdoesittaketoreachit?ThenextandmainpartofthechaptercharacterizesthestationaryorequilibriumdistributionofMarkovchains.Thesedistributionsarethebasisoflimitingaveragesofvariouscostandperformanceparam-
5、etersassociatedwithMarkovchains.Considerablediscussionisdevotedtobranchingphenomena,stochasticnetworks,andtime-reversiblechains.In-cludedareexamplesofMarkovchainsthatrepresentqueueing,productionsystems,inventorycontrol,reliability,andMonteCarlosimulations.Beforegettingintothemaintext,areaderw
6、ouldbenefitbyabriefreviewofconditionalprobabilitiesinSection1.22ofthischapterandrelatedmaterialonrandomvariablesanddistributionsinSections1–4intheAppendix.TherestoftheAppendix,whichprovidesmorebackgroundonprobability,wouldbeappropriateforlaterreading.R.Serfozo,BasicsofAppliedStochasticProcesse
7、s,1ProbabilityanditsApplications.cSpringer-VerlagBerlinHeidelberg200921MarkovChains1.1IntroductionThissectionintroducesMarkovchainsanddescribesafewexamples.Adiscrete-timestochasticprocess{Xn:n≥0}onacountablesetSisacollectionofS-valuedrandomv
