POMDP_lecture

《POMDP_lecture》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、PartiallyObservableMarkovDecisionProcesses(POMDPs)GeoffHollingerGraduateArtificialIntelligenceFall,2007*SomemediafromReidSimmons,TreySmith,TonyCassandra,MichaelLittman,andLeslieKaelbling1OutlineforPOMDPLecture¢IntroductionßWhatisaPOMDPanyway?ßAsimpleexa

2、mple¢SolvingPOMDPsßExactvalueiterationßPolicyiterationßWitnessalgorithm,HSVIßGreedysolutions¢ApplicationsandextensionsßWhenamIevergoingtousethis(otherthaninhomeworkfive)?2SowhoisthisMarkovguy?¢AndreyAndreyevichMarkov(1856-1922)¢Russianmathematician¢Know

3、nforhisworkinstochasticprocessesßLaterknownasMarkovChains3WhatisaMarkovChain?¢Finitenumberofdiscretestates¢Probabilistictransitionsbetweenstates¢NextstatedeterminedonlyRewards:S1=10,S2=0bythecurrentstateßThisistheMarkovproperty4WhatisaHiddenMarkovModel?

4、¢Finitenumberofdiscretestates¢Probabilistictransitionsbetweenstates¢NextstatedeterminedonlybyRewards:S1=10,S2=0thecurrentstate¢We’reunsurewhichDonotknowstate:statewe’reinS1emitsO1withprob0.75ßThecurrentstatesS2emitsO2withprob0.75emitsanobservation5Whati

5、saMarkovDecisionProcess?¢Finitenumberofdiscretestates¢Probabilistictransitionsbetweenstatesandcontrollableactionsineachstate¢NextstatedeterminedonlybyRewards:S1=10,S2=0thecurrentstateandcurrentactionßThisisstilltheMarkovproperty6WhatisaPartiallyObservab

6、leMarkovDecisionProcess?¢Finitenumberofdiscretestates¢Probabilistictransitionsbetweenstatesandcontrollableactions¢NextstatedeterminedonlybythecurrentstateandcurrentactionRewards:S1=10,S2=0¢We’reunsurewhichstatewe’reinDonotknowstate:S1emitsO1withprob0.75

7、ßThecurrentstateemitsobservationsS2emitsO2withprob0.757AVeryHelpfulChart8POMDPversusMDP¢MDPß+Tractabletosolveß+Relativelyeasytospecifyß-Assumesperfectknowledgeofstate¢POMDPß+Treatsallsourcesofuncertaintyuniformlyß+Allowsforinformationgatheringactionsß-H

8、ugelyintractabletosolveoptimally9SimpleExample¢Initialdistribution:[0.1,0.9]¢Discountfactor:0.5¢Reward:S1=10,S2=0¢Observations:S1emitsO1withprob1.0,S2emitsO2withprob1.010SimpleExample¢Initialdistribution:[0.9,0.1]¢Discountfactor: