2 edition of Exploiting structure to efficiently solve large scale partially observable Markov decision processes. found in the catalog.
Exploiting structure to efficiently solve large scale partially observable Markov decision processes.
Written in English
Partially observable Markov decision processes (POMDPs) provide a natural and principled framework to model a wide range of sequential decision making problems under uncertainty. To date, the use of POMDPs in real-world problems has been limited by the poor scalability of existing solution algorithms, which can only solve problems with up to ten thousand states. In fact, the complexity of finding an optimal policy for a finite-horizon discrete POMDP is PSPACE-complete. In practice, two important sources of intractability plague most solution algorithms: Large policy spaces and large state spaces.In practice, it is critical to simultaneously mitigate the impact of complex policy representations and large state spaces. Hence, this thesis describes three approaches that combine techniques capable of dealing with each source of intractability: VDC with BPI, VDC with Perseus (a randomized point-based value iteration algorithm by Spaan and Vlassis ), and state abstraction with Perseus. The scalability of those approaches is demonstrated on two problems with more than 33 million states: synthetic network management and a real-world system designed to assist elderly persons with cognitive deficiencies to carry out simple daily tasks such as hand-washing. This represents an important step towards the deployment of POMDP techniques in ever larger, real-world, sequential decision making problems.On the other hand, for many real-world POMDPs it is possible to define effective policies with simple rules of thumb. This suggests that we may be able to find small policies that are near optimal. This thesis first presents a Bounded Policy Iteration (BPI) algorithm to robustly find a good policy represented by a small finite state controller. Real-world POMDPs also tend to exhibit structural properties that can be exploited to mitigate the effect of large state spaces. To that effect, a value-directed compression (VDC) technique is also presented to reduce POMDP models to lower dimensional representations.
|The Physical Object|
|Number of Pages||144|
Blast furnace blowing engines
United States maverick tokens
1662 Royal Brevier Prayer Book with Hymns Ancient and Modern Red imitation leather, 4920 2482
Small unit action in Vietnam, summer 1966
Dialects in decline
A collection of poetical essays
Old English mansions
Optimum and limiting hydrogen-ion concentrations for B. botulinus and quantitative estimation of its growth
Recent theistic discussion
Dissertations on the subjects and mode of Gospel baptism
future of the Indian past
Abraham Lincoln, 1809-1858.