Comprehensive Uncertainty Management in MDPs

Abstract

Multistage decision-making in robots involved in real-world tasks is a process affected by uncertainty. The effects of the agent’s actions in a physical en- vironment cannot be always predicted deterministically and in a precise manner. Moreover, observing the environment can be a too onerous for a robot, hence not continuos. Markov Decision Processes (MDPs) are a well-known solution inspired to the classic probabilistic approach for managing uncertainty. On the other hand, including fuzzy logics and possibility theory has widened uncertainty representa- tion. Probability, possibility, fuzzy logics, and epistemic belief allow treating dif- ferent and not always superimposable facets of uncertainty. This paper presents a new extended version of MDP, designed for managing all these kinds of uncertainty together to describe transitions between multi-valued fuzzy states. The motivation of this work is the design of robots that can be used to make decisions over time in an unpredictable environment. The model is described in detail along with its computational solution.