Navigation with memory in a partially observable environment

The paper presents an architecture that allows the reactive visual navigation via an unsupervised reinforcement learning. This objective is reached using Q-learning and a hierarchical approach to the developed architecture. Using these techniques requires a deviation from the Partially Observable Markov Decision Processes (POMDP) and some innovations: heuristic techniques for generalizing the experience and for treating the partial observability; a technique for the speed adjournment of the Q function; the definition of a special reinforcement policy adequate for learning a complex task without supervision. The result is a satisfactory learning of the navigation assignment in a simulated environment. © 2005 Elsevier B.V. All rights reserved.