An Improved Neural Q-learning Approach for Dynamic Path Planning of Mobile Robots
Keywords:
Reinforcement learning, Q-learning, mobile robot, dynamic path planning.Abstract
Dynamic path planning is an important task for mobile robots in complex and uncertain environments. This paper proposes an improved neural Q-learning (INQL) approach for dynamic path planning of mobile robots. In the proposed INQL approach, the reward function is designed based on a bio-inspired neural network so that the rate of convergence of INQL is improved compared with the Q-learning algorithm. In addition, by combining the INQL algorithm with cubic B-spline curves, the robot can move to the target position successfully via a feasible planned path in different dynamic environments. Simulation and experimental results illustrate the superior of the INQL-based path planning method compared with existing popular path planning methods.
References
Watkins C?Q-Learning[J]?Machine Learning?1992?8(3)?279-292.
QIU Xue-na, LIU Shi-rong, SONG Jia-tao?Simon X. YANG. A Complete Coverage Path Planning Method for Mobile Robots in Uncertain Dynamic Environments[J].Robot. 2006,28(6):586-592.
Amit Konar et al. A Deterministic Improved Q-Learning for Path Planning of a Mobile Robot. IEEE Transactions on Systems, Man, and Cybernetics: Systems, Vol. 43, No. 5, 2013,1141-1153.
I. Goswami, P. K. Das, A. Konar, and R. Janarthanan,
Caihong Li, Ping Chen. Adaptive Behavior Design Based on FNN for the Mobile Robot. IEEE International Conference on Automation and Logistics. 2009. 1952-1956.
Velappa Ganapathy, Soh Chin Yun. Neural Q-Learning Controller for Mobile Robot. IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Singapore, 2009.
Zheng Qin and Jason Gu. Neural Q-learning in Motion Planning for Mobile Robot. Proceedings of the IEEE International Conference on Automation and Logistics, 2009.
Zenhua Huang. Structured Enhancement Learning based on Manifold and its Application to Mobile Robot [D]. phd thesis, National University of Defence Technology, 2012.
Zaobin Li. Reinforcement Learning in Autonomous Mobile Robot Navigation and Control [D]. phd thesis, National University of Defence Technology, 2010.
Bain M, Sammut C. A Framework for Behavioural Cloning[C], Machine Intelligence 15. 1995: 103-129.
Ng A Y, Russell S J. Algorithms for inverse reinforcement learning[C], Icml. 2000: 663-670.
Ng A Y, Harada D, Russell S. Policy invariance under reward transformations: Theory and application to reward shaping[C], ICML. 1999, 99: 278-287.
A. L. Hodgkin and A. F. Huxley. A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Phys. Lond., vol. 117, pp. 500
S. Grossberg,
Simon X. Yang, Max Meng. Neural Network Approaches to Dynamic Collision-Free Trajectory Generation. IEEE Transactions on Systems, Vol. 31, No. 3, pp.302-318, 2001.
S.m.LaYalle. Rapidly-exploring random trees: a new tool for path planning [R]. Technical Report TR98-11, Computer Science Dept, Iowa State University, 1998.
L.P. Kaelbling, M.L. Littman, et al. Reinforcement learning: A survey[J]. Journal of Artificial Intelligence Research, 1996, 4:237-285.
Qin, Zheng, J. Gu. Neural Q-learning in motion planning for mobile robot. IEEE International Conference on Automation and Logistics IEEE, 2009:1024 - 1028.
Zong, Ziliang, et al. "Elevator group control algorithm based on residual gradient and Q-learning." SICE 2004 Annual Conference IEEE, 2004:329-331 vol. 1.
Liu, Wei Peng, et al. "A dispatching algorithm for elevator group control system of recurrent neural networks based on q-learning." Journal of Hebei University of Technology , 2013.
Gaskett C. Q-Learning for Robot Control[J]. phd thesis, australian national university, 2002.Kuzmin V.
Connectionist Q-Learning in Robot Control Task[J]. Scientific Proceedings of Riga Technical University, 2002.
Wang Y, Silva C W D. Assess Team Q-Learning Algorithm in a Purely Cooperative Multi-Robot Task[C], ASME 2007 International Mechanical Engineering Congress and Exposition. 2007:627-633.
Wang Y, Silva C W D. A Modified Q-Learning Algorithm for Multi-Robot Decision Making[C], ASME 2007 International Mechanical Engineering Congress and Exposition. 2007:1275-1281.
Krishnan V, Lakshmivarahan S. Probability and Random Processes[J]. Journal of the Royal Statistical Society, 2001, 40(1):164-165.
Downloads
Published
How to Cite
Issue
Section
License
Authors who submit papers with this journal agree to the following terms.
