Reinforcement Learning: Optimal Feedback Control with Industrial Applications

by Jinna Li, Frank L. Lewis, Jialu Fan

Overview

This book offers a thorough introduction to the basics and scientific and technological innovations involved in the modern study of reinforcement-learning-based feedback control. The authors address a wide variety of systems including work on nonlinear, networked, multi-agent and multi-player systems.

A concise description of classical reinforcement learning (RL), the basics of optimal control with dynamic programming and network control architectures, and a brief introduction to typical algorithms build the foundation for the remainder of the book. Extensive research on data-driven robust control for nonlinear systems with unknown dynamics and multi-player systems follows. Data-driven optimal control of networked single- and multi-player systems leads readers into the development of novel RL algorithms with increased learning efficiency. The book concludes with a treatment of how these RL algorithms can achieve optimal synchronization policies for multi-agentsystems with unknown model parameters and how game RL can solve problems of optimal operation in various process industries. Illustrative numerical examples and complex process control applications emphasize the realistic usefulness of the algorithms discussed.

The combination of practical algorithms, theoretical analysis and comprehensive examples presented in Reinforcement Learning will interest researchers and practitioners studying or using optimal and adaptive control, machine learning, artificial intelligence, and operations research, whether advancing the theory or applying it in mineral-process, chemical-process, power-supply or other industries.

Product Details

ISBN-13:	9783031283932
Publisher:	Springer International Publishing
Publication date:	07/25/2023
Series:	Advances in Industrial Control
Edition description:	2023
Pages:	310
Product dimensions:	6.10(w) x 9.25(h) x (d)

About the Author

Professor Jinna Li received the M.S. degree and the Ph. D. degree from Northeastern University, Shenyang, China, 2006 and 2009, respectively. She is an associate professor at Shenyang University of Chemical Technology, Shenyang, China. From April 2009 to April 2011, she carried out postdoctoral research at the Lab of Industrial Control Networks and Systems, Shenyang Institute of Automation, Chinese Academy of Sciences. From June 2014 to June 2015, she was a Visiting Scholar granted by China Scholarship Council with Energy Research Institute, Nanyang Technological University, Singapore. From September 2015 to June 2016, she was a Domestic Young Core Visiting Scholar granted by Ministry of Education of China with State Key Lab of Synthetical Automation for Process Industries, Northeastern University. From Jan. 2017 to Jul. 2017, she was a Visiting Scholar with the School of Electrical and Electronic Engineering, the University of Manchester, UK. Her current research interests include neural networks, reinforcement learning, optimal operational control, distributed optimization control and data-based control. She has authored two P.R.China patents, more than 40 journal papers, more than 20 conference papers, one books. Dr. Li is a Senior Fellow of the Institute of Electrical and Electronic Engineers. She presided 3 projects from the National Natural Science Foundation of China and 5 projects from provincial funding in P. R. China.

Frank L. Lewis is a Distinguished Scholar Professor and Moncrief-O’Donnell Chair at University of Texas at Arlington’s Automation & Robotics Research Institute. He obtained his Bachelor’s Degree in Physics/EE and MSEE at Rice University, his MS in Aeronautical Engineering from Univ. W. Florida, and his Ph.D. at Ga. Tech. He received the Fulbright Research Award, the Outstanding Service Award from Dallas IEEE Section, and was selected as Engineer of the year by Ft. Worth IEEE Section. He is an elected Guest Consulting Professor at South China University of Technology and Shanghai Jiao Tong University. He is a Fellow of the IEEE, Fellow of IFAC, Fellow of the U.K. Institute of Measurement & Control, and a U.K. Chartered Engineer. His current research interests include distributed control on graphs, neural and fuzzy systems, and intelligent control.

Associate Professor Jialu Fan, graduated from Northeastern University with a bachelor's degree in automation in 2006; in 2011, she graduated from Zhejiang University with a Ph.D. in engineering; from 2009 to 2010, she was sponsored by the China Scholarship Council as a visiting researcher at Pennsylvania State University. Her research interests include networked operation control, industrial wireless networks and mobile social networks. She is co-author of Data Dissemination and Query in Mobile Social Networks (ISBN 978-1-4614-2253-2, Springer) and (at the invitation of Professor Pedro Albertos, it’s second author) of a Chinese translation of Feedback and Control for Everyone (Original English ISBN 978-3-642-03445-9, Springer). She has published more than 30 academic papers in the field of network communication in various IEEE Journals and in conferences in the field of control. Associate Professor Fan has been responsible for two National Natural Science Foundation key projects along with a National Natural Science Foundation youth project and 3 provincial and ministerial-level projects. She particapted as a key member, in the National 973 Program, 863 Key Program and National Natural Science Foundation of China. She has served as a reviewer for top international SCI journals such as the IEEE Transactions on Automatic Control, IEEE Transactions on Control Systems Technology and IEEE Network Magazine. She served as Secretary-General of World Congress on Intelligent Control and Automation in 2014, as Branch Chair of the Asian Control Conference 2015, and on the program committees of editions of the Indian Control Conference, Globecom and SmartGridComm. He is now a member of the Youth Working Committee of the Chinese Society of Automation and a member of the International Association of Electrical and Electronics Engineers. She has won numerous honors as an outstanding educator, supervisor and mentor.

Table of Contents

1. Background on Reinforcement Learning and Optimal Control.- 2. H-infinity Control Using Reinforcement Learning.- 3. Robust Tracking Control and Output Regulation.- 4. Interleaved Robust Reinforcement Learning.- 5. Optimal Networked Controller and Observer Design.- 6. Interleaved Q-Learning.- 7. Off-Policy Game Reinforcement Learning.- 8. Game Reinforcement Learning for Process Industries.