Regular Paper

International Journal of Control, Automation and Systems 2023; 21(4): 1319-1330

Published online March 3, 2023

https://doi.org/10.1007/s12555-021-0674-z

© The International Journal of Control, Automation, and Systems

Robust Near-optimal Control for Constrained Nonlinear System via Integral Reinforcement Learning

Yu-Qing Qiu, Yan Li, and Zhong Wang*

Northwestern Polytechnical University

Abstract

This paper proposes a robust near-optimal control algorithm for uncertain nonlinear systems with state constraints and input saturation. By incorporating a barrier function and a non-quadratic term, the robust stabilization problem with constraints and uncertainties is converted into an unconstrained optimal control problem of the nominal system, which requires the solution of the Hamilton-Jacobi-Bellman (HJB) equation. The proposed integral reinforcement learning (IRL)-based method can obtain the approximate solution of the HJB equation without requiring any knowledge of system drift dynamics. An online gain-adjustable update law of the actor-critic architecture is developed to relax the persistence of excitation (PE) condition and ensure the closed-loop system stability throughout learning. The uniform ultimate boundedness of the closed-loop system is verified using Lyapunov’s direct method. Simulation results demonstrate the effectiveness and feasibility of the proposed method.

Keywords Constrained nonlinear system, integral reinforcement learning, optimal control, robust control.

Article

Regular Paper

International Journal of Control, Automation and Systems 2023; 21(4): 1319-1330

Published online April 1, 2023 https://doi.org/10.1007/s12555-021-0674-z

Copyright © The International Journal of Control, Automation, and Systems.

Robust Near-optimal Control for Constrained Nonlinear System via Integral Reinforcement Learning

Yu-Qing Qiu, Yan Li, and Zhong Wang*

Northwestern Polytechnical University

Abstract

This paper proposes a robust near-optimal control algorithm for uncertain nonlinear systems with state constraints and input saturation. By incorporating a barrier function and a non-quadratic term, the robust stabilization problem with constraints and uncertainties is converted into an unconstrained optimal control problem of the nominal system, which requires the solution of the Hamilton-Jacobi-Bellman (HJB) equation. The proposed integral reinforcement learning (IRL)-based method can obtain the approximate solution of the HJB equation without requiring any knowledge of system drift dynamics. An online gain-adjustable update law of the actor-critic architecture is developed to relax the persistence of excitation (PE) condition and ensure the closed-loop system stability throughout learning. The uniform ultimate boundedness of the closed-loop system is verified using Lyapunov’s direct method. Simulation results demonstrate the effectiveness and feasibility of the proposed method.

Keywords: Constrained nonlinear system, integral reinforcement learning, optimal control, robust control.

IJCAS
May 2024

Vol. 22, No. 5, pp. 1461~1759

Stats or Metrics

Share this article on

  • line

Related articles in IJCAS

IJCAS

eISSN 2005-4092
pISSN 1598-6446