Double Deep Q Network Optimized Linear Active Disturbance Rejection Control for Ship Course Keeping

Yuemin Zheng; Zhongxin Liu; Jin Tao; Qinglin Sun; Hao Sun; Mingwei Sun; Zengqiang Chen

doi:10.1007/978-981-16-6328-4_29

Double Deep Q Network Optimized Linear Active Disturbance Rejection Control for Ship Course Keeping

Yuemin Zheng, Zhongxin Liu, Jin Tao^*, Qinglin Sun, Hao Sun, Mingwei Sun, Zengqiang Chen

^*Corresponding author for this work

Nankai University

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

Abstract

This paper mainly proposes a parameter-optimized linear active disturbance rejection controller (LADRC) based on a double deep Q network (DDQN) and applies it to ship course control. Firstly, based on the separate mathematical models’ equation, a ship’s dynamic model is established. Then, a LADRC based course keeping controller is designed to overcome the ship’s environmental disturbances and internal uncertainty during navigation. Furthermore, to facilitate LADRC parameter adjustment and obtain a better performance of ship course keeping control, the DDQN is applied to tune the adaptive parameters of LADRC. Finally, simulation results and comparisons on ship course keeping show that the proposed DDQN optimized LADRC can control the ship’s heading angle to track the planned course, and the control performance outperforms the traditional LADRC.

Original language	English
Title of host publication	Proceedings of 2021 Chinese Intelligent Systems Conference
Editors	Yingmin Jia, Weicun Zhang, Yongling Fu, Zhiyuan Yu, Song Zheng
Publisher	Springer
Pages	259-274
Number of pages	16
Volume	1
ISBN (Electronic)	978-981-16-6328-4
ISBN (Print)	978-981-16-6327-7
DOIs	https://doi.org/10.1007/978-981-16-6328-4_29
Publication status	Published - 2022
MoE publication type	A4 Conference publication
Event	Chinese Intelligent Systems Conference - Fuzhou, China Duration: 16 Oct 2021 → 17 Oct 2021 Conference number: 17

Publication series

Name	Lecture Notes in Electrical Engineering
Volume	803 LNEE
ISSN (Print)	1876-1100
ISSN (Electronic)	1876-1119

Conference

Conference	Chinese Intelligent Systems Conference
Abbreviated title	CISC
Country/Territory	China
City	Fuzhou
Period	16/10/2021 → 17/10/2021

Keywords

Double deep Q network
Linear active disturbance rejection control
Parameter optimization
Reinforcement learning
Ship course keeping control

Access to Document

10.1007/978-981-16-6328-4_29

Cite this

Zheng, Y., Liu, Z., Tao, J., Sun, Q., Sun, H., Sun, M., & Chen, Z. (2022). Double Deep Q Network Optimized Linear Active Disturbance Rejection Control for Ship Course Keeping. In Y. Jia, W. Zhang, Y. Fu, Z. Yu, & S. Zheng (Eds.), Proceedings of 2021 Chinese Intelligent Systems Conference (Vol. 1, pp. 259-274). (Lecture Notes in Electrical Engineering; Vol. 803 LNEE). Springer. https://doi.org/10.1007/978-981-16-6328-4_29

@inproceedings{dc8891a217904e55ad4f78dccb388d07,

title = "Double Deep Q Network Optimized Linear Active Disturbance Rejection Control for Ship Course Keeping",

abstract = "This paper mainly proposes a parameter-optimized linear active disturbance rejection controller (LADRC) based on a double deep Q network (DDQN) and applies it to ship course control. Firstly, based on the separate mathematical models{\textquoteright} equation, a ship{\textquoteright}s dynamic model is established. Then, a LADRC based course keeping controller is designed to overcome the ship{\textquoteright}s environmental disturbances and internal uncertainty during navigation. Furthermore, to facilitate LADRC parameter adjustment and obtain a better performance of ship course keeping control, the DDQN is applied to tune the adaptive parameters of LADRC. Finally, simulation results and comparisons on ship course keeping show that the proposed DDQN optimized LADRC can control the ship{\textquoteright}s heading angle to track the planned course, and the control performance outperforms the traditional LADRC.",

keywords = "Double deep Q network, Linear active disturbance rejection control, Parameter optimization, Reinforcement learning, Ship course keeping control",

author = "Yuemin Zheng and Zhongxin Liu and Jin Tao and Qinglin Sun and Hao Sun and Mingwei Sun and Zengqiang Chen",

note = "Funding Information: This work was supported by the National Natural Science Foundation of China (Grant Nos. 61973172, 61973175, 62003175 and 62003177), the National Key Research and Development Project (Grant No. 2019YFC1510900), the key Technologies Research and Development Program of Tianjin (Grant No. 19JCZDJC32800), this project also funded by China Postdoctoral Science Foundation (Grant No. 2020M670633) and Academy of Finland (Grant No. 315660). Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.; Chinese Intelligent Systems Conference, CISC ; Conference date: 16-10-2021 Through 17-10-2021",

year = "2022",

doi = "10.1007/978-981-16-6328-4_29",

language = "English",

isbn = "978-981-16-6327-7",

volume = "1",

series = "Lecture Notes in Electrical Engineering",

publisher = "Springer",

pages = "259--274",

editor = "Yingmin Jia and Weicun Zhang and Yongling Fu and Zhiyuan Yu and Song Zheng",

booktitle = "Proceedings of 2021 Chinese Intelligent Systems Conference",

address = "Germany",

}

Zheng, Y, Liu, Z, Tao, J, Sun, Q, Sun, H, Sun, M & Chen, Z 2022, Double Deep Q Network Optimized Linear Active Disturbance Rejection Control for Ship Course Keeping. in Y Jia, W Zhang, Y Fu, Z Yu & S Zheng (eds), Proceedings of 2021 Chinese Intelligent Systems Conference. vol. 1, Lecture Notes in Electrical Engineering, vol. 803 LNEE, Springer, pp. 259-274, Chinese Intelligent Systems Conference, Fuzhou, China, 16/10/2021. https://doi.org/10.1007/978-981-16-6328-4_29

Double Deep Q Network Optimized Linear Active Disturbance Rejection Control for Ship Course Keeping. / Zheng, Yuemin; Liu, Zhongxin; Tao, Jin et al.
Proceedings of 2021 Chinese Intelligent Systems Conference. ed. / Yingmin Jia; Weicun Zhang; Yongling Fu; Zhiyuan Yu; Song Zheng. Vol. 1 Springer, 2022. p. 259-274 (Lecture Notes in Electrical Engineering; Vol. 803 LNEE).

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Double Deep Q Network Optimized Linear Active Disturbance Rejection Control for Ship Course Keeping

AU - Zheng, Yuemin

AU - Liu, Zhongxin

AU - Tao, Jin

AU - Sun, Qinglin

AU - Sun, Hao

AU - Sun, Mingwei

AU - Chen, Zengqiang

N1 - Conference code: 17

PY - 2022

Y1 - 2022

N2 - This paper mainly proposes a parameter-optimized linear active disturbance rejection controller (LADRC) based on a double deep Q network (DDQN) and applies it to ship course control. Firstly, based on the separate mathematical models’ equation, a ship’s dynamic model is established. Then, a LADRC based course keeping controller is designed to overcome the ship’s environmental disturbances and internal uncertainty during navigation. Furthermore, to facilitate LADRC parameter adjustment and obtain a better performance of ship course keeping control, the DDQN is applied to tune the adaptive parameters of LADRC. Finally, simulation results and comparisons on ship course keeping show that the proposed DDQN optimized LADRC can control the ship’s heading angle to track the planned course, and the control performance outperforms the traditional LADRC.

AB - This paper mainly proposes a parameter-optimized linear active disturbance rejection controller (LADRC) based on a double deep Q network (DDQN) and applies it to ship course control. Firstly, based on the separate mathematical models’ equation, a ship’s dynamic model is established. Then, a LADRC based course keeping controller is designed to overcome the ship’s environmental disturbances and internal uncertainty during navigation. Furthermore, to facilitate LADRC parameter adjustment and obtain a better performance of ship course keeping control, the DDQN is applied to tune the adaptive parameters of LADRC. Finally, simulation results and comparisons on ship course keeping show that the proposed DDQN optimized LADRC can control the ship’s heading angle to track the planned course, and the control performance outperforms the traditional LADRC.

KW - Double deep Q network

KW - Linear active disturbance rejection control

KW - Parameter optimization

KW - Reinforcement learning

KW - Ship course keeping control

UR - http://www.scopus.com/inward/record.url?scp=85117918340&partnerID=8YFLogxK

U2 - 10.1007/978-981-16-6328-4_29

DO - 10.1007/978-981-16-6328-4_29

M3 - Conference article in proceedings

AN - SCOPUS:85117918340

SN - 978-981-16-6327-7

VL - 1

T3 - Lecture Notes in Electrical Engineering

SP - 259

EP - 274

BT - Proceedings of 2021 Chinese Intelligent Systems Conference

A2 - Jia, Yingmin

A2 - Zhang, Weicun

A2 - Fu, Yongling

A2 - Yu, Zhiyuan

A2 - Zheng, Song

PB - Springer

T2 - Chinese Intelligent Systems Conference

Y2 - 16 October 2021 through 17 October 2021

ER -

Zheng Y, Liu Z, Tao J, Sun Q, Sun H, Sun M et al. Double Deep Q Network Optimized Linear Active Disturbance Rejection Control for Ship Course Keeping. In Jia Y, Zhang W, Fu Y, Yu Z, Zheng S, editors, Proceedings of 2021 Chinese Intelligent Systems Conference. Vol. 1. Springer. 2022. p. 259-274. (Lecture Notes in Electrical Engineering). Epub 2021 Oct 7. doi: 10.1007/978-981-16-6328-4_29

Double Deep Q Network Optimized Linear Active Disturbance Rejection Control for Ship Course Keeping

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Three-dimensional Acoustic Manipulation of Multiple Micro-objects

Cite this

Double Deep Q Network Optimized Linear Active Disturbance Rejection Control for Ship Course Keeping

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Projects

Three-dimensional Acoustic Manipulation of Multiple Micro-objects

Cite this