Q-Learning Based Autonomous Control of the Auxiliary Power Network of a Ship

Research output: Contribution to journalArticleScientificpeer-review

115 Downloads (Pure)


We present a reinforcement learning (RL) model that is based on Q-learning for the autonomous control of ship auxiliary power networks. The development and application of the proposed model is demonstrated using a case-study ship as the platform. The auxiliary power network of the ship is represented as a Markov Decision Process (MDP). Q-learning is then used to teach an agent to operate in this MDP by choosing actions in each operating state which would minimize fuel consumption while also respecting the boundary conditions of the network. The presented work is based on an extensive data set received from one of the cruise-line operators on the Baltic Sea. This data set was preprocessed to extract information for the state representation of the auxiliary network, which was used for training and validating the model. As a result, it is shown that the developed method produces an autonomous control policy for the auxiliary power network that outperforms the current human operated manual control of the case-study ship. An average of 0.9 % fuel oil savings are attained over the analyzed round-trips with control that displayed similar robustness against blackouts as the current operation of the ship. This amounts to 32 tons of fuel oil saved annually. In addition, it is shown that the developed model can be reconfigured for different levels of robustness, depending on the preferred trade-off between maintained reserve power and fuel savings.
Original languageEnglish
Article number2169-3536
Pages (from-to)152879-152890
Number of pages11
JournalIEEE Access
Publication statusPublished - 30 Oct 2019
MoE publication typeA1 Journal article-refereed


  • Autonomous shipping
  • energy consumption reduction
  • Ferry
  • machinery
  • Q-learning
  • reinforcement learning
  • ship


Dive into the research topics of 'Q-Learning Based Autonomous Control of the Auxiliary Power Network of a Ship'. Together they form a unique fingerprint.
  • INTENS Tammi

    Ritari, A. & Pietola, M.


    Project: Business Finland: Other research funding

Cite this