Reinforcement Learning Methods for Setpoint Optimization and Control Method Design in Process Industry with Case Studies in Steel Strip Rolling and District Heating

Process industry necessitates precise control and monitoring for operational efficiency, safety, and productivity. Traditional approaches, such as first-principles models, empirical models, and trial-and-error methods, have been utilized, often involving simplification and linearization to address the intricate and dynamic nature of industrial processes. However, to enhance product quality and energy efficiency, there is a growing demand for intelligent and adaptive methodologies to compute optimal solutions for industrial processes. One significant challenge lies in the realm of setpoint optimization, where precise computation of equipment parameters to align with quality specifications is paramount. In the domain of process control, achieving high-quality products relies on the implementation of feedback control methods. However, devising adaptive control methodologies capable of dynamically responding to evolving conditions poses a substantial challenge.

Recognizing the potential of reinforcement learning (RL) to learn from interactions, RL techniques have been adopted to learn policies for setpoint optimization and process control. In the context of setpoint optimization in strip rolling and fuel cost reduction in district heating, RL methodologies have been investigated to calculate and optimize setpoints for the systems. Leveraging environment models of the processes, RL agents generate optimal solutions based on machine capacity to meet customer demands. Furthermore, RL-based adaptive control methodologies have been developed for the steel strip rolling process, enabling dynamic responses to evolving conditions. To make the RL-based control policy more accurate and practical for industrial processes, an offline RL method that learns control policies directly from the data has been proposed to address biases originating from approximated environment models that impact the accuracy.

Steel strip rolling and district heating have been selected to evaluate the efficacy of RL-based methods in addressing setpoint optimization and process control challenges. The results indicate that the proposed methods outperform the traditional approaches, marking substantial advancements in automation, optimization, and control methodologies within the process industry.