Optimal operation of reverse osmosis desalination process with deep reinforcement learning methods

Abstract

The reverse osmosis (RO) process is a well-established desalination technology, wherein energy-efficient techniques and advanced process control methods significantly reduce production costs. This study proposes an optimal real-time management method to minimize the total daily operation cost of an RO desalination plant, integrating a storage tank system to meet varying daily freshwater demand. Utilizing the dynamic model of the RO process, a cascade structure with two reinforcement learning (RL) agents, namely the deep deterministic policy gradient (DDPG) and deep Q-Network (DQN), is developed to optimize the operation of the RO plant. The DDPG agent, manipulating the high-pressure pump, controls the permeate flow rate to track a reference setpoint value. Simultaneously, the DQN agent selects the optimal setpoint value and communicates it to the DDPG controller to minimize the plant’s operation cost. Monitoring storage tanks, permeate flow rates, and water demand enables the DQN agent to determine the required amount of permeate water, optimizing water quality and energy consumption. Additionally, the DQN agent monitors the storage tank’s water level to prevent overflow or underflow of permeate water. Simulation results demonstrate the effectiveness and practicality of the designed RL agents.