Problem-Specific Knowledge Based Multi-Objective Meta-Heuristics Combined Q-Learning for Scheduling Urban Traffic Lights With Carbon Emissions

Abstract

In complex and variable traffic environments, efficient multi-objective urban traffic light scheduling is imperative. However, the carbon emission problem accompanying traffic delays is often neglected in most existing literature. This study focuses on multi-objective urban traffic light scheduling problems (MOUTLSP), concerning traffic delays and carbon emissions simultaneously. First, a multi-objective mathematical model is firstly developed to describe MOUTLSP to minimize vehicle delays, pedestrian delays, and carbon emissions. Second, three well-known meta-heuristics, namely genetic algorithm (GA), particle swarm optimization (PSO), and differential evolution (DE), are improved to solve MOUTLSP. Six problem-feature-based local search operators (LSO) are designed based on the solution structure and incorporated into the iterative process of meta-heuristics. Third, the problem nature is utilized to design two novel Q-learning-based strategies for algorithm and LSO selection, respectively. The Q-learning-based algorithm selection (QAS) strategy guides non-dominated solutions to obtain a good trade-off among three objectives and generates high-quality solutions by selecting suitable algorithms. The Q-learning-based local search selection (QLSS) strategies are employed to seek premium neighborhood solutions throughout the iterative process for improving the convergence speed. The effectiveness of the improvement strategies is verified by solving 11 instances with different scales. The proposed algorithms with Q-learning-based strategies are compared with two classical multi-objective algorithms and some state-of-the-art algorithms for solving urban traffic light scheduling problems. The experimental results and comparisons demonstrate that the proposed GA<inline-formula> <tex-math notation="LaTeX">$+$</tex-math> </inline-formula>QLSS, a variant of GA, is the most competitive one. This research proposes new ideas for urban traffic light scheduling with three objectives by Q-learning assisted evolutionary algorithms firstly. It provides strong support for achieving more efficient and environmentally friendly urban traffic management.