A Comparative Study of Reinforcement Learning and Metaheuristic Algorithms for Real-Time Last-Mile Delivery Scheduling

Abstract

The exponential growth of e-commerce has intensified the need for efficient real-time delivery-scheduling algorithms that can handle dynamic urban logistics environments. This study presents a comprehensive comparative analysis of reinforcement learning and metaheuristic algorithms for last-mile delivery optimization under time-sensitive constraints. We implement and evaluate Adaptive Large Neighborhood Search (ALNS) and Deep Q-Network (DQN) approaches on benchmark instances representing realistic urban delivery scenarios with 50-200 customer nodes. The experimental results demonstrate that ALNS achieves superior solution quality with an average optimality gap of 3.2% under an approximately 5-second (≤6 s) operational time budget, while DQN exhibits better runtime scalability for networks exceeding 150 customers (in the stress-test setting). The sensitivity analysis reveals critical trade-offs between computational efficiency and solution robustness under varying traffic conditions. This research provides empirical guidelines for logistics practitioners to select appropriate algorithms based on operational constraints, thereby contributing to sustainable urban transportation systems.