LPG-SAC-TCO: A Large Language Model--Policy-Guided Soft Actor-Critic Framework for Total Cost of Ownership--Optimal Energy Management of Hydrogen Fuel Cell Heavy-Duty Trucks

Zonghui Hua; Yue Luo; Yunting Ling

doi:10.71222/3t13x215

Authors

Zonghui Hua Chongqing University, Chongqing, China Author
Yue Luo University of Southern California, Los Angeles, USA Author
Yunting Ling University of California, San Diego, La Jolla, USA Author

DOI:

https://doi.org/10.71222/3t13x215

Keywords:

Hydrogen fuel cell truck, Energy management, Total cost of ownership, Deep reinforcement learning, Soft Actor-Critic, Large language model, Financialized control

Abstract

Hydrogen fuel cell heavy-duty trucks (HFCTs) are regarded as a promising solution for decarbonizing long-haul freight transportation. However, existing energy management strategies primarily focus on short-term energy efficiency and neglect long-term economic impacts, such as component degradation, hydrogen price volatility, and asset depreciation. This study reformulates energy management as a financialized long-term optimization problem and proposes LPG-SAC-TCO, a Large Language Model (LLM)--Policy-Guided Soft Actor-Critic framework for total cost of ownership (TCO)--optimal control. The proposed framework integrates a physics-based powertrain model, a full lifecycle TCO model, and a deep reinforcement learning controller. An LLM is introduced as a high-level policy advisor to extract semantic and financial insights from complex operating conditions, dynamically generating cost-priority signals that guide the downstream Soft Actor-Critic (SAC) controller. Unlike conventional reinforcement learning approaches that minimize fuel consumption alone, the proposed method directly minimizes the incremental TCO, incorporating hydrogen consumption cost, battery aging cost, fuel cell degradation cost, and depreciation loss. Simulation experiments are conducted under real-world driving cycles, stochastic hydrogen price scenarios, and varying payload conditions. Experimental results demonstrate that LPG-SAC-TCO reduces lifecycle total cost of ownership by approximately 19% compared with rule-based energy management and by about 11% relative to energy-oriented SAC controllers, while significantly mitigating battery cycling stress and fuel cell power degradation. Long-horizon simulations further show that the proposed framework extends component lifetime and yields more stable cost trajectories under stochastic hydrogen price fluctuations. These results confirm the effectiveness and robustness of LPG-SAC-TCO as a practical decision-support tool for fleet operators, leasing companies, and large-scale logistics enterprises.

References

1. P. Rodatz, G. Paganelli, A. Sciarretta, et al., "Optimal power management of an experimental fuel cell/supercapacitor-powered hybrid vehicle," Control Engineering Practice, vol. 13, no. 1, pp. 41–53, 2005.

2. A. Sciarretta and L. Guzzella, "Control of hybrid electric vehicles," IEEE Control Systems Magazine, vol. 27, no. 2, pp. 60–70, 2007.

3. Y. Gao, L. Chen, and M. Ehsani, "Investigation of the effectiveness of regenerative braking for EV and HEV," in Future Transportation Technology Conference & Exposition, SAE Technical Paper, 1999.

4. L. Nan, F. Xu, P. C. Sui, et al., "Techno-economic analysis of fuel cell trucks with different powertrain hybridization and hydrogen resources by 2040: case study of China," International Journal of Hydrogen Energy, vol. 132, pp. 174–182, 2025.

5. U. R. Sontakke, S. Jaju, and D. K. Mahajan, "Total cost of ownership analysis of fuel cell electric vehicles in India," in Towards Hydrogen Infrastructure, Elsevier, 2024, pp. 379–400.

6. B. Yu, X. Jia, T. Han, et al., "Modeling and degradation-aware energy management strategy for fuel cell vehicle considering cathode catalyst layer status," Journal of Power Sources, vol. 668, p. 239228, 2026.

7. Y. Li and H. He, Deep Reinforcement Learning-Based Energy Management for Hybrid Electric Vehicles. Morgan & Claypool Publishers, 2022.

8. R. Xiong, J. Cao, and Q. Yu, "Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle," Applied Energy, vol. 211, pp. 538–548, 2018.

9. T. Haarnoja, A. Zhou, P. Abbeel, et al., "Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor," in International Conference on Machine Learning, PMLR, 2018, pp. 1861–1870.

10. S. Yao, J. Zhao, D. Yu, et al., "React: Synergizing reasoning and acting in language models," arXiv preprint arXiv:2210.03629, 2022.

11. Q. Nie and T. Liu, "Large language models: Tools for new environmental decision-making," Journal of Environmental Management, vol. 375, p. 124373, 2025.

12. Y. Huang, L. J. Wan, H. Ye, et al., "New solutions on LLM acceleration, optimization, and application," in Proceedings of the 61st ACM/IEEE Design Automation Conference, 2024, pp. 1–4.

13. Z. Ning, H. Zeng, and Z. Tian, "Research on data-driven energy efficiency optimisation algorithm for air compressors," in *Third International Conference on Advanced Materials and Equipment Manufacturing (AMEM 2024)*, SPIE, vol. 13691, pp. 1068–1075, 2025.

14. S. Xu, L. Jiang, and B. Gu, "Design and Validation of a Smart Neuromorphic System Architecture for Algorithmic Trading," in *Proceedings of the 2nd International Symposium on Integrated Circuit Design and Integrated Systems*, 2025, pp. 127–136.

15. P. Fan, H. Li, and M. Hu, "Profit-Oriented Production and Pricing Optimization for Manufacturing Enterprises Using Proximal Policy Optimization," Economics and Management Innovation, vol. 3, no. 2, pp. 8–17, 2026.