Research on Cloud Computing Resource Scheduling Strategy Based on Big Data and Machine Learning

Abstract

Optimizing the efficient scheduling of cloud platforms is crucial for enhancing the performance and resource utilization of these platforms. The intelligent scheduling optimization framework proposed in this article combines big data and deep learning technologies, which can effectively solve a series of severe challenges in cloud platform resource scheduling, such as redundant resource preparation, inaccurate capacity estimation, and high scheduling costs. Suggest building a multidimensional feature model that combines historical and real-time monitoring information, and using LSTM for accurate resource prediction. Based on this result, a reinforcement learning based automatic adjustment hybrid scheduling algorithm was designed to achieve intelligent dynamic scheduling of different resources. Meanwhile, a graph partitioning mechanism is adopted to reduce scheduling complexity and improve system scalability. On the Google Cluster Trace dataset, the proposed solution can significantly improve resource utilization, increasing it from raw utilization to 21.5%, reducing the average waiting time of tasks by 18.3%, and lowering SLA default rates by 13%. The deployment experiment in the Kubernetes environment further validated the feasibility and effectiveness of the proposed solution. The research results provide evidence and understanding for the application of learning based intelligent scheduling technology in cloud infrastructure.