Intelligent workflow scheduling for Big Data applications in IoT cloud computing environments

Effective task scheduling is recognized as one of the main critical challenges in cloud computing; it is an essential step for effectively exploiting cloud computing resources, as several tasks may need to be efficiently scheduled on various virtual machines by minimizing makespan and maximizing resource utilization. Task scheduling is an NP-hard problem, and consequently, finding the best solution may be difficult, particularly for Big Data applications. This paper presents an intelligent Big Data task scheduling approach for IoT cloud computing applications using a hybrid Dragonfly Algorithm. The Dragonfly algorithm is a newly introduced optimization algorithm for solving optimization problems which mimics the swarming behaviors of dragonflies. Our algorithm, MHDA, aims to decrease the makespan and increase resource utilization, and is thus a multi-objective approach. β-hill climbing is utilized as a local exploratory search to enhance the Dragonfly Algorithm’s exploitation ability and avoid being trapped in local optima. Two experimental studies were conducted on synthetic and real trace datasets using the CloudSim toolkit to compare MHDA to other well-known algorithms for solving task scheduling problems. The analysis, which included the use of a t-test, revealed that MHDA outperformed other well-known algorithms: MHDA converged faster than other methods, making it useful for Big Data task scheduling applications, and it achieved 17.12% improvement in the results.