Predicting job execution time on a high-performance computing cluster using a hierarchical data-driven methodology

Nowadays, evaluating the performance of a vehicle before the production phase is challenging and important. In the automotive industry, many virtual simulations are needed to model the vehicle behavior in the best possible way. However, these simulations require a lot of time without the user knowing their runtime in advance. Knowing the required time in advance would allow the user to manage the simulations more effectively and choose the best strategy to use the available computational resources. For this reason, we present an innovative data-driven method to estimate in advance the execution time of simulations. Our approach integrates unsupervised techniques, such as constrained k-means clustering, with classification and regression algorithms based on tree structures. In this paper, we present an innovative and hierarchical data-driven method for estimating the execution time of jobs. Numerous experiments were conducted on a real dataset to verify the effectiveness of the proposed approach. The experimental results show that the proposed method is promising.