Computational complexity analysis of linear optimization algorithms

Abstract

Computational complexity and performance analysis in Linear Optimization algorithms have always been topics of particular interest among the Operational Research community. In the current thesis, we are presenting all aspects and analysis results of our study on the performance of the Exterior Point Simplex algorithm, the Interior Point Method, and the Primal and Dual Simplex algorithms. Our objective is to generate valid and accurate prediction models for the computational performance of these algorithms. Our analysis is separated in three main parts, as described below. First, we investigate the computational behavior of the Exterior Point Simplex algorithm (EPSA). Up until now, a significant difference has been observed between the theoretical worst case complexity and practical performance of simplex-type algorithms. To appropriately examine the latter, computational tests have been carried out on randomly generated sparse linear problems and on a small set of benchmark problems. Specifically, 6780 linear problems have been randomly generated, in order to formulate a respectable amount of experiments. This first part of our study consists of the measurement of the number of iterations that EPSA needs for the solution of the above mentioned problems and benchmark dataset. Our purpose is to form representative regression models, which would be significant for the evaluation of the algorithm’s efficiency and could act as predictive models for the algorithm’s performance. From each linear problem, we have taken several characteristics into account, such as the number of constraints and variables, the sparsity and bit length, and the condition of the constraint matrix. It is remarkable that the formulated model for the randomly generated problems reveals a linear relation between the number of EPSA iterations and the above mentioned characteristics. Next, we extend our analysis, being concerned about the ability to choose the most efficient algorithm, in terms of execution time, for a given set of linear programming problems. Algorithm selection has been a significant, but at the same time, challenging process in all linear programming solvers. For the purpose of this part of our study, we utilize CPLEX Optimizer, which supports Primal and Dual variants of the Simplex algorithm and the Interior Point Method (IPM). We examine a performance prediction model using artificial neural networks for the CPLEX’s Interior Point Method on a set of 295 benchmark linear programming problems (Netlib, Kennington, Mészáros, Mittelmann) and measure the execution time needed for their solution. Specific characteristics of the linear programming problems are examined, such as the number of constraints and variables, the nonzero elements of the constraint matrix and the right-hand side, and the rank of the constraint matrix of the linear programming problems. Our purpose is to identify a model, which could be used for prediction of the algorithm’s efficiency on linear programming problems of similar structure. This model can be used prior to the execution of the interior point method in order to estimate its execution time. Experimental results show a good fit of our model both on the training and test set, with the coefficient of determination value at 78% and 72%, respectively. The current study is concluded by examining a prediction model using artificial neural networks for the performance of CPLEX’s Primal and Dual Simplex algorithms on the same dataset and with the same variables as in IPM. The extracted results prove that a regression model cannot predict accurately the execution time of CPLEX’s Primal and Dual Simplex algorithms. To overcome this issue, we treat the problem as a classification problem. Instead of estimating the execution time, our models estimate the class under which the execution time will fall. Experimental results show a good performance of the models both for Primal and Dual Simple algorithms, with an accuracy score of 0.83 and 0.84, respectively.