Experimental Research in Evolutionary Computation

We present a comprehensive, effective and very efficient methodology for the design and experimental analysis of search heuristics such as evolutionary algorithms, differential evolution, pattern search or even classical methods such as the Nelder-Mead simplex algorithm. Our approach extends the sequential parameter optimization (SPO) method that has been successfully applied as a tuning procedure to numerous heuristics for practical and theoretical optimization problems. The benefit of combining modern and classical statistical methods is demonstrated. Optimization practitioners receive valuable hints for choosing an adequate heuristic for their optimization problems - theoreticians receive guidelines for testing results systematically on real problem instances. We demonstrate how SPO improves the performance of many search heuristics significantly. However, this performance gain is not available for free. Therefore, costs of this tuning process are discussed. Several examples from theory and practice are used to illustrate typical pitfalls in experimentation. Software tools implementing procedures described in this tutorial are freely available.