The present work deals with optimization of costly objective functions. The thesis objective is to minimize a function f(x) which is costly to evaluate. In this context costly means time consuming, and one single function ...