Optimization Science

31 Oct 2015

For extremely complicated optimization problems, mathematical methods usually fail in finding the exact optima. Smart researchers resort to approximations, which are sometimes based on experience or intuition, and often unexplainable in rigorously mathematical ways. However, they may perform pretty well in practice.

To find, or to develop, approximations for extremely complicated optimization problems, we can not rely only on known mathematics, which provides us specific points of view. What if there is any point of view, or notion, that is unknown by far but actually would lead us to brand new mathematics?

If we compare the known mathematics to the Geocentrism (Geocentric Model), they do provide some descriptions and clues about planets, but in an extremely complicated way. However, when it comes to Heliocentrism, the orbiting of planets seems to be much simpler.

We are probably able to state that complication may be a lack of appropriate model. A scientific way is to observe, guess, experiment and revise. By repeating this iteration enough times, a much more accurate new model for the problem of interest is expected to form. However, the resulted model may be unable to be totally explained mathematically. So, it might be the mathematics’ fault, isn’t it?

The equation, $E=mc^{2}$, seems to be absolute nonsense in mathematical meaning for a person doesn’t know the physical relationship.

If we develop a mathematical model for a physical system, it usually means that we have enough knowledge about the system. This is plausible for relatively small problems with straightforward mechanisms. However, for large complicated system, an appropriate mathematical model can be never developed without looking into the physical system itself.

An optimization problem can be formulated as objective function(s) and the valid domain, e.g. $\max f(x,y)= \sin(x+\max{|x|, y})$, where $x\in X, y\in Y$. A typical search procedure can be described as the following:

• Select an initial point ( $x_{0}, y_{0}$ ) in the domain

• Proceed to next point according to a direction, or to stop

• Repeat the above two steps (until stopped)

Obviously, the initial point selection and directions of each proceeding are critical to the performance of the procedure. In special cases, e.g., linear or convex, there are mature procedures and guidelines, e.g., the simplex method.

Meta-strategy

The term Meta-Strategy is used to refer to a method for a strategy, i.e., a rule or procedure that generates outcomes based on available information and knowledge.