Simulated Annealing - Related Methods

Related Methods

• Quantum annealing uses "quantum fluctuations" instead of thermal fluctuations to get through high but thin barriers in the target function.

• Stochastic tunneling attempts to overcome the increasing difficulty simulated annealing runs have in escaping from local minima as the temperature decreases, by 'tunneling' through barriers.

• Tabu search normally moves to neighbouring states of lower energy, but will take uphill moves when it finds itself stuck in a local minimum; and avoids cycles by keeping a "taboo list" of solutions already seen.

• Reactive search optimization focuses on combining machine learning with optimization, by adding an internal feedback loop to self-tune the free parameters of an algorithm to the characteristics of the problem, of the instance, and of the local situation around the current solution.

• Stochastic gradient descent runs many greedy searches from random initial locations.

• Genetic algorithms maintain a pool of solutions rather than just one. New candidate solutions are generated not only by "mutation" (as in SA), but also by "recombination" of two solutions from the pool. Probabilistic criteria, similar to those used in SA, are used to select the candidates for mutation or combination, and for discarding excess solutions from the pool.

• Graduated optimization digressively "smooths" the target function while optimizing.

• Ant colony optimization (ACO) uses many ants (or agents) to traverse the solution space and find locally productive areas.

• The cross-entropy method (CE) generates candidates solutions via a parameterized probability distribution. The parameters are updated via cross-entropy minimization, so as to generate better samples in the next iteration.

• Harmony search mimics musicians in improvisation process where each musician plays a note for finding a best harmony all together.

• Stochastic optimization is an umbrella set of methods that includes simulated annealing and numerous other approaches.

• Particle swarm optimization is an algorithm modelled on swarm intelligence that finds a solution to an optimization problem in a search space, or model and predict social behavior in the presence of objectives.

• Intelligent Water Drops (IWD) which mimics the behavior of natural water drops to solve optimization problems

• Parallel tempering is a simulation of model copies at different temperatures (or Hamiltonians) to overcome the potential barriers.