Natural selection is the world optimizing for survival on Earth. Every life form on Earth is a solution generated by evolution's algorithm, which evolves a population of individuals over generations, optimizing for survival. Below is a way to describe this algorithm:
- The algorithm begins by creating a random initial population.
- Select the fittest individuals to be the parents of the next generation (a score). Randomly select some of the non-fittest individuals to be parents as well, increasing the chance of finding a global optimum.
- Crossover the selected parents, creating new individuals. There will be a chance that the child will have a random mutation of its numbers.
- Calculate the average population fitness. Rinse and repeat.
- When the average population fitness is ~0 (or close to it), stop evolving.
This is the pseudocode:
START
Generate the initial population
Compute fitness
REPEAT
Selection
Crossover
Mutation
Compute fitness
UNTIL the population has converged
STOP
GA algorithm are known for decades, but they are still fun as an optimization technique. Combined with neural networks, they can help to find the best hyperparameters, by creating a population of many NNs and letting it evolve.