Adaptive Mutation and Crossover in sklearn-genetic-opt

Genetic algorithms use two main operators to explore the hyperparameter space: mutation (random changes to individuals) and crossover (combining two solutions to generate offspring). Both operators have associated probabilities that control how frequently they are applied during evolution.

You can keep these probabilities fixed or schedule how they change. This article explains the four adapters, checks their curves over a finite run, and shows how to compare a schedule with a fixed baseline.

The examples use sklearn-genetic-opt 0.14.0, which requires Python 3.12 or later. Install the plotting extra for the chart:

python -m pip install "sklearn-genetic-opt[plot]==0.14.0"

Fixed vs. Adaptive Parameters

With a fixed adapter, the mutation probability remains the same across generations. A schedule lets you allocate exploration differently over the run:

• Early on: a higher mutation probability can sample more new configurations.
• Later: a lower mutation probability can preserve more of the combinations produced by selection and crossover.

Those are choices to evaluate, rather than a guarantee of convergence. A fixed probability can be a useful baseline; the better policy depends on the search space, evaluation noise and available budget.

The Exploration-Exploitation Trade-off

This trade-off is fundamental in all heuristic search methods. In the context of hyperparameter search with genetic algorithms:

• Exploration: trying diverse configurations, risking less promising individuals to discover unexpectedly good regions.
• Exploitation: concentrating computational resources on refining the best configurations found so far.

Adapters express a probability schedule. Selection, the search-space distributions and the chosen genetic algorithm still determine which configurations are evaluated. Inspect the schedule before fitting, then measure whether it helps on your problem.

Available Adapters in sklearn-genetic-opt

The sklearn-genetic-opt library offers four types of adapters for mutation and crossover probabilities:

ConstantAdapter

The value stays constant. Passing a plain probability such as mutation_probability=0.2 also creates a constant adapter internally. To construct one explicitly, supply its three required arguments:

from sklearn_genetic import ConstantAdapter
# always returns 0.2, regardless of generation
adapter = ConstantAdapter(initial_value=0.2, end_value=0.2, adaptive_rate=0)

ExponentialAdapter

Approaches end_value exponentially. This example decreases mutation from a higher initial probability; reversing the endpoints produces an increasing schedule.

from sklearn_genetic import ExponentialAdapter
adapter = ExponentialAdapter(initial_value=0.4, end_value=0.05, adaptive_rate=0.1)

InverseAdapter

Approaches end_value with an inverse curve: it changes faster early, then slows. This example increases crossover; that direction is a policy choice, not an intrinsic advantage of the adapter.

from sklearn_genetic import InverseAdapter
adapter = InverseAdapter(initial_value=0.6, end_value=0.9, adaptive_rate=0.1)

PotentialAdapter

Uses the geometric expression end_value + (initial_value - end_value) * (1 - adaptive_rate)**t. For 0 < adaptive_rate < 1, the changes get smaller over time. With the endpoints below it increases toward the target, with the largest changes early in the run.

from sklearn_genetic import PotentialAdapter
adapter = PotentialAdapter(initial_value=0.6, end_value=0.9, adaptive_rate=0.1)

Math Behind ExponentialAdapter

The exponential decay formula is:

value(t) = end_value + (initial_value - end_value) * exp(-adaptive_rate * t)

Here t is the zero-based adapter step: the first .step() call uses t=0. The 0.14.0 implementation defines these curves. To understand the effect of adaptive_rate:

• With adaptive_rate=0.05: decay is slow, the transition happens gradually over many generations.
• With adaptive_rate=0.2: decay is fast, most of the transition happens in the first few generations.

import numpy as np
import matplotlib.pyplot as plt

def exponential_adapter(t, initial=0.4, end=0.05, rate=0.1):
    return end + (initial - end) * np.exp(-rate * t)

def inverse_adapter(t, initial=0.6, end=0.9, rate=0.1):
    return end - (end - initial) / (1 + rate * t)

generations = np.arange(0, 25)

plt.figure(figsize=(10, 4))
plt.subplot(1, 2, 1)
plt.plot(generations, [exponential_adapter(t) for t in generations])
plt.title("ExponentialAdapter (mutation)")
plt.xlabel("Adapter step"); plt.ylabel("Probability")

plt.subplot(1, 2, 2)
plt.plot(generations, [inverse_adapter(t) for t in generations])
plt.title("InverseAdapter (crossover)")
plt.xlabel("Adapter step"); plt.ylabel("Probability")
plt.tight_layout()
plt.show()

Over the 25 plotted steps, t=0 through t=24, mutation decreases from 0.4 to approximately 0.0818 and crossover increases from 0.6 to approximately 0.8118. These are finite-step values; end_value is the target approached over time.

Complete Example: GASearchCV with Adapters

Here is how to integrate adapters into a real hyperparameter search with a scikit-learn classifier:

from sklearn_genetic import (
    GASearchCV, ExponentialAdapter, InverseAdapter, EvolutionConfig, OptimizationConfig
)
from sklearn_genetic.space import Integer, Continuous, Categorical
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split, StratifiedKFold

# Sample data
X, y = make_classification(
    n_samples=1000, n_features=20, n_informative=10, random_state=42
)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# Hyperparameter search space
param_grid = {
    "n_estimators": Integer(50, 300),
    "max_depth": Integer(3, 15),
    "min_samples_split": Integer(2, 20),
    "max_features": Continuous(0.1, 1.0),
    "criterion": Categorical(["gini", "entropy"])
}

clf = RandomForestClassifier(random_state=42)
cv = StratifiedKFold(n_splits=3, shuffle=True, random_state=42)

# GASearchCV with adaptive parameters
search = GASearchCV(
    estimator=clf,
    param_grid=param_grid,
    scoring="accuracy",
    cv=cv,
    evolution_config=EvolutionConfig(
        population_size=30,
        generations=25,
        mutation_probability=ExponentialAdapter(
            initial_value=0.4,
            end_value=0.05,
            adaptive_rate=0.1
        ),
        crossover_probability=InverseAdapter(
            initial_value=0.6,
            end_value=0.9,
            adaptive_rate=0.1
        )
    ),
    optimization_config=OptimizationConfig(diversity_control=False),
    random_state=42,
    verbose=True,
    n_jobs=1
)

search.fit(X_train, y_train)

print(f"\nBest hyperparameters: {search.best_params_}")
print(f"Best CV score: {search.best_score_:.4f}")
print(f"Test score: {search.score(X_test, y_test):.4f}")

Pass the probabilities inside EvolutionConfig: its fields take precedence over the outer arguments. The example disables the separate diversity controller so that it does not boost mutation on top of the schedule. It uses one job to limit parallel work. Population size, generations, cross-validation folds and estimator cost all affect runtime; the printed scores come from your execution, not a promised benchmark.

Using Only ExponentialAdapter for Mutation

If you prefer a more conservative approach and only want to adapt mutation:

search_mutation_only = GASearchCV(
    estimator=RandomForestClassifier(random_state=42),
    param_grid=param_grid,
    scoring="accuracy",
    cv=cv,
    evolution_config=EvolutionConfig(
        population_size=30,
        generations=25,
        mutation_probability=ExponentialAdapter(
            initial_value=0.4,
            end_value=0.05,
            adaptive_rate=0.1
        ),
        crossover_probability=0.6,  # fixed
    ),
    optimization_config=OptimizationConfig(diversity_control=False),
    random_state=42,
    n_jobs=1
)
search_mutation_only.fit(X_train, y_train)

This isolates the mutation schedule while keeping crossover fixed. The default eaMuPlusLambda algorithm requires a combined crossover/mutation probability budget of at most one: fixing crossover at 0.6 leaves room for the initial mutation probability of 0.4. A larger crossover value can clip the effective mutation probability. Compare this run with the same fixed-crossover baseline before deciding whether to schedule both operators.

Comparing Fixed vs. Adaptive

To inspect two runs with the same space, folds, population size and generation limit, compare their recorded fitness histories:

# Search with fixed parameters
search_fixed = GASearchCV(
    estimator=RandomForestClassifier(random_state=42),
    param_grid=param_grid,
    scoring="accuracy",
    cv=cv,
    evolution_config=EvolutionConfig(
        population_size=30,
        generations=25,
        mutation_probability=0.1,   # fixed
        crossover_probability=0.8,  # fixed
    ),
    optimization_config=OptimizationConfig(diversity_control=False),
    random_state=42,
    n_jobs=1
)
search_fixed.fit(X_train, y_train)

# Compare fitness history
import pandas as pd

history_adaptive = pd.DataFrame(search.history)
history_fixed = pd.DataFrame(search_fixed.history)

print("Convergence: max fitness score per generation:")
print(f"{'Gen':>4} | {'Adaptive':>12} | {'Fixed':>12}")
print("-" * 35)
for gen in [0, 5, 10, 15, 20, 24]:
    a = history_adaptive.loc[gen, "fitness_max"]
    f = history_fixed.loc[gen, "fitness_max"]
    print(f"{gen:>4} | {a:>12.4f} | {f:>12.4f}")

This table describes one run per policy. Repeat with multiple search seeds and record actual candidate evaluations and elapsed time: equal generation limits need not mean equal cost when caching reuses evaluations. Choose schedules using training/validation data and reserve the held-out test set for final assessment. A higher maximum CV score in one history does not establish faster search, robustness or better generalization.

When to Use Each Adapter

Pick the curve whose transition fits the run you want to test, then compare it against fixed probabilities under a documented evaluation budget.

Choosing adaptive_rate

The adaptive_rate parameter controls how quickly the transition happens. A practical way to choose it is to think about at which generation you want to reach roughly half the transition:

• For ExponentialAdapter: half the decay from initial_value to end_value happens at t = ln(2) / adaptive_rate.
• With adaptive_rate=0.1, the probability is halfway between its endpoints near step 7. This describes the curve, not how much useful exploration the optimizer has performed.

For a desired half-transition at step h, choose adaptive_rate = ln(2) / h. Check the resulting curve over your actual step budget before fitting.

Conclusion

An adapter makes the probability policy explicit. Start with fixed probabilities, inspect the scheduled values over your available steps, and compare reproducible runs before adding a schedule to your training workflow.

ExponentialAdapter, InverseAdapter, and PotentialAdapter are part of sklearn-genetic-opt. If you're new to evolutionary hyperparameter search, start with Tune Your Scikit-learn Model Using Evolutionary Algorithms.