netcl wiki
concepts

Neuroevolution & Evolutionary Optimization

Neuroevolution & Evolutionary Optimization

The netcl.evo module provides hardware-accelerated evolutionary optimization algorithms for OpenCL devices. It optimizes model parameters for non-differentiable objectives and performs automated Neural Architecture Search (NAS) directly within GPU memory.

[!TIP] Theory & Curriculum Link: For the first-principles mathematical derivation of Natural Evolution Strategies (NES), CMA-ES covariance scaling, and GPU-resident population dynamics, see Curriculum: Evolutionary & Black-Box Optimization.

1. Quick Example: Optimizing a Model Without Backpropagation

import numpy as np
from netcl import evo
from netcl.core.device import manager
from netcl.core.tensor import Tensor
from netcl.nn import Sequential, Linear, ReLU

q = manager.default("auto").queue

# 1. Define model architecture
model = Sequential(
    Linear(q, 8, 16), ReLU(),
    Linear(q, 16, 2),
)

# 2. Evaluation dataset and non-differentiable scoring function (Accuracy)
x = Tensor.from_host(q, np.random.randn(32, 8).astype(np.float32))
y_true = np.random.randint(0, 2, size=32, dtype=np.int32)

def score_fn(m):
    preds = np.argmax(m(x).to_host(), axis=1)
    return float((preds == y_true).mean())

# 3. Launch GPU-accelerated neuroevolution
result = evo.neuroevolve(
    model,
    score_fn=score_fn,
    algorithm="nes",
    pop_size=32,
    generations=20,
    sigma=0.03,
    seed=42,
    maximize=True,
)

print(f"Best Population Accuracy: {result.best_fitness * 100:.1f}%")

2. Mathematical Formulations of Supported Strategies

The entire candidate population is maintained inside a single contiguous OpenCL device buffer. Mutation, recombination, and fitness scoring execute as parallel GPU kernels without transferring candidate parameters back to the host CPU.

OpenAI Natural Evolution Strategy ("nes" or "openai")

NES maximizes expected fitness under Gaussian parameter perturbations $\theta \in \mathbb{R}^D$:

$$J(\theta) = \mathbb{E}_{\epsilon \sim \mathcal{N}(0, I)} [f(\theta + \sigma \epsilon)]$$

The Monte Carlo pseudo-gradient across a population of $\lambda$ individuals is:

$$\nabla_\theta J(\theta) \approx \frac{1}{\lambda \sigma} \sum_{i=1}^\lambda f(\theta + \sigma \epsilon_i) \epsilon_i$$

Parameter updates follow:

$$\theta \leftarrow \theta + \alpha \nabla_\theta J(\theta)$$

NetCL's OpenCL kernel generates Gaussian perturbations $\epsilon_i$ on the fly using the Philox pseudorandom number generator, requiring zero VRAM allocation for random perturbation matrices.

Differential Evolution ("de")

For each individual candidate $\mathbf{x}_i$, a mutant vector is generated from three randomly selected population members $\mathbf{r}_1, \mathbf{r}_2, \mathbf{r}_3$:

$$\mathbf{v}_i = \mathbf{x}_{r_1} + F \cdot (\mathbf{x}_{r_2} - \mathbf{x}_{r_3})$$

Where $F \in [0.5, 1.0]$ is the differential weight. In binomial crossover, trial candidate $\mathbf{u}_{i, j}$ inherits gene $\mathbf{v}_{i, j}$ with probability $CR$, or retains $\mathbf{x}_{i, j}$ otherwise.

Separable CMA-ES ("cmaes")

SepCMAES dynamically adapts the search mean $\mathbf{m} \in \mathbb{R}^D$ and coordinate-wise standard deviations $\boldsymbol{\sigma} \in \mathbb{R}^D$ independently for each dimension, scaling with linear $O(D)$ memory rather than quadratic $O(D^2)$:

$$\mathbf{m}^{(g+1)} = \mathbf{m}^{(g)} + c_m \sum_{i=1}^\mu w_i (\mathbf{x}_{i:\lambda} - \mathbf{m}^{(g)})$$

3. Algorithm Selection Guide

Application Strategy Algorithm Key Practical Advantage
Weight Optimization (10k to 1M parameters) OpenAIES "nes", "openai" Minimal VRAM footprint; perturbations generated via Philox PRNG
Neural Architecture Search (NAS) GeneticAlgorithm "ga" Mutations across discrete layer depths, kernel sizes, and activations
Continuous Hyperparameter Tuning SepCMAES "cmaes" Adaptive step sizes per coordinate with linear $O(D)$ memory
Non-Convex Benchmarks with Few Parameters DifferentialEvolution "de" Extreme resistance to getting trapped in local minima

4. Automated Neural Architecture Search (NAS)

ArchitectureSearch discovers the optimal layer topology, channel capacities, and hyperparameters for a specific dataset:

from netcl import evo

# 1. Define search space
space = evo.ArchSpace(
    in_channels=3,
    input_hw=(32, 32),
    num_classes=10,
    max_layers=8,
    layer_types=("conv", "pool", "dropout"),
    channels=(16, 32, 64, 128),
    kernels=(3, 5),
    activations=("ReLU", "LeakyReLU"),
)

# 2. Initialize initial architecture population
pop = evo.ArchPopulation.random(space, q, size=16, seed=0)

# 3. Evaluation function
def evaluate_arch(arch_model):
    return float(np.random.rand())

fitness = evo.ArchFitness(space, evaluate_arch, queue=q)

# 4. Execute search
search = evo.ArchitectureSearch(
    pop,
    fitness,
    generations=10,
    elitism=2,
    mutation_rate=0.08,
    verbose=True,
)
result = search.run()

# 5. Build and export optimal architecture
best_model = result.build(q)
result.save("best_architecture.json")
print("Optimal discovered layer structure:", result.describe())

Weight Inheritance

Via evo.inherit_weights, newly generated child candidate models inherit overlapping trained weights from their parent networks. Consequently, successive generations converge significantly faster than starting each architecture from random initialization.