Architecture: Module Overview

netcl is organized as a modular hierarchy of cooperating subpackages. Every subpackage has a single, strictly bounded responsibility. The dependency direction is strictly acyclic from high-level APIs down to foundational device management: high-level trainers and modules import from low-level core abstractions, while the core never imports from user-facing trainers.

This clean separation allows developers to optimize or replace internal components (such as memory sub-allocators or JIT backends) without breaking the public neural network APIs.

The Four-Layer Architecture Stack

Subpackages Overview

core: Hardware Abstraction and Memory Management

The foundation of netcl. Manages device enumeration, memory pooling, and tensor buffer lifetimes. - core/device.py: DeviceManager discovers OpenCL platforms and devices, exposing standardized DeviceHandle instances. - core/tensor.py: The Tensor class wrapping low-level cl_mem pointers with shape, strides, and dtype. - core/memory.py: High-throughput memory reuse via BufferPool, PinnedBufferPool, and PersistentBufferPool. - core/backend/opencl.py: Asynchronous Host-to-Device (H2D) and Device-to-Host (D2H) memory transfers. - core/backend/cpu.py: Pure NumPy fallback executing on CPU when OpenCL drivers are absent.

ops: Elementwise, Reduction and Linear Algebra Kernels

Pure forward-pass OpenCL kernels. Functions in ops take raw Tensor inputs and return new device Tensor outputs. They perform no gradient recording and store no historical graph metadata.

autograd: Dynamic Tape and Reverse-Mode Differentiation

Automatic differentiation engine. Provides the active Tape singleton and Node wrappers. During the forward pass, operations between nodes register backward execution closures onto the tape. Calling tape.backward(loss) traverses the tape in reverse topological order, accumulating .grad on leaf parameters.

nn: Modular Neural Network Layers

PyTorch-compatible neural network modules built on top of autograd. Implements Linear, Conv2d, BatchNorm2d, Dropout, Sequential, activation layers (ReLU, GELU, SwiGLU), and complete architectures like ResNet18.

optim: Optimization Algorithms

Fused GPU implementations of parameter update rules, including SGD, Momentum, RMSprop, Adam, and AdamW. Includes learning rate schedulers (CosineAnnealingLR, WarmupCosine, ReduceLROnPlateau) and gradient norm clipping.

data: Prefetching and Shared-Memory Input Pipelines

High-speed data loading pipeline featuring DataLoader, dataset transformations, GPU-side augmentations, and a multiprocess shared-memory ring buffer for low-latency batch staging.

distributed: Workstation Multi-Device Training

Single-host multi-device training across arbitrary OpenCL devices. Provides host-staged collectives (all_reduce, broadcast, scatter, gather) and the unified data_parallel_step orchestrator.

transformer: Modern Sequence Architectures

Modern decoder-only transformer building blocks: RMSNorm, RotaryAttention (RoPE embeddings with QK-normalization), SwiGLU feed-forward networks, DecoderBlock, KVCache, and streaming FlashAttention.

text: Tokenization and Chat Templates

Lossless byte-level BPE tokenizer (BPETokenizer) with regex pre-splitting, vocabulary training, and structured multi-turn conversation formatting (ChatFormat).

cluster: GPU-Accelerated Unsupervised Clustering

High-throughput clustering algorithms engineered to avoid materializing $(N \times K)$ distance matrices: KMeans, MiniBatchKMeans, SphericalKMeans, BalancedKMeans (Sinkhorn optimal transport), GaussianMixture, DBSCAN, and graph-based SpectralClustering with LOBPCG eigensolvers.

ssl: Self-Supervised Representation Learning

Self-supervised learning algorithms spanning contrastive methods (SimCLR, MoCo), non-contrastive methods (BYOL, SimSiam), redundancy reduction (BarlowTwins, VICReg), and clustering SSL (SwAV, DINO).

evo: Neuroevolution and Architecture Search

Hardware-accelerated evolutionary optimization keeping entire candidate populations in contiguous device memory. Implements GA, ES, OpenAIES, DE, SepCMAES, and Neural Architecture Search (ArchSpace, ArchitectureSearch) with weight inheritance.

Data Flow During a Training Step

The diagram below tracks tensor buffers and autograd nodes through a single training iteration:

Executable code outline matching this pipeline:

import netcl.autograd as ag
from netcl.core.tensor import Tensor
from netcl.io import save_model

# 1. DataLoader yields host data
for x_batch, y_batch in loader:
    # 2. Transfer raw batch to GPU
    x_dev = Tensor.from_host(queue, x_batch)
    y_dev = Tensor.from_host(queue, y_batch)

    # 3. Record forward pass on the Tape
    with ag.Tape() as tape:
        # Wrap raw tensors into autograd nodes
        x_node = ag.tensor(x_dev)
        y_node = ag.tensor(y_dev)
        logits = model(x_node)
        loss = ag.cross_entropy(logits, y_node)

    # 4. Reverse-mode automatic differentiation
    tape.backward(loss)

    # 5. Fused GPU parameter update
    opt.step()
    opt.zero_grad()

    # 6. Save checkpoint
    save_model(model, "checkpoint.netcl")

Related Documentation

• Curriculum Hub: Explore the theoretical and mathematical foundations behind netcl.
• Tensor Backend: Details on strided views, buffer handles, and memory pools.
• Autograd and Tape: Deep dive into the reverse-mode autograd tape.
• Distributed Architecture: Multi-device shared memory and collective communication.