netcl wiki
main

netcl: OpenCL Deep Learning from First Principles

netcl: OpenCL Deep Learning from First Principles

netcl is a high-performance deep learning framework built from scratch on PyOpenCL. It runs on GPUs, CPUs, and accelerators from any hardware vendor supporting OpenCL 1.2 or 2.0: AMD Radeon, Intel Arc, Apple Silicon, NVIDIA GeForce, ARM Mali, and embedded SoCs. There is no CUDA dependency anywhere in the codebase.

On top of low-level OpenCL memory buffers, netcl provides a dynamic Tape-based autograd engine, a runtime JIT compiler that fuses elementwise operations into single kernel launches, native mixed precision with automatic FP16 detection, and a host-staged distributed training engine for multi-GPU workstations.

Core Conceptual Framework (4 Fundamental Premises)

If you are coming from PyTorch, TensorFlow, or JAX, netcl differs in four foundational ways:

flowchart LR Q["1. Command Queue
Explicit Hardware Wire"] --> T["2. Tensor vs Node
cl_mem vs Tape Graph"] T --> F["3. Fused Kernels
Local Memory Tiling"] F --> M["4. Multi-GPU
Host Staged Ring Reduction"]
  1. The Queue is the Physical Wire: PyTorch has an implicit global CUDA device context (x.cuda()). OpenCL does not. Every tensor creation, memory copy, and kernel execution requires an explicit command queue. The queue is your direct pipe to GPU hardware cores.
  2. Tensor is Raw Memory; Node is the Autograd Graph: In netcl, netcl.Tensor is purely a wrapper around device memory (cl_mem) with shapes and strides. It stores zero gradient history. To compute gradients, wrap your tensors with ag.tensor(x) inside a with ag.Tape() as tape: block. This creates graph Node objects that the tape tracks for reverse-mode automatic differentiation.
  3. Device-Resident & Fused Kernels: Consumer GPUs often have 8 GB to 16 GB of VRAM. Naive operations that materialize full $(N \times K)$ distance matrices (clustering) or $(S \times S)$ attention maps (transformers) trigger out-of-memory errors. NetCL fuses reductions, online Softmax, and elementwise operations directly into GPU local memory and registers.
  4. Universal Workstation Multi-GPU: Multi-GPU setups in datacenter clusters rely on expensive NVLink bridges and NCCL. NetCL enables distributed data-parallel training across mismatched, commodity GPUs (such as an AMD Radeon combined with an Intel Arc) on a single workstation using shared-memory multiprocessing and host-based all-reduce.

30-Second Example: Device, Model & Training Step

The snippet below demonstrates device discovery, tensor creation, a forward pass, backpropagation through the tape, and an optimizer update:

import numpy as np
import netcl.autograd as ag
from netcl.core.device import manager
from netcl.core.tensor import Tensor
from netcl.nn import Linear, ReLU, Sequential
from netcl.optim import AdamW

# 1. Connect to the hardware (auto-detects GPU, falls back to CPU)
dev = manager.default("auto")
q = dev.queue
print(f"Active Device: {dev.device_name} ({dev.backend})")

# 2. Build a neural network
model = Sequential(
    Linear(q, 784, 256), ReLU(),
    Linear(q, 256, 128), ReLU(),
    Linear(q, 128, 10),
)
opt = AdamW(model.parameters(), lr=1e-3, weight_decay=1e-4)

# 3. Create raw device tensors (synthetic batch: 32 samples, 10 classes)
x_raw = Tensor.from_host(q, np.random.randn(32, 784).astype(np.float32))
y_raw = Tensor.from_host(q, np.random.randint(0, 10, size=(32,)).astype(np.int32))

# 4. Record forward operations on the Autograd Tape
with ag.Tape() as tape:
    # ag.tensor() wraps raw Tensors into Autograd Nodes
    x_node = ag.tensor(x_raw)
    y_node = ag.tensor(y_raw)
    logits = model(x_node)
    loss = ag.cross_entropy(logits, y_node)

# 5. Reverse-mode automatic differentiation and optimizer step
tape.backward(loss)
opt.step()
opt.zero_grad()

print("Step completed. Batch loss:", loss.value.to_host()[0])

Start Learning: The 7 Guided Curriculum Paths

Explore the complete, first-principles deep learning knowledge curriculum tailored to your background:

Feature Matrix

Device and Tensor Layer

  • Tensor: Device buffer wrapped over cl_mem with integrated BufferPool, supporting FP16, FP32, and FP64 data types.
  • OpenCLBackend: Asynchronous command queues, pinned host memory staging, thread-safe execution, and clean error reporting.
  • DeviceManager: Discovers all OpenCL platforms and devices on the system, exposing them via standardized DeviceHandle objects.

Compute and Operations

  • Accelerated Ops: High-throughput matrix multiplication, 2D convolutions (im2col and Winograd), pooling, and BatchNorm2d.
  • Fused Kernels: Single-launch fused primitives including linear+relu, conv+relu, batchnorm+relu, and matmul+bias+relu.
  • JIT Compiler: Traces elementwise computational subgraphs and emits unified OpenCL C programs at runtime.

Training & Architecture

  • Autograd Tape: Dynamic tape-based reverse-mode automatic differentiation with topological graph sorting.
  • Optimizers: Fused GPU implementations of SGD, Momentum, Adam, AdamW, RMSProp, and learning rate schedulers.
  • Transformers & Modern LM: RMSNorm, Rotary Positional Embeddings (RoPE), SwiGLU activations, KV caching, and streaming FlashAttention.
  • Tokenization & Chat: Byte-Pair Encoding (BPETokenizer) with regex pre-splitting and multi-turn chat templates (ChatFormat).
  • Distributed Engine: Multi-process data-parallelism with shared-memory host reduction and device-side pairwise sums.

Where to Next