ONNX Export

Status: Public API in netcl.io.export_onnx (io/onnx_export.py)

export_onnx turns a trained netcl Sequential into a standard ONNX graph, so the model can be deployed to any ONNX-compatible runtime — onnxruntime on CPU or GPU, TensorRT, or an embedded/mobile runtime — on a machine that has neither netcl nor PyOpenCL installed. See the io API page for the full call signature and supported-layer list; this page covers why the translation is correct and how it performs against netcl's own inference.

Why the layer mapping is exact, not approximate

The two layers that carry the most weight data map onto ONNX ops with zero layout transformation, because netcl's internal storage already matches the ONNX convention:

• Linear stores its weight as (in_features, out_features) and computes x @ W + b — exactly ONNX Gemm with transB=0.
• Conv2d stores its weight as (out_channels, in_channels, kH, kW) with a symmetric pad and a square stride — exactly ONNX Conv's kernel_shape / pads / strides layout.

BatchNorm2d is exported in its inference form — BatchNormalization using the layer's running_mean / running_var — which is why export_onnx always expects model.eval() to have been called first; exporting a model still in training mode would bake in whatever the running stats happen to be at that point, not a meaningful inference-time value.

The exporter deliberately walks the unfused layer list rather than the fused inference path Sequential.eval() otherwise takes internally (see JIT Compiler and the Conv+BN+ReLU fusion in nn/layers.py::Sequential._try_fuse_layers). This keeps the exported graph a literal, auditable translation of the declared architecture — what you'd draw on a whiteboard — rather than of an internal performance optimization that has nothing to do with the model's semantics.

Verification methodology

Correctness was checked the way any cross-runtime numerical claim should be: by running the same input through both systems and comparing outputs, not by reasoning about the op mapping alone.

1. Build a netcl model, call .eval().
2. Run it on netcl's own backend, keep the host-side output array.
3. Call export_onnx(...), load the file with onnxruntime.InferenceSession.
4. Run the identical input through the ONNX session.
5. Compare with np.allclose(atol=1e-4, rtol=1e-3).

This was done for two shapes of model — a plain MLP (Linear/ReLU/Tanh stack) and a CNN (Conv2d/BatchNorm2d/ReLU/MaxPool2d/LeakyReLU/Flatten/Linear) — and, as a real-world end-to-end case, the trained WIDER FACE detector: a 26-layer Conv/BatchNorm/LeakyReLU/MaxPool2d/Sigmoid stack (scripts/wider_face_detector_clean.netcl) run on real WIDER FACE validation images. In every case the maximum absolute difference between netcl's own output and the ONNX-Runtime output was on the order of 1e-6–1e-7 — i.e. indistinguishable from float32 rounding noise, run to run.

# scripts/verify_onnx_export.py — condensed
y_netcl = model(Tensor.from_host(q, x)).to_host()
export_onnx(model, x.shape, "model.onnx")
sess = ort.InferenceSession("model.onnx", providers=["CUDAExecutionProvider"])
y_onnx = sess.run(None, {"input": x})[0]
assert np.allclose(y_netcl, y_onnx, rtol=1e-3, atol=1e-4)   # holds to ~1e-6/1e-7 in practice

tests/test_onnx_export.py runs this same check as part of the regular test suite (it skips cleanly if onnx / onnxruntime are not installed — see the netcl[onnx] extra), and also asserts that exporting an unsupported layer (e.g. MultiheadAttention) raises NotImplementedError rather than emitting a silently-wrong graph.

GPU verification and a performance comparison

The exported graph was also run with onnxruntime's CUDAExecutionProvider on an actual GPU (not just the CPU provider) — sess.get_providers()[0] was checked at runtime to confirm CUDA was really in use rather than a silent fallback to CPU. On the WIDER FACE detector (256×256 input, 26 layers), steady-state median latency across 50 iterations:

onnxruntime on GPU is consistently ~2.1–2.2× faster than netcl's own OpenCL inference path for this model, because it dispatches to vendor-tuned cuDNN/cuBLAS kernels rather than netcl's hand-written OpenCL kernels — while netcl's native GPU path still comfortably beats the CPU execution provider at larger batch sizes. In other words: export_onnx is not just a compatibility escape hatch, it is frequently the faster deployment target on NVIDIA hardware. See scripts/benchmark_onnx_vs_native.py for the benchmark harness (it measures full host→device→host round trips on all three paths, so the comparison is apples-to-apples).

Known environment sensitivities (GPU execution, not export correctness)

These affect whether onnxruntime's CUDA provider runs at all on a given machine — they are unrelated to whether the exported ONNX graph is correct, which the CPU provider always confirms independently:

• onnxruntime-gpu must match the installed CUDA driver's major version (e.g. driver CUDA 12.2 needs onnxruntime-gpu==1.19.x/1.20.x, not the CUDA-13-only 1.27+ releases).
• On older (Pascal-generation) GPUs, recent cuDNN 9.x point releases can fail Conv execution with CUDNN_STATUS_EXECUTION_FAILED_CUDART even though the session initializes cleanly; pinning nvidia-cudnn-cu12==9.1.0.70 resolved this in testing. Gemm/cuBLAS is not affected by this — only convolution.
• A sandboxed shell that blocks the CUDA driver's ioctls will make cuInit fail (surfacing as CUDA error 999) even though nvidia-smi reports the GPU as healthy; running outside the sandbox resolves it.

See also

• io API — export_onnx call signature, supported layers, code examples.
• Checkpoint — the native .netcl / training-checkpoint formats; use these (not ONNX) for resuming training, since ONNX only captures the inference graph.
• Conv2d and BatchNorm — the two layer types whose netcl-internal layout maps directly onto their ONNX op.
• JIT Compiler — the fused inference path export_onnx deliberately bypasses in favor of a literal, unfused translation.