concepts

no_grad

Status: Public API in netcl.autograd

no_grad is a context manager that disables autograd recording. Inside a no_grad block, ops run normally but no Node is added to the tape and no grad_fn closure is built. The output tensors have requires_grad=False.

Use it for evaluation, inference, and any code path that should not contribute to gradients.

API

import netcl.autograd as ag

# Context manager
with ag.no_grad():
    out = model(x)          # no nodes recorded, no grad_fn allocated

# Functional form
ag.set_grad_enabled(False)
out = model(x)
ag.set_grad_enabled(True)

# Query
enabled = ag.is_grad_enabled()   # True by default

How it works

A module-global flag _GRAD_MODE (default True) controls recording. no_grad sets it to False for the duration of the context block. apply_op checks the flag before creating a Node:

_GRAD_MODE = True → output is a Node (recorded on tape if one is active)
_GRAD_MODE = False → output is a raw Tensor (no node, no tape entry)

All built-in layers (Linear, Conv2d, BatchNorm2d, ResNet blocks, etc.) accept both Node and raw Tensor inputs — no_grad works transparently with the full model API.

Standard evaluation loop

model.eval()
correct, total = 0, 0

with ag.no_grad():
    for x, y in test_loader:
        logits = model(x)                        # raw Tensor, no graph built
        pred = logits.value.to_host().argmax(-1) if hasattr(logits, 'value') else logits.to_host().argmax(-1)
        correct += (pred == y_np).sum()
        total   += y_np.shape[0]

print(f"accuracy: {correct / total:.2%}")

Simpler when you know the model is in eval mode and inputs are plain Tensor:

with ag.no_grad():
    out = model(x_tensor)        # returns Tensor directly
    probs = softmax(out.to_host())

Memory and performance

	with tape	with `no_grad`
Node allocated per op	yes	no
`grad_fn` closure built	yes	no
Activation tensors kept alive	yes (until backward)	no
Peak VRAM	~2–3× inference	1× inference
Backward possible	yes	no

For a ResNet18 at batch 24 the tape adds ~600 MB of intermediate activations over a plain forward pass. no_grad eliminates this entirely.

Common patterns

Inference script

ag.set_grad_enabled(False)      # disable globally for the whole script

model = load_model(checkpoint_path)
model.eval()

for batch in dataloader:
    out = model(batch)
    process(out)

Mixed training / eval in one loop

for x, y in loader:
    # --- training step ---
    with ag.Tape() as tape:
        loss = criterion(model(x), y)
    tape.backward(loss)
    optimizer.step()
    optimizer.zero_grad()

    # --- quick eval on same batch ---
    with ag.no_grad():
        acc = accuracy(model(x), y)

Gradient checkpointing workaround (memory vs compute)

If VRAM is tight, run the three triplet branches sequentially each on its own tape instead of one shared tape — this thirds peak activation memory at the cost of 3× serial compute:

optimizer.zero_grad()
for branch in [anchor, positive, negative]:
    with ag.Tape() as tape:
        loss = partial_loss(encoder(branch))
    tape.backward(loss)          # grads accumulate; activations freed after each step
optimizer.step()