concepts

no_grad

Status: Public API in netcl.autograd

no_grad is a context manager that disables autograd recording. Inside a no_grad block, ops run normally but no Node is added to the tape and no grad_fn closure is built. The output tensors have requires_grad=False.

Use it for evaluation, inference, and any code path that should not contribute to gradients.

API

import netcl.autograd as ag

# Context manager
with ag.no_grad():
    out = model(x)          # no nodes recorded, no grad_fn allocated

# Functional form
ag.set_grad_enabled(False)
out = model(x)
ag.set_grad_enabled(True)

# Query
enabled = ag.is_grad_enabled()   # True by default

How it works

A module-global flag _GRAD_MODE (default True) controls recording. no_grad sets it to False for the duration of the context block. apply_op checks the flag before creating a Node:

_GRAD_MODE = True → output is a Node (recorded on tape if one is active)
_GRAD_MODE = False → output is a raw Tensor (no node, no tape entry)

All built-in layers (Linear, Conv2d, BatchNorm2d, ResNet blocks, etc.) accept both Node and raw Tensor inputs, no_grad works transparently with the full model API.

Standard evaluation loop

model.eval()
correct, total = 0, 0

with ag.no_grad():
    for x, y in test_loader:
        logits = model(x)                        # raw Tensor, no graph built
        pred = logits.value.to_host().argmax(-1) if hasattr(logits, 'value') else logits.to_host().argmax(-1)
        correct += (pred == y_np).sum()
        total   += y_np.shape[0]

print(f"accuracy: {correct / total:.2%}")

Simpler when you know the model is in eval mode and inputs are plain Tensor:

with ag.no_grad():
    out = model(x_tensor)        # returns Tensor directly
    probs = softmax(out.to_host())

Memory and performance

	with tape	with `no_grad`
Node allocated per op	yes	no
`grad_fn` closure built	yes	no
Activation tensors kept alive	yes (until backward)	no
Peak VRAM	~2–3× inference	1× inference
Backward possible	yes	no

For a ResNet18 at batch 24 the tape adds ~600 MB of intermediate activations over a plain forward pass. no_grad eliminates this entirely.

Common patterns

Inference script

ag.set_grad_enabled(False)      # disable globally for the whole script

model = load_model(checkpoint_path)
model.eval()

for batch in dataloader:
    out = model(batch)
    process(out)

Mixed training / eval in one loop

for x, y in loader:
    # --- training step ---
    with ag.Tape() as tape:
        loss = criterion(model(x), y)
    tape.backward(loss)
    optimizer.step()
    optimizer.zero_grad()

    # --- quick eval on same batch ---
    with ag.no_grad():
        acc = accuracy(model(x), y)

Gradient checkpointing workaround (memory vs compute)

If VRAM is tight, run the three triplet branches sequentially each on its own tape instead of one shared tape, this thirds peak activation memory at the cost of 3× serial compute:

optimizer.zero_grad()
for branch in [anchor, positive, negative]:
    with ag.Tape() as tape:
        loss = partial_loss(encoder(branch))
    tape.backward(loss)          # grads accumulate; activations freed after each step
optimizer.step()