Editorial illustration for Equinox Tutorial Shows JAX Native Modules, Filtered Transforms, and Debug Tips
JAX Native Modules: Building ResNet-Style MLPs with Equinox
Equinox Tutorial Shows JAX Native Modules, Filtered Transforms, and Debug Tips
The new Equinox tutorial walks you through building a ResNet‑style MLP with JAX native modules, filtered transforms, and stateful layers, then stitches everything together into a full training loop. While the code shows how to assemble the model, it also drops a handful of practical shortcuts that many users overlook. One of those shortcuts is a debug routine that captures the JAXpr of a vectorized forward pass, counts its primitive operations, and saves a quick visual of the loss curve.
The author even tacks on a “bonus” section that demonstrates eqx.filter_jit combined with shape‑inference tricks—something that can shave seconds off compilation time on larger batches. If you’ve ever wondered how many equations a compiled ResNetMLP actually contains, or how to verify that your plot was written to disk, the snippet that follows answers both questions in a single, terse block of code.
---
Plot saved to equinox_tutorial.png") print("\n" + "="*60) print("BONUS: eqx.filter_jit + shape inference debug tip") print("="*60) jaxpr = jax.make_jaxpr(jax.vmap(model))(x_plot) n_eqns = len(jaxpr.jaxpr.eqns) print(f"Compiled ResNetMLP jaxpr has {n_eqns} equations (ops) for batch input {x_plot.shap
Plot saved to equinox_tutorial.png") print("\n" + "="*60) print("BONUS: eqx.filter_jit + shape inference debug tip") print("="*60) jaxpr = jax.make_jaxpr(jax.vmap(model))(x_plot) n_eqns = len(jaxpr.jaxpr.eqns) print(f"Compiled ResNetMLP jaxpr has {n_eqns} equations (ops) for batch input {x_plot.shape}") BATCH = 128 EPOCHS = 30 steps_per_epoch = len(X_train) // BATCH train_losses, val_losses = [], [] t0 = time.time() for epoch in range(EPOCHS): key, sk = jax.random.split(key) perm = jax.random.permutation(sk, len(X_train)) X_s, Y_s = X_train[perm], Y_train[perm] epoch_loss = 0.0 for step in range(steps_per_epoch): xb = X_s[step*BATCH:(step+1)*BATCH] yb = Y_s[step*BATCH:(step+1)*BATCH] model, opt_state, loss = train_step(model, opt_state, xb, yb) epoch_loss += loss.item() val_loss = evaluate(model, X_val, Y_val).item() train_losses.append(epoch_loss / steps_per_epoch) val_losses.append(val_loss) if (epoch + 1) % 5 == 0: print(f"Epoch {epoch+1:3d}/{EPOCHS} " f"train_loss={train_losses[-1]:.5f} " f"val_loss={val_losses[-1]:.5f}") print(f"\nTotal training time: {time.time()-t0:.1f}s") print("\n" + "="*60) print("SECTION 7: Save & load model weights") print("="*60) eqx.tree_serialise_leaves("model_weights.eqx", model) key, mk2 = jax.random.split(key) model_skeleton = ResNetMLP(1, 64, 1, n_blocks=4, key=mk2) model_loaded = eqx.tree_deserialise_leaves("model_weights.eqx", model_skeleton) diff = jnp.max(jnp.abs( jax.tree_util.tree_leaves(eqx.filter(model, eqx.is_array))[0] - jax.tree_util.tree_leaves(eqx.filter(model_loaded, eqx.is_array))[0] )) print(f"Max weight difference after reload: {diff:.2e} (should be 0.0)") fig, axes = plt.subplots(1, 2, figsize=(12, 4)) axes[0].plot(train_losses, label="Train MSE", color="#4C72B0") axes[0].plot(val_losses, label="Val MSE", color="#DD8452", linestyle="--") axes[0].set_xlabel("Epoch") axes[0].set_ylabel("MSE") axes[0].set_title("Training curves") axes[0].legend() axes[0].grid(True, alpha=0.3) x_plot = jnp.linspace(-1, 1, 300).reshape(-1, 1) y_true = jnp.sin(2 * jnp.pi * x_plot) y_pred = jax.vmap(model)(x_plot) axes[1].scatter(X_val[:100], Y_val[:100], s=10, alpha=0.4, color="gray", label="Data") axes[1].plot(x_plot, y_true, color="#4C72B0", linewidth=2, label="True f(x)") axes[1].plot(x_plot, y_pred, color="#DD8452", linewidth=2, linestyle="--", label="Predicted") axes[1].set_xlabel("x") axes[1].set_ylabel("y") axes[1].set_title("Sine regression fit") axes[1].legend() axes[1].grid(True, alpha=0.3) plt.tight_layout() plt.savefig("equinox_tutorial.png", dpi=150) plt.show() print("\nDone! Plot saved to equinox_tutorial.png") print("\n" + "="*60) print("BONUS: eqx.filter_jit + shape inference debug tip") print("="*60) jaxpr = jax.make_jaxpr(jax.vmap(model))(x_plot) n_eqns = len(jaxpr.jaxpr.eqns) print(f"Compiled ResNetMLP jaxpr has {n_eqns} equations (ops) for batch input {x_plot.shape}") We run the complete training loop across multiple epochs, shuffle the data, process mini-batches, and track both training and validation losses over time.
Equinox presents a lightweight, JAX‑native approach to neural‑network construction, and the tutorial walks readers through its core concepts step by step. By treating eqx.Module as a PyTree, the library makes parameter handling, transformation and serialization appear simple and explicit, a design choice that the author emphasizes early on. The guide then covers static fields, filtered transforms such as filter_jit and filter_grad, and provides utilities for PyTree manipulation, all illustrated with concise code snippets.
Stateful layers, notably BatchNorm, are integrated without hidden tricks, showing how Equinox can manage mutable state alongside pure functions. An end‑to‑end training workflow caps the walkthrough, demonstrating that a full model can be built, compiled and trained within the same framework. The bonus section adds a practical debug tip: using eqx.filter_jit to expose shape‑inference information, and prints the compiled ResNetMLP jaxpr, reporting the number of equations for a given batch input.
While the tutorial showcases functional completeness, it leaves open whether these patterns scale smoothly to larger, production‑level projects.
Further Reading
- All of Equinox - Equinox Documentation
- nanoJAXGPT: A pedagogical introduction to JAX/Equinox - Hugging Face Blog
- Equinox: neural networks in JAX via callable PyTrees and filtered transformations - arXiv
Related Reading
AI Tools & Apps Claude Code 2.1.0 launches with smoother workflows, smarter agents for power users
AI Tools & Apps Build a Smart AI Voice Assistant Quickly with Vapi: Step-by-Step
AI Tools & Apps Demystifying AI Workflows: 7 Tools That Boost Transparency and Efficiency
From Archives AI tools aid North Korean hackers targeting victims without security software
From Archives Mozilla employs Anthropic's Mythos AI to locate and fix 151 Firefox bugs
Common Questions Answered
How does Equinox handle neural network module construction in JAX?
Equinox treats eqx.Module as a PyTree, which simplifies parameter handling, transformation, and serialization. By using this approach, the library makes neural network module creation more explicit and straightforward compared to traditional frameworks.
What debug technique does the tutorial demonstrate for analyzing JAX model compilation?
The tutorial shows how to use jax.make_jaxpr() to capture the JAXpr of a vectorized forward pass and count its primitive operations. This technique allows developers to inspect the computational graph and understand the number of equations generated during model compilation.
What key transformations does Equinox provide for working with neural network modules?
Equinox offers filtered transforms like filter_jit and filter_grad, which enable efficient and targeted transformations of neural network modules. These utilities allow for precise control over how computations and gradients are applied to different parts of a PyTree-based model.