Menu

Post image 1
Post image 2
1 / 2
0

PyTrees Are Not One Thing: JAX, PyTorch, and TensorFlow Compared

DEV Community: tensorflow·Shixin Zhang·3 months ago
#UTyi4m2u
#dev#fullscreen#tree#enter#exit#article
Reading 0:00
15s threshold

PyTrees look deceptively simple. You flatten a nested Python object into leaves, keep a structure descriptor, and later rebuild or map over the same shape. That abstraction is powerful enough to carry optimizer states, model parameters, batched inputs, gradients, and sharding annotations. It is also just ambiguous enough that three major frameworks implement three subtly different languages under the same idea. This note compares JAX jax.tree_util , PyTorch torch.utils._pytree , and TensorFlow tf.nest . I tested the behavior in two environments: an older stack with JAX 0.4.35, PyTorch 2.2.2, TensorFlow 2.20.0, and a newer stack with JAX 0.10.0, PyTorch 2.12.0, TensorFlow 2.21.0. Most flatten/unflatten semantics were stable across these versions. The main version-sensitive result is PyTorch: _pytree.tree_map in 2.2.2 accepts only one pytree, while 2.12.0 supports multiple pytrees and behaves much closer to JAX prefix-style mapping.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More