@@ -54,7 +54,6 @@ tags: [hide-output]
5454We will use the following imports.
55555656```{code-cell} ipython3
57-import random
5857from functools import partial
59586059import numpy as np
@@ -483,18 +482,67 @@ Notice that the second run is significantly faster after JIT compilation complet
483482484483Numba's compilation is typically quite fast, and the resulting code performance is excellent for sequential operations like this one.
485484485+486486### JAX Version
487487488-Now let's create a JAX version using `lax.scan`:
488+Now let's create a JAX version using `at[t].set` style syntax, which, as
489+{ref}`discussed in the JAX lecture <jax_at_workaround>`, provides a workaround for immutable arrays.
489490490-(We'll hold `n` static because it affects array size and hence JAX wants to
491-specialize on its value in the compiled code.)
491+We'll apply a `lax.fori_loop`, which is a version of a for loop that can be compiled by XLA.
492492493493```{code-cell} ipython3
494494cpu = jax.devices("cpu")[0]
495495496-@partial(jax.jit, static_argnames=('n',), device=cpu)
497-def qm_jax(x0, n, α=4.0):
496+@partial(jax.jit, static_argnames=("n",), device=cpu)
497+def qm_jax_fori(x0, n, α=4.0):
498+499+ x = jnp.empty(n + 1).at[0].set(x0)
500+501+ def update(t, x):
502+ return x.at[t + 1].set(α * x[t] * (1 - x[t]))
503+504+ x = lax.fori_loop(0, n, update, x)
505+ return x
506+507+```
508+509+* We hold `n` static because it affects array size and hence JAX wants to specialize on its value in the compiled code.
510+* We pin to the CPU via `device=cpu` because this sequential workload consists of many small operations, leaving little opportunity for GPU parallelism.
511+512+Although `at[t].set` appears to create a new array at each step, inside a JIT-compiled function the compiler detects that the old array is no longer needed and performs the update in place.
513+514+Let's time it with the same parameters:
515+516+```{code-cell} ipython3
517+with qe.Timer():
518+ # First run
519+ x_jax = qm_jax_fori(0.1, n)
520+ # Hold interpreter
521+ x_jax.block_until_ready()
522+```
523+524+Let's run it again to eliminate compilation overhead:
525+526+```{code-cell} ipython3
527+with qe.Timer():
528+ # Second run
529+ x_jax = qm_jax_fori(0.1, n)
530+ # Hold interpreter
531+ x_jax.block_until_ready()
532+```
533+534+JAX is also quite efficient for this sequential operation.
535+536+537+There's another way we can implement the loop that uses `lax.scan`.
538+539+This alternative is arguably more in line with JAX's functional approach ---
540+although the syntax is difficult to remember.
541+542+543+```{code-cell} ipython3
544+@partial(jax.jit, static_argnames=("n",), device=cpu)
545+def qm_jax_scan(x0, n, α=4.0):
498546 def update(x, t):
499547 x_new = α * x * (1 - x)
500548 return x_new, x_new
@@ -505,20 +553,12 @@ def qm_jax(x0, n, α=4.0):
505553506554This code is not easy to read but, in essence, `lax.scan` repeatedly calls `update` and accumulates the returns `x_new` into an array.
507555508-```{note}
509-We specify `device=cpu` in the `jax.jit` decorator because this computation
510-consists of many small sequential operations, leaving little opportunity for the
511-GPU to exploit parallelism. As a result, kernel-launch overhead tends to
512-dominate on the GPU, making the CPU a better
513-fit.
514-```
515-516556Let's time it with the same parameters:
517557518558```{code-cell} ipython3
519559with qe.Timer():
520560 # First run
521- x_jax = qm_jax(0.1, n)
561+ x_jax = qm_jax_scan(0.1, n)
522562 # Hold interpreter
523563 x_jax.block_until_ready()
524564```
@@ -528,13 +568,11 @@ Let's run it again to eliminate compilation overhead:
528568```{code-cell} ipython3
529569with qe.Timer():
530570 # Second run
531- x_jax = qm_jax(0.1, n)
571+ x_jax = qm_jax_scan(0.1, n)
532572 # Hold interpreter
533573 x_jax.block_until_ready()
534574```
535575536-JAX is also quite efficient for this sequential operation.
537-538576Both JAX and Numba deliver strong performance after compilation.
539577540578@@ -547,9 +585,11 @@ array and fill it element by element using a standard Python loop.
547585548586This is exactly how most programmers think about the algorithm.
549587550-The JAX version, on the other hand, requires using `lax.scan`, which is significantly less intuitive.
588+The JAX versions, on the other hand, require either `lax.fori_loop` or
589+`lax.scan`, both of which are less intuitive than a standard Python loop.
551590552-Additionally, JAX's immutable arrays mean we cannot simply update array elements in place, making it hard to directly replicate the algorithm used by Numba.
591+While JAX's `at[t].set` syntax does allow element-wise updates, the overall code
592+remains harder to read than the Numba equivalent.
553593554594For this type of sequential operation, Numba is the clear winner in terms of
555595code clarity and ease of implementation.
@@ -575,12 +615,12 @@ For **sequential operations**, Numba has clear advantages.
575615The code is natural and readable --- just a Python loop with a decorator ---
576616and performance is excellent.
577617578-JAX can handle sequential problems via `lax.scan`, but the syntax is less
579-intuitive.
618+JAX can handle sequential problems via `lax.fori_loop` or `lax.scan`, but
619+the syntax is less intuitive.
580620581621```{note}
582-One important advantage of `lax.scan` is that it supports automatic
583-differentiation through the loop, which Numba cannot do.
622+One important advantage of `lax.fori_loop` and `lax.scan` is that they
623+support automatic differentiation through the loop, which Numba cannot do.
584624If you need to differentiate through a sequential computation (e.g., computing
585625sensitivities of a trajectory to model parameters), JAX is the better choice
586626despite the less natural syntax.