Full Jacobian using reverse-mode AD in JAX скачать в хорошем качестве

Full Jacobian using reverse-mode AD in JAX

We can use the primitive of reverse-mode automatic differentiation, the pullback (=vector-Jacobian product, vJp) for obtaining full (and dense) Jacobian matrices of vector-valued functions in the JAX deep learning framework. Often times, one does not need full Jacobian matrices in the applications of automatic differentiation. In the reverse-mode sense, their effect can almost always be given just in terms of vector-Jacobian products (=pullbacks). Still, some usecases like Gauss-Newton optimization methods (and versions thereof) need this full matrix. We can extract one row of it by querying the vJp with a particular unit vector. As a consequence, we obtain the full matrix by first getting all rows and then concatenating them together. That is what this video will show you how to do. We thereby also observe the complexity of reverse-mode AD which is linear in the number of rows, but constant in the number of columns. ------- 📝 : Check out the GitHub Repository of the channel, where I upload all the handwritten notes and source-code files (contributions are very welcome): https://github.com/Ceyron/machine-lea... 📢 : Follow me on LinkedIn or Twitter for updates on the channel and other cool Machine Learning & Simulation stuff:   / felix-koehler   and   / felix_m_koehler   💸 : If you want to support my work on the channel, you can become a Patreon here:   / mlsim   🪙: Or you can make a one-time donation via PayPal: https://www.paypal.com/paypalme/Felix... ------- Timestamps: 00:00 Intro 00:19 vector-valued function 00:37 Jacobian matrix using JAX convenience function 01:12 Recap: vector-Jacobian product (vJp) 03:30 Row-wise extraction 04:55 Jacobian function looping over the rows 07:12 Compare own implementation with JAX'