My personal website and notes
CubeCL is a Rust crate that is used for writing GPU programs in pure Rust, which is compatible with a wide range of GPU platforms. It is the underlying library powering the “ Burn ” machine learning framework. This article will look at some of the basics of how to use CubeCL and some of its quirks. You can find the example code in this repository: thomasantony/cubecl-recipes . CubeCL Topology…
The inner/dot product for Euclidean vectors is defined as: $$ \langle u, v \rangle = \sum_{i} u_i v_i $$ Following on from how we defined a norm for square integrable functions , we can similarly define an inner product for such functions in $\mathbb{L}^2$: $$ \langle f, g \rangle = \int_{-\infty}^{\infty} f(x) g(x) dx $$ This inner product can now be used to define a norm /length for our space:…
We know that functions can be treated as infinite dimensional vectors . A vector has a norm or length defined for it. How do we do that for a function? Square integrable functions Assume that we have a function: $f: \mathbb{R} \to \mathbb{C}$. It is said to be square integrable if it satisfies the following condition: $$ \int_{-\infty}^{\infty} |f(x)|^2 dx < \infty $$ This is similar to the…
Functions can be treated as infinite dimensional vectors. For example, typically a vector is interpreted as an ordered list of numbers. This could also be interpreted as a function mapping the indices to the values. E.g. a vector in $\mathbb{R}^3$ can be considered to be a function mapping the domain $1, 2, 3$ to the codomain $\mathbb{R}$ (since the values in the vector are real numbers). $$ v:…
An inner product space 1 is a real or complex vector space that has an operation called an “inner product” defined for it. The inner product of two vectors is a scalar, and is typically denoted by angle brackets like $\langle a, b\rangle$. This is a generalization of the dot product that is defined in Euclidean vector spaces ($a \cdot b = \sum a_i b_i $). The existence of an inner product also…
A Hilbert space is an inner product space that is also “complete” w.r.t the inner product - i.e. no sequence of elements in the set converges to an element outside the set. This is a generalization of the Euclidean vector space to infinite dimensions. The key intuition behind this is to treat functions as infinite dimensional vectors , which have inner products and norms just like Euclidean…
A volumetric scene represenation is a data structure or algorithmic representation of a scene - typically associated with computer vision. It can be either “implicit” or “explicit”. Some examples of scene representations are: Point Clouds Depth Maps Meshes Voxel Grids Octrees Signed Distance Functions Neural Fields NeRF - Neural Radiance Fields 3D Gaussian Splats
Volumetric rendering is an algorithm used for rendering radiance fields such as [[NeRFs]] and [[3D Gaussian Splats]] 1 2 . You start with some implicit or explicit scene representation that can be sampled at 3D positions along a ray in a scene, to return a color, $c$, and an opacity $\alpha$. $$ C = \sum_{i=1}^{N} T_i \alpha_i c_i ~\text{ where } T_i = \prod_{j=1}^{i-1} (1 - \alpha_i) $$ The…
Git worktrees allow you to check out multiple branches simultaneously without re-cloning the repository multiple times 1 . This is an alternative to using “stash” to save uncommited changes and switching branches. For example, if we are currently on branch feature/ticket-number-xxxx and want to go work on bugfix/ticket-yyyy , we can run the command git worktree add ../proj-ticket-yyyy…
Ingredients Water - 131ml 2% Ultra-Filtered Milk - 93ml All Purpose Flour - 320g Red Star Instant Yeast - 8 g Diamond Crystal salt - 6g Granulated Sugar - 15g Vital Wheat Gluten (Anthony’s) - 1.0 tbsp Kerrygold Unsalted Butter - 30g Diastatic malt (Hoosier Farms) - 10g Ascorbic acid powder - 1/8 tsp (very helpful for making the dough stronger) Equipment Stand Mixer (you could make this with a…
The 1-2-3 Rule For an IFR flight, you do not have to file an alternative if the following conditions are forecast at ETA +/- 1 hour: At least 2 000 ft ceiling At least 3 SM visibility Alternate Minimums In order to qualify as an alternate, an airport must be forecast to have at least: 600 ft ceiling and 2 SM visibility for precision approaches 800 ft ceiling and 2 SM visibility for non-precision…
Cold Start (First start of the day or Oil temp < 100F) Mixture: Full Rich Power Lever: Full Forward Fuel Pump - PRIME (2-3 seconds), then BOOST Power Lever: Open 1/4 inch Start Oil pressure check, Amp check Lean for Max RPM Hot Start (Oil temp > 100F) If the engine is warm, no priming is required. Remaining procedure is the same. Hot Start (Oil temp is nw/ 150F and 200F) Alternate technique: No…
In the event of an unexplained loss of manifold pressure in an SR22T, the engine reverts to operating like a naturally aspirated engine. However, depending on the cause of the anomaly, this could be anything from a benign to a serious event. However, since it is not possible to accurately determine the exact cause, any such event should require an immediate diversion. Induction leak This is on the…
A Schmitt trigger is a Comparator with hysteris. A typical comparator returns HIGH when the input exceeds a threshold and LOW once it is below. This can cause oscillations/noise at borderline values. A Schmitt trigger instead has two thresholds. Between the two thresholds, the output does not change. The output goes LOW only once the input goes below the lower threshold and HIGH when it goes above…
SINDy 1 is an algorithm for identifying a nonlinear dynamic model from data. The algorithm fits a dynamic model, $\dot{x} = f(\mathbf{x}(t))$, for a state vector $\mathbf{x}$. Steps: Sample $\mathbf{x}$ and $\mathbf{\dot{x}}$ at $m$ points in time into matrices $\mathbf{X}$ and $\mathbf{\dot{X}}$ . The derivative may also be approximated here using numerical methods. Compute a “library” matrix…
A fragment shader is a program that is part of a rendering pipeline 1 which takes as input a fragment (generated by rasterizing the outputs of a vertex shader ) and outputs a pixel color value and depth. 1 Fragment - OpenGL Wiki 2 Fragment Shader - OpenGL Wiki 3 Rendering Pipeline Overview - OpenGL Wiki
A vertex shader is a program that runs on the GPU that is one of the stages in a “rendering pipeline” (e.g. the OpenGL rendering pipelline 1 ). It is typically written ina special programming language such as GLSL 2 or WGSL 3 . The job of a vertex shader is to take a single vertex as input and output a single vertex (usually in 4D homogenous coordinates), in “clip-space”, after doing some…
If you want to skip to the code, you can find it here . 3D gaussian splats have been all the rage since they were published. They may end up revolutionizing how we model realistic 3D worlds and seems to be the successor to NeRFs . Most of the articles that I have seen about them addresses the training-side of it (which is arguably more important). Hardly any addresses the nitty-gritty details on…
If you want to skip to the code, you can find it here . 3D gaussian splats have been all the rage since they were published. They may end up revolutionizing how we model realistic 3D worlds and seems to be the successor to NeRFs . Most of the articles that I have seen about them addresses the training-side of it (which is arguably more important). Hardly any addresses the nitty-gritty details on…
Here are some rough notes on the “Hyena Hierarchy” architecture described in the paper 1 . This is a new way of getting sub-quadratic scaling for attention it uses convolution filter typical convolution filters are in the form of an array of values which are learned and applied like an Finite-Impulse-Response discrete filter (FIR) 2 this doesn’t scale well instead the filter parameters are…
This paper 1 describes a method for augmenting an existing language model with external memory to improve its performance without requiring any extra training. The datastore is initialized for a given dataset and used during inference time. The authors demonstrate performance improvements (measured in perplexity over a given dataset) even when operating on data-stores for datasets the model was…
The following command sends the 4-byte packet ‘\x01\x02\x03\x04’ to 127.0.0.1 on port 2222 echo -n -e '\x01\x02\x03\x04' > /dev/udp/127.0.0.1/2222
Troubleshooting If you get EXC_BAD_INSTRUCTION on M1 Mac, downgrade openssl from 3.1.0 to 3.0.7 (and Python to 3.10.9)
Train a GPT-like model to “understand langauge”. This could be based on a data-set of prompts and expected responses. Sample several outputs from the model for a given prompt. Have human labeler rank the outputs. Train yet another transformer based model (the “reward model”) that can predict this rank/“goodness” of the answer based on the human labeled answers. Stack the reward model (RM) at the…
Plasma, also called the “fourth state of matter”, consists of a collection of ions and electrons that is quasi-neutral and exhibits collective behavior. This behavior is primarily caused due to the electromagnetic interactions between the particles. The degree of interactions will depend on the average density of charged particles to neutral particles.
I have been experimenting with OpenAI’s ChatGPT since it came out a couple of days ago. You can see the transcripts of some of my conversations with it here . Today I tried to get it to generate audio, specifically music. TLDR: It can generate some notes. But maybe it can do more with careful prompting. Anyway, my musical background extends as far as trying to teach myself to play the piano and…
3 Hamiltonian Mechanics Numerical experiments are just what their name implies: experiments. In describing and evaluating them, one should enter the state of mind of the experimental physicist, rather than that of the mathematician. Numerical experiments cannot be used to prove theorems; but, from the physicist’s point of view, they do often provide convincing evidence for the existence of a…
2.12 Nonsingular Coordinates and Quaternions The Euler angles provide a convenient way to parameterize the orientation of a rigid body. However, the equations of motion derived for them have singularities. Though we can avoid the singularities by using other Euler-like combinations with different singularities, this kludge is not very satisfying. We know from Euler’s theorem that any orientation…
2.11 Spin Orbit Coupling The rotation of planets and natural satellites is affected by the gravitational forces from other celestial bodies. As an extended application of the Lagrangian method for forced rigid bodies, we consider the rotation of celestial objects subject to gravitational forces. In this section, we will develop expressions for potential energy for the graviational interaction of…
Exercise 2.13: Bicycle Wheel a. Imagine that you are holding a bicycle wheel by the axle (in both hands) and the wheel is spinning so that the top edge is going away from your face. If you torque the wheel by pushing down with your right hand and pulling up with your left hand the wheel will precess. Which way does it try to turn? $$\require{cancel}$$ Assume that the axle of the wheel forms the…
Exercise 2.12: Derivation of Euler Angle Kinematics Fill in the details of the derivation of equation (2.73). You may want to use the computer to help with the algebra. $$ \begin{pmatrix}D\theta \\ D\varphi \\ D\psi \end{pmatrix} = \frac{1}{\sin\theta} \begin{pmatrix} \cos\psi \sin\theta & -\sin\psi\sin\theta & 0 \\ \sin\psi & \cos\psi & 0 \\ -\sin\psi \cos\theta & -\cos\psi\cos\theta & \sin\theta…
Exercise 2.11: Conservation of Angular Momentum Fill in the details of the argument that Noether’s theorem implies that vector angular momentum is conserved by the motion of the free rigid body. $$\require{cancel}$$ According to Noether’s theorem , for any continuous symmetry in a system (aka a parametric family of symmetries), there is a conserved quantity. In the case of rigid bodies, rotations…
2.9 Euler’s Equations For a free rigid body we have seen that the components of the angular momentum on the principal axes comprise a self-contained dynamical system: the variation of the principal axis components depends only on the principal axis components. Here we derive equations that govern the evolution of these components. The components of angular momentum (in column matrix form) on the…
2.8.2 Qualitative Features of Free Rigid Body Motion The evolution of the components of the angular momentum on the principal axes has a remarkable property. For almost every initial condition the body components of the angular momentum periodically trace a simple closed curve. Figure 2.3: Trajectories of the components of the angular momentum vector on the principal axes, projected onto a plane.…
2.8.1 Computing the Motion of Free Rigid Bodies Lagrange’s equations for the motion of a free rigid body in terms of Euler angles are quite disgusting, so we will not show them here. Some configurations may have coordinate singularities when using Euler angles (e.g. gimbal lock ). In the explicit Lagrange equations, the singularity arises when we try to find the expression for generalized…
Exercise 2.10: Uniformly accelerated rigid body Show that a rigid body subject to a uniform acceleration rotates as a free rigid body, while the center of mass has a parabolic trajectory. $$\require{cancel}$$ The kinetic energy of the rigid body is equal to the sum of the translational and rotational kinetic energy ($T_R$). $$ T = \frac{1}{2} m \left(\dot{x}^2 + \dot{y}^2 + \dot{z}^2 \right) +…
2.8 Motion of a Free Rigid Body The kinetic energy of a rigid body, expressed in suitable generalized coordinates, is a Lagrangian for its motion. In section 2.2 we showed that the kinetic energy of a rigid body can be separated into translational and rotational kinetic energy. If we use two separate sets of coordinats to represent the translation and rotation, the Lagrangian becomes the sum of a…
Exercise 2.9: Euler Angles It is not immediately obvious that all orientations can be represented in terms of the Euler angles. To show that the Euler angles are adequate to represent all orientations, solve for the Euler angles that give an arbitrary rotation $R$. Keep in mind that some orientations do not correspond to a unique representation in terms of Euler angles. We will solve this for the…
2.7 Euler Angles Euler Angles are a set of generalized coordinates for describing the orientation of a rigid body. To quote the book: Though the Euler angles allow us to specify all orientations and thus can be used as generalized coordinates, the definition of Euler angles is pretty arbitrary. In fact no reasoning has led us to them. This is reflected in our presentation of them by just saying…
Exercise 2.5: Principal moments of inertia For each of the configurations described below find the principal moments of inertia with respect to the center of mass, and find the corresponding principal axes. a. A regular tetrahedron consisting of four equal point masses tied together with rigid massless wire. Let the sides of the tetrohedron be two units long. The coordinates of the masses are (…
Exercise 2.8: Rotational angular momentum Verify that expression in Eq. (2.50) for the components of the rotational angular momentum Eq. (2.49) in terms of the inertia tensor is correct. $$ \vec{L} = \sum_\alpha m_\alpha \vec{\xi}_\alpha \times \left(\vec{\omega} \times \vec{\xi}_\alpha \right)\tag{2.49} $$ $$ L_j = \sum_k I_{jk} \omega^k\tag{2.50} $$ Consider an arbitrary inertial coordinate…
2.6 Vector Angular Momentum The vector angular momentum of a particle is the cross product of its position vector and its linear momentum vector. For a rigid body the vector angular momentum is the sum of the vector angular momentum of each of the constituents. The vector angular momentum of a rigid body is $$ \sum_\alpha \vec{x}_\alpha \times (m_\alpha \dot{\vec{x}}_\alpha) $$ Similar to…
1.8.3 Central Forces in Three Dimensions Consider the motion of a particle in a potential field $V(r)$ in three dimensions using spherical coordinates, $r, \theta, \varphi$, where $\theta$ is the colatitude and $\varphi$ is the longitude. The kinetic energy is: $$ T(t; r, \theta, \varphi; \dot{r}, \dot{\theta}, \dot{\varphi} = \frac{1}{2} m \left(\dot{r}^2 + r^2\dot{\theta}^2 +…
Exercise 2.5: A constraint on the moments of inertia Show that the sum of any two of the moments of inertia is greater than or equal to the third moment of inertia. You may assume the moments of inertia are with respect to orthogonal axes. Consider an arbitrary orthogonal coordinate frame with axes $x$, $y$ and $z$. The inertia matrix of a body in this coordinate frame is: $$ \begin{align*} I =…
Exercise 2.4: Jupiter a. The density of a planet increases toward the center. Provide an argument that the moment of inertia of a planet is less than that of a sphere of uniform density of the same mass and radius. The higher masses at the center of the planet are scaled by the smaller distances and the smaller masses at the periphery are multiplied the larger distances. With uniform density,…
Exercise 2.3: Some useful moments of inertia Show that the moments of inertia of the following objects are as given: a. The moment of inertia of a sphere of uniform density with mass $M$ and radius $R$ about any line through the center is $\frac{2}{5} M R^2$ $$\require{cancel}$$ The moment of inertia of a body about some axis is given by the expression: $$ I_{P}=\iiint _{Q}\rho…
2.5 Principal Moments of Inertia In the previous section , we showed how the inertia tensor transforms under a rotation: $$ \mathbf{I}' = \mathbf{R}^{\mathscr{T}} \mathbf{I} \mathbf{R} \tag{2.35} $$ This transformation can be used to show that there are special rectangular coordinate frames for which the inertia tensor is diagonal, that is, $I^\prime_{ij} = 0\text{ for }i \neq j$. Let’s assume…
Exercise 2.2: Steiner’s theorem Let I be the moment of inertia of a body with respect to some given line through the center of mass. Show that the moment of inertia I′ with respect to a second line parallel to the first is $$ I' = I + M R^2\\ $$ where $M$ is the mass of the body and $R$ is the distance between the lines. $$\require{cancel}$$ The position of any particle with respect to the axis…
2.4 Inertia Tensor The representation of kinetic energy in terms of the inertia tensor involved a rectangular coordinate frame with basis vectors, $\hat{e}$. However the kinetic energy must be the same regardless of the coordinate frame use. This fact can be used to derive how the inertia tensor transforms if the body or the coordinate frame are rotated. Active and passive rotations Rotating the…
Seam carving is an algorithm that can be used to resize an image while removing the least important pixels from it. One possible implementation is to use an edge-detection algorithm first to create an edge-map 1 . This is followed by finding continuous lines that run from top to the bottom, where the only directions of movement allowed are “south”, “south-east” and “south-west”, that minimize the…