Math, Language, Programming
In this post (the 2nd in this series on Tensors and Language Models ), we will be interested in collections of tables or dataframes that look something like: We will call these databases . This example database comes from the paper on quantifying the knowledge capacity of attention layers . By the end of this post, we will see how to define a tensor associated to a database. But first, we will…
This is the first in a series of posts explaining and expanding on the ideas introduced in the following papers: Paying attention to facts: quantifying the knowledge capacity of attention layers [ arXiv:2502.05076 ] ‘Generalization is hallucination’ through the lens of tensor completions [ arXiv:2502.17305 ] The overarching theme behind these papers is that tensors and tensor completions are a…
I’ve been participating in the Kan Extension Seminar II , and this week it’s my turn to post about Jon Beck’s “Distributive Laws” at the n-Category Cafe ! The post uses lots of string diagrams for monads, resulting in pictures like the following: See you there !
In this post, I’ll demonstrate 3 ways to define non-commutative rings in Sage. They’re essentially different ways of expressing the non-commutative relations in the ring: Via g_algebra : define the relations directly Via NCPolynomialRing_plural : define a pair of structural matrices Via a quotient of a letterplace ring : define the ideal generated by the relations (only works for homogeneous…
I’ve been away from this blog for quite a while - almost a year, in fact! My excuses are my wedding and the prelims (a.k.a. quals), as well as all the preparation that had to go into them (although, to be honest, those things only occupied me till September last year!). Looking back at my previous posts, I’ve realized that in attempting to teach both math and code, I probably ended up doing…
This post illustrates some of SageMath’s character theory functionality, as well as some basic results about characters of finite groups. Basic Definitions and Properties Given a representation $(V,\rho)$ of a group $G$, its character is a map $ \chi: G \to \mathbb{C}$ that returns the trace of the matrices given by $\rho$: \[\chi(g) = \text{trace}(\rho(g)).\] A character $\chi$ is irreducible if…
I really should be posting about character theory, but I got distracted making some aesthetic changes to this blog (new icon and favicon!) and creating animations like this: I’m not putting this in a SageCell because this could take quite a while, especially if you increase the number of frames (by changing the parameters in srange ), but feel free to try it out on your own copy of Sage. It saves…
In the previous post , we saw how to decompose a given group representation into irreducibles. But we still don’t know much about the irreducible representations of a (finite) group. What do they look like? How many are there? Infinitely many? In this post, we’ll construct the group ring of a group. Treating this as a vector space, we get the regular representation , which turns out to contain all…
In this post, we’ll implement an algorithm for decomposing representations that Dixon published in 1970 . As a motivating example, I’ll use the permutation matrix representation of $D_4$ that we saw in an earlier post . To make the code more generally applicable, let’s call the group $G$ and the representation $\rho$: (The Sage cells in this post are linked, so things may not work if you don’t…