A Relation Is a Rotation

This is the fourth post in Knowledge Graphs as Geometry, a weekly series that turns the model zoo into one idea. So far a relation has been a verb that moves a head toward a tail: TransE made it a step, DistMult and ComplEx made it a stretch. This week the verb becomes a turn and a relation that was forced to collapse finally gets to be itself.

Go back to the family graph (https://agussudjianto.substack.com/p/a-relation-is-an-operation). Ann is married to Bob, Carol is married to Eve, and Ann and Bob parent Carol and Dave. We have a problem with spouse that has followed us for three chapters, and it is time to fix it.

The problem is symmetry. If Ann is married to Bob, then Bob is married to Ann — the relation is its own reverse. Back in the TransE post we saw what that does to a translation. If spouse is a fixed step 𝐫, then Ann + 𝐫 ≈ Bob and Bob + 𝐫 ≈ Ann, and adding the two forces 𝐫 ≈ 0. The step has to be no step at all. A translation can only be symmetric by being nothing, which makes every entity its own spouse. The geometry refuses to say what we mean.

There is a shape that is its own reverse without being nothing: a half-turn. Spin the plane by 180°, do it again, and you are back where you started. The turn is genuine — it moves points, Ann is not Bob — but applying it twice is the identity, so Ann→Bob and Bob→Ann are the same operation read in two directions. That is exactly the property spouse needs, and a translation can never have it, because doing a step twice gives you two steps, not zero. So the fix is not to patch the translation or add a correction term. It is to change what the relation does at the root. Make the relation a rotation, and symmetry stops being a trap and becomes a natural special case — the case where the angle is π.

We do not have to invent rotation from scratch. We already have it sitting inside ComplEx from last week (https://agussudjianto.substack.com/p/a-relation-in-knowledge-graph-is). ComplEx put each entity in the complex plane and let the relation multiply, channel by channel — a complex number rₖ acting on a complex coordinate hₖ. A general complex number both scales and turns. RotatE makes one restriction: force every relation channel onto the unit circle, so |rₖ| = 1. A unit-modulus complex number is a pure phase, rₖ = eⁱᶿᵏ, and multiplying by it rotates the plane by θₖ without changing any length.

That single constraint is the whole model. The relation carries the head to a predicted tail by rotating it,

and we score a candidate fact by how far the true tail lands from that prediction:

Read against the general operator this whole series is building, Fᵣ(𝐡) = Uᵣ Σᵣ Vᵣᵀ 𝐡 + cᵣ, RotatE is the case where the stretch Σᵣ is gone — every singular value pinned to 1 — and there is no shift. The operator is pure orthogonal turn. Compare it to ComplEx and the bookkeeping is striking: same carrier (the complex plane), same operator (channelwise product), same kind of relation object (a complex vector). Two things change, and only two. The relation is restricted to phases, and the comparator is a distance instead of a matching inner product. ComplEx is “complex carrier + product + matching.” RotatE is “complex carrier + unit product + distance.” Same operator, read two ways — which is exactly why we insisted on naming the comparator as its own separate choice.

Why distance, and not the matching score ComplEx used? Because a unit-modulus relation is an isometry — it preserves length. Rotating the head and then measuring the distance to the tail behaves precisely like TransE’s distance decoder, with a rotation where TransE had a translation. RotatE is, in one phrase, TransE done multiplicatively: a rigid motion of the embedding space, scored by how close you land.

The claim is that rotation lets a symmetric relation be its own inverse without vanishing. We can read it straight off a trained model. Train RotatE on the family graph — four complex dimensions, a couple thousand steps — and then ask two questions of every learned relation.

First, how big is each relation? The natural measure is the modulus of its channels, mean |rₖ|. By construction every channel sits on the unit circle, so this should be 1 for every relation, symmetric or not. A RotatE relation cannot shrink to zero the way TransE’s step had to.

Second, is the relation its own inverse? A rotation is symmetric exactly when applying it twice gives the identity, 𝐫 ⊙ 𝐫 = 𝟏. We measure the involution defect ‖𝐫 ⊙ 𝐫 − 𝟏‖ — how far the relation is from being its own undo. For a symmetric relation it should be near zero.

The numbers tell the story cleanly. spouse and siblingOf — the two symmetric relations in the graph — have involution defect 0.000: they are exact half-turns, do-it-twice-and-return. parentOf, which is directional, has defect 3.203: it is a genuine rotation that is not its own inverse, which is correct, because being a parent is not the same as being a child. And every mean modulus is 1.000. The defect is not a free-floating number, either — on the unit circle two channels can disagree with the identity by at most a fixed amount per channel, so a defect of 3.203 across four complex dimensions means the model has pushed parentOf close to the far end of “as far from its own inverse as it can be.” It found the most asymmetric rotation the geometry allows, which is the right answer for a relation that should never read the same in both directions. Symmetry got represented, asymmetry got represented, and the relation never collapsed. That is the trap from the TransE post, sprung.

The half-turn is not just an internal property; it shows up in the scores. Ask the model to score spouse in both directions and the two numbers are identical — Ann→Bob and Bob→Ann score 1.021, Carol→Eve and Eve→Carol score 0.269. The relation reads the same forward and backward, which is what symmetry should mean, and it does so while still being a real motion of the space. Contrast that with parentOf: there the model should not score both directions alike, because Ann is Carol’s parent but Carol is not Ann’s, and the defect of 3.203 is precisely the model refusing to make that relation symmetric. The geometry encodes the asymmetry and the symmetry in the same currency — a single angle per channel — and lets the data decide which each relation gets.

Rotation does not just rescue symmetry. It hands us a clean account of the other relation patterns too, and it is worth walking through them because each one is a small story about angles.

Start with inverses. Every rotation has an undo — turn by −θ — so RotatE represents the inverse of a relation by the complex conjugate, 𝐫⁻¹ = 𝐫̄. If parentOf is a turn by θ, then childOf is the turn by −θ, and the model gets the pair for free without learning them separately. TransE had this too, with −𝐫, but TransE could not also have symmetry; RotatE has both at once, because a half-turn (θ = π) happens to be its own negation modulo a full turn. The same parameter does double duty.

Now composition. Rotations chain by adding angles: do θ then φ and you have turned by θ + φ. So RotatE can learn that parentOf composed with parentOf is grandparentOf, because the phases simply add — θ_grandparent = 2·θ_parent — and the model can discover that relationship from the data rather than being told it. Symmetry, inversion, composition: the rotation handles all three with the same arithmetic of angles, which is a remarkable amount of structure to get from one constraint.

But there is a ceiling, and it is structural. Angles add, and addition does not care about order: θ + φ = φ + θ. So in the complex plane, applying relation r then s is the same as s then r. RotatE forces every pair of relations to commute. That is fine for grandparentOf, but it is wrong for relations where order genuinely matters — “your parent’s sibling” is your aunt or uncle, “your sibling’s parent” is your own parent. A model that makes everything commute cannot tell those two apart.

We can see the wall directly. Take two RotatE relations and compute the commutator ‖𝐫 ⊙ 𝐬 − 𝐬 ⊙ 𝐫‖ — how much the order changes the result. For complex relations it is exactly zero.

Zero, to floating-point. Complex relations commute as a matter of algebra, not training. If we want order to matter, we need a carrier whose multiplication itself depends on order.

The fix is to climb one more rung. We went from real numbers (DistMult) to complex numbers (ComplEx, RotatE); the next step is the quaternions, ℍ. A quaternion has four real components and multiplies by the Hamilton product, and the Hamilton product is famously order-dependent. The basis units obey i·j = k but j·i = −k. Swap the order and you flip a sign. QuatE keeps the exact same template — a unit relation acting channelwise on the carrier — and just swaps in this multiplication table. That one change brings the property the complex models structurally lack.

Look at the same commutator for QuatE in the figure above: 3.281, decisively nonzero. And the source of it is visible in the basis units themselves — i∘j = (0, 0, 0, 1) while j∘i = (0, 0, 0, −1), so i∘j = −(j∘i). Flip the order of the two units and the result flips sign. Order matters in the carrier, so it can matter in the relations. QuatE can keep “parent’s sibling” and “sibling’s parent” apart, a composition no complex model can represent.

The price is four real numbers per channel instead of two, and a multiplication table that no longer matches everyday intuition. Whether that price is worth paying depends on the graph. A graph whose relations genuinely chain in an order-dependent way — kinship, spatial direction, process steps — has structure that RotatE provably cannot fit and QuatE can. A graph dominated by symmetry and simple inverses gets little from the extra components, and the smaller model trains faster and overfits less. The geometry tells you which regime you are in before you spend a single epoch: count whether the relations you care about commute.

There is a thread here worth pulling, though we save the knot for later in the series. Real, complex, quaternion is not three unrelated tricks — it is the first three rungs of a single ladder. DistMult lived on the reals and could only stretch. ComplEx and RotatE moved to the complex plane and could turn. QuatE moves to the quaternions and can turn in a way that remembers order. Each rung adds exactly one capability, and each one is the same operator template with a richer multiplication table underneath.

In the geometric-algebra language the book uses as its design language, ℂ and ℍ are small Clifford algebras, and the noncommutativity that makes QuatE work is the same noncommutativity that makes general rotations work in any dimension. That is the lens; it is not the implementation. The algebra is how we see that these models are one family — it tells us where the next rung is and why it has the property it has. But the computation stays plain channelwise multiplication, batched on the GPU through PyTorch’s ordinary kernels, with no geometric-algebra engine at run time. Design language above, linear algebra underneath. We make no claim that the algebra is necessary or that it computes anything faster; the speed is the matrix representation’s, and the algebra’s job is to organize the zoo, not to run it.

One modeling choice inside RotatE looks arbitrary and is not: scoring by the ordinary ℓ₂ distance. It pays off because a unit-modulus relation is orthogonal, and an orthogonal map can be moved to the other side of a distance for free. Concretely,

Both arguments inside the norm get rotated by 𝐫⁻¹, the length is preserved, and the relation lands on the tail instead of the head. We check it on random vectors in the notebook and it holds to floating precision — ‖𝐡 ⊙ 𝐫 − 𝐭‖ = ‖𝐡 − 𝐭 ⊙ 𝐫⁻¹‖ = 5.2867 on both sides.

The payoff is practical and large. To rank every head against a query (r, t), the naive route rotates all |ℰ| candidate entities — one rotation per candidate. The identity replaces that with a single rotation of the tail, 𝐭 ⊙ 𝐫⁻¹, followed by one distance computation against the fixed entity table. On WN18RR, with about 40,000 entities and dimension 200, that is the difference between forty thousand small rotations and one, repeated for every query in the test set. The rotation work per query drops by a factor of |ℰ| — four orders of magnitude on this graph — and what remains is a single distance against a table that never moves, so it can be precomputed once and reused across the whole evaluation. That last point is what makes it fast in practice and not just on paper: an all-pairs distance against a fixed matrix is exactly the dense kernel a GPU is built to saturate, while per-candidate rotation is a scatter of tiny operations that leaves the hardware idle.

This is the same move that made TransE’s evaluation cheap, and it is not a coincidence that it reappears: both models score by distance after an isometry, and an isometry is precisely the thing you can slide across a norm. It is also the identity the unified operator later in the book leans on for exact head prediction. There is a catch worth naming. The original RotatE paper scored with an ℓ₁ sum of per-channel moduli rather than the ℓ₂ norm, and the ℓ₁ version is not rotation-invariant in this way — it forfeits the identity and needs a chunked per-candidate computation. The library keeps the ℓ₁ variant so it can reproduce published benchmarks, but defaults to ℓ₂ because that is where the isometry structure actually pays off. The geometry and the speed are the same fact seen twice.

Mechanics on six people are convincing about what the model can represent. To see that it helps, we need a real graph. UMLS is a small medical knowledge graph — 135 entities, 46 relations, a few thousand facts — that trains in seconds on a laptop CPU. Train RotatE and TransE on it under identical settings and compare where each one wins.

The pattern is the point. Back in the TransE post we saw its signature: a steep climb from Hits@1 to Hits@3, meaning it found the right neighborhood but could not pick the single answer out of the cluster it had to collapse symmetric and many-to-many relations into. RotatE shows the opposite fingerprint. Because rotation does not collapse anything, it resolves that last step. On UMLS, RotatE lifts Hits@1 from TransE’s 0.535 to 0.686 — it separates the answers translation had squashed together — while both already saturate at Hits@10 (0.995 versus 0.992). The two models agree about which neighborhood the answer lives in; they disagree about whether you can name the single right entity, and that is exactly the gap rotation closes.

The same effect holds at scale, where the gap is far wider. On WN18RR (about 40,000 entities, trained on GPU and reported here as a cached single-seed result, set_seed(0)) RotatE reaches MRR 0.444 with Hits@1 0.416, against a TransE Hits@1 of about 0.01. That is not a typo. On WN18RR translation gets the single best answer right roughly one time in a hundred, while rotation gets it right two times in five — the steepest separation we will see in the whole series, and it falls exactly where the theory says it should. WN18RR is dense with symmetric and inverse relations — similar_to, derivationally_related_form, the hypernym/hyponym pairs — exactly the structure that forces TransE to collapse and lets rotation shine. The two models even land within a hair of each other on Hits@10 (TransE 0.499, RotatE 0.496): both find the right neighborhood, but only rotation can name the one entity inside it. The geometry that fixed spouse on six people is the same geometry that lifts the hardest metric on forty thousand. Nothing about the model changed between the family graph and WN18RR; only the size did.

The whole rescue is a few lines: train RotatE on the family graph, then read the involution defect of each relation.

from kge.data import TripleFactory
from kge.models import RotatE
from kge.train import Trainer, TrainConfig
from kge.carrier import COMPLEX
import torch
fam = TripleFactory(train=family_triples, valid=[], test=[], name="family")
model = RotatE(fam.n_entities, fam.n_relations, dim=4, margin=6.0)
Trainer(model, fam, TrainConfig(mode="1vsall", epochs=2000, lr=0.1), device="cpu").fit()
one = torch.zeros(model.dim, COMPLEX.components); one[:, 0] = 1.0   # identity rotation
for rel, i in fam.rel2id.items():
    r = model.relation_element(torch.tensor([i]))[0]               # |r_k| = 1
    defect = (model.compose_elements(r, r) - one).norm().item()    # ||r o r - 1||
    print(f"{rel:10s} involution defect {defect:.3f}")             # ~0 for spouse, siblingOf

The companion notebook trains RotatE on the family graph, measures the involution defect, checks both commutators (complex zero, quaternion nonzero, with i∘j = −j∘i), verifies the isometry identity to floating precision, and trains RotatE against TransE on UMLS to show the Hits@1 lift.

▶ Run the notebook in Colab: https://colab.research.google.com/github/asudjianto-xml/Knowledge-Graph-Geometry/blob/main/notebooks/ch04_rotational_models.ipynb
📦 Code: github.com/asudjianto-xml/Knowledge-Graph-Geometry — pip install "kge-geometric @ git+https://github.com/asudjianto-xml/Knowledge-Graph-Geometry.git"

A relation can be a turn, not just a step. Constrain the complex relation to the unit circle and it acts by pure rotation: symmetry becomes a half-turn instead of a collapse, inverses become conjugates, composition becomes added angles. That last one is also the ceiling — added angles commute, so RotatE makes every relation order-blind, and only the quaternion’s noncommutative product breaks the cap. With this the diagonal zoo is complete, and every member — DistMult, ComplEx, RotatE, QuatE — is one operator seen through a different carrier and a different comparator. Next week we name the three choices behind all of them, and the lens snaps into focus.

Knowledge Graphs as Geometry is a free weekly series adapted from my book Knowledge Graph Embeddings as Geometric Operators. The posts carry the intuition and the runnable code; the book carries the full derivations. Subscribe to follow the whole argument — from a single translation to one operator that contains the entire model zoo.