Blog / Linear algebra through pictures

Linear algebra through pictures

Published on December 26, 2022
Reading time 43 minutes

This is a write-up of a talk I gave on May 27th, 2022 at the Kiddie colloquium.

The beginning#

In the beginning, the world was a formless void, and darkness was upon the face of the deep. And God said, “Let there be finite-dimensional vector spaces!”

We fix a base field $k$ throughout the talk. Each finite-dimensional vector space has its own unique color.

VWU

And God said, “Let there be time.” Time follows from top to bottom in this world, so a vector space $V$ following through time can be drawn as follows.

V

And God said, “Let there be transformations.” By transformation, of course we mean linear transformations between vector spaces. A linear transformation $T \colon V \to W$, can be thought of as a vector space $V$ changing into a vector space $W$. So we depict this as a box $T$ that $V$ flows into and $W$ flows out of. We can also compose transformations, and this has a natural picture representation.

VTWVTWSU

And God said, “Let there be interactions.” When a vector space $V$ is next to $W$, this denotes the tensor product $V \otimes_k W$. A linear map $T \colon V \otimes_k W \to U$ can then be drawn as a box that has two input legs $V, W$ and one output leg $U$. Given $T_1 \colon V_1 \to W_1$ and $T_2 \colon V_2 \to W_2$, their tensor product $T_1 \otimes T_2 \colon V_1 \otimes V_2 \to W_1 \otimes W_2$ is just $T_1$ next to $T_2$, with no interaction between the two.

VWTU

Note that there is this special vector space called $k$, which has the property that there exist canonical isomorphisms $V \otimes_k k \cong V$ for all $V$. This means that whenever there is a line with the color $k$, we can just ignore it. For instance, the following is equivalent to the picture above.

And God said, “Let there be antiparticles.” This means that particles, in other words, vector spaces, can move backwards in time! And when a vector space $V$ moves backwards in time, the should be thought of as the dual $V^\ast$ instead of $V$. The following is an example of a map $V \otimes W^\ast \to U$; note that the blue arrow is reversed.

Given a linear map $T \colon V \to W$, we know that we can represent it as $V$ flowing down, going into a box labeled $T$, and coming out as $W$. We can then dualize it and get the transpose $T^\ast \colon W^\ast \to V^\ast$. We will represent this as the same box $T$, but where $V$ flows upwards and coming out as $W$ on the top.

VTW

Since our vector spaces are finite-dimensional, there are two very special maps. Given any vector space $V$, there is a map $\id \colon k \to V \otimes V^\ast$ that basically sends $1$ to the element in $V \otimes V^\ast$ that corresponds to the identity map. There is also the evaluation/contraction map $\mathrm{ev} \colon V^\ast \otimes V \to k$.

idev

Proposition 1.

idev==evid
Proof.

It’s easiest if we work in terms of a basis. Let $e_1, \dotsc, e_n \in V$ be a basis and denote by $f_1, \dotsc, f_n \in V^\ast$ the dual basis. Then in the left diagram, we have $$ e_i \mapsto \sum_j e_j \otimes f_j \otimes e_i \mapsto \sum_j e_j \delta_{ij} = e_i, $$ and on the right diagram, we have $$ e_i \mapsto \sum_j e_i \otimes f_j \otimes e_j \mapsto \sum_j \delta_{ij} e_j = e_i. $$ These are just the identity map.

There is a pleasant consequence of this fact. Let’s suppress drawing these boxes $\mathrm{id}$ and $\mathrm{ev}$, and just think that whenever the particle decides to change time direction, it’s implicitly invoking these functions. Then given a diagram, we can perturb in an arbitrary way and will get the same linear map. In other words, we never have to worry about the time direction at all.

In a similar vein, we can slide linear transformations around.

Proposition 2.

==

God saw all that he had made, and it was very good.

The trace of an endomorphism#

Let $T \colon V \to V$ be a linear map. We can think of this as a matrix, and then take its trace. Can we interpret this in terms of these diagrams?

Proposition 3. The number $\tr(T) \in k$, interpreted as a linear map $k \to k$, is the following diagram.

T
Proof.

Again, we can use basis $e_1, \dotsc, e_n \in V$ with dual basis $f_1, \dotsc, f_n \in V^\ast$. Then we see that $1 \in k$ maps to $$ 1 \mapsto \sum_i e_i \otimes f_i \mapsto \sum_i T(e_i) \otimes f_i \mapsto \tr(T). $$ Here, we are thinking of $\tr(T)$ as the sum of the diagonal entries of the matrix representation of $T$.

From this, we immediately see that $\tr(AB) = \tr(BA)$. Here is the proof.

BA=BA=AB

It is clear from this argument that why in an expression like $\tr(ABC)$, you’re only allowed to cyclically permute the matrices and not arbitrarily. The matrices are strung together in a cycle!

Everybody knows that $\tr(AB) = \tr(BA)$, but here is something you might not have known about traces.

Theorem 4 (Taylor). Let $A_1, \dotsc, A_n \colon V \to V$ be linear maps, where $\dim V \lt n$. Then $$ \sum_{\sigma \in S_n} (-1)^{\operatorname{sgn} \sigma} \tr(A_{i_{1,1}} \dotsm A_{i_{1,j_1}}) \dotsm \tr(A_{i_{k,1}} \dotsm A_{i_{k,j_k}}) = 0, $$ where $S_n$ is the symmetric group on ${1, \dotsc, n}$, and the $i$’s are defined so that $$ \sigma = (i_{1,1} \dotsm i_{1,j_1}) \dotsm (i_{k,1} \dotsm i_{k,j_k}). $$

For instance, when $n = 3$, we are saying that $$ \tr(ABC) + \tr(ACB) + \tr(A)\tr(B)\tr(C) $$ is equal to $$ \tr(A) \tr(BC) + \tr(B) \tr(AC) + \tr(C) \tr(AB). $$

Proof.

For each permutation $\sigma$, we can define a linear map $V^{\otimes n} \to V^{\otimes n}$ that basically permutes the components. In terms of pictures, it will look like just mixing up the $n$ strings according to the permutation $\sigma$. Using this, we can draw the expression of interest as follows.

A12nσ

On the other hand, if we look at this $\sigma$ considered as an endomorphism of $V^{\otimes n}$, and add them up to get $$ \sum_{\sigma \in S_n} (-1)^{\operatorname{sgn} \sigma} \sigma \colon V^{\otimes n} \to V^{\otimes n}, $$ this factors through the $n$-th exterior power $\wedge^n V$. But because $\dim V \lt n$, this vector space is zero. Hence this endomorphism of $V^{\otimes n}$ is the zero map. This implies that the above diagram, summed up over all $\sigma \in S_n$ with signs, is zero as well.

Representation theory of finite groups#

We turn our attention to representation theory. Let $G$ be a finite group, and let $k$ be an algebraically closed field of characteristic $0$. We have the following input from representation theory.

Lemma 5 (Schur). Let $V$ be a finite-dimensional $k$-linear irreducible representation of $G$. If $T \colon V \to V$ is a linear map satisfying $g \circ T = T \circ g$ for all $g \in G$, then $T = c \cdot \id_V$ for some $c \in k$.

Let $W$ be another irreducible representation of $G$, not isomorphic to $V$. If $S \colon V \to W$ is a linear map satisfying $g \circ S = S \circ g$ for all $g \in G$, then $S = 0$.

Let’s think about what this means in terms of diagrams. Assume $X$ is a black box with one $V$ going in and another $V$ coming out. It can also have other legs, but we don’t really care. Assume that the following equation is true for all $g \in G$.

gX=gX

Then there exists $Y$ such that $X$ splits into $Y$ and $\id_V$.

X=Y

One can give a similar interpretation for the second part as well; there, if some black box $X$ satisfies a similar equivariance condition, then it has to be zero.

Using Schur’s lemma, we will prove a “master identity” that will imply most of the character formulae we care about.

Theorem 6. Let $V$ be an irreducible representation. Then the following holds.

gg-1
Proof.

We apply Schur’s lemma to the left hand side. We see that when we attach a $h \in G$ on the upper left leg, we can do a change of variables $g \mapsto gh^{-1}$, and see that this is the same as attaching $h$ on the lower left leg. Therefore we can split off this and find an $X$ such that the left hand side is equal to the following.

X

By the same argument, attaching an $h \in G$ on the upper right leg of the left hand side (in its original form) is the same as attaching it on the lower right leg. This means that attaching $h$ to the upper leg of $X$ is equal to attaching it to the lower leg of $X$. Applying the lemma again, to $X$, we can split off the diagram further.

Y

At this point, $Y$ is just a constant. So what is this constant? Let’s connect the two legs on the right. If we do this on the original diagram, the $g$ and $g^{-1}$ in each summand cancel out, so we get $\lvert G \rvert \id_V$. If we do this on the diagram right above, we get this constant $Y$ times $(\dim V) \id_V$. This shows that the constant $Y$ must be $\lvert G \rvert / \dim V$.

Similarly, we can prove the following.

Theorem 7. Let $V, W$ be two non-isomorphic irreducible representation. Then the following is true.

gg-1= 0

As a simple application, we can prove Schur orthogonality.

Corollary 8 (Schur orthogonality). If $V$ is an irreducible representation of $G$, then $$ \sum_{g \in G} \chi_V(g) \chi_V(g^{-1}) = \lvert G \rvert. $$ If $W$ is another irreducible representation of $G$, not isomorphic to $V$, then $$ \sum_{g \in G} \chi_V(g) \chi_W(g^{-1}) = 0. $$

Proof.

For the first identity, just look at the diagram

gg-1

and apply the “master identity” above. We get a factor of $\lvert G \rvert / \dim V$ and end up with one long loop, which is just $\dim V$. Therefore the result is $\lvert G \rvert$. For the second identity, do the same thing.

With this tool, we can now do the following computation for fun. Given $V$ an irreducible representation, what does $$ \sum_{a_1, b_1, \dotsc, a_g, b_g \in G} \chi_V(a_1 b_1 a_1^{-1} b_1^{-1} \dotsm a_g b_g a_g^{-1} b_g^{-1}) $$ evaluate to? I’m not going to draw out the diagram, but you can convince yourself that when we apply the “master identity” $2g$ times, we get a single long long loop. Therefore $$ \sum_{a_1, b_1, \dotsc, a_g, b_g \in G} \chi_V(a_1 b_1 a_1^{-1} b_1^{-1} \dotsm a_g b_g a_g^{-1} b_g^{-1}) = \frac{\lvert G \rvert^{2g}}{(\dim V)^{2g-1}}. $$

But wait, all characters are algebraic integers, because they are traces of matrices with roots of unities as eigenvalues. So the left hand side is an algebraic integer, and therefore so is the right hand side. This must be true for all $g$, and by studying the prime factorization, we conclude the following.

Theorem 9. Let $V$ be an irreducible representation of a finite group $G$ over an algebraically closed field of characteristic $0$. Then $\dim V$ divides the order $\lvert G \rvert$.

Towards a two-dimensional theory#

Does the previous formula remind you of a genus $g$ surface? Well, you’re not alone. So far, we’ve only talked about vector spaces, but once we add a group $G$ to the picture, there is a way of extending our universe. In this enriched universe, each time slice is not just a bunch of points, i.e., an oriented $0$-manifold, but an oriented $1$-manifold with boundary! The “open ends” of these $1$-manifolds correspond to having $G$-actions, with the action being either a left action or the right action depending on the orientation of the end. As time goes on, these $1$-manifolds will cut out an oriented $2$-manifold in spacetime. With all the right interpretations in place, what we are doing above is evaluating a genus $g$ surface minus an open disk, with $V$ going around the boundary. I won’t go into the details, but it’s certainly fun to think about different ways you can play around with this theory.