# Engel’s theorem

Before proceeding, it will be useful to recall the definition of a nilpotent Lie algebra. Let $\mathfrak{g}$ be a Lie algebra. The lower central series of $\mathfrak{g}$ is defined to be the filtration of ideals

 $\mathcal{D}_{0}\mathfrak{g}\supset\mathcal{D}_{1}\mathfrak{g}\supset\mathcal{D% }_{2}\mathfrak{g}\supset\ldots,$

where

 $\mathcal{D}_{0}\mathfrak{g}=\mathfrak{g},\qquad\mathcal{D}_{k+1}\mathfrak{g}=[% \mathfrak{g},\mathcal{D}_{k}\mathfrak{g}],\quad k\in\mathbb{N}.$

To say that $\mathfrak{g}$ is nilpotent is to say that the lower central series has a trivial termination, i.e. that there exists a $k$ such that

 $\mathcal{D}_{k}\mathfrak{g}=0,$

or equivalently, that $k$ nested bracket operations always vanish.

###### Theorem 1 (Engel)

Let $\mathfrak{g}\subset\mathop{\mathrm{End}}\nolimits V$ be a Lie algebra of endomorphisms of a finite-dimensional vector space $V$. Suppose that all elements of $\mathfrak{g}$ are nilpotent transformations. Then, $\mathfrak{g}$ is a nilpotent Lie algebra.

###### Lemma 1

Let $X:V\rightarrow V$ be a nilpotent endomorphism of a vector space $V$. Then, the adjoint action

 $\mathop{\mathrm{ad}}\nolimits(X):\mathop{\mathrm{End}}\nolimits V\rightarrow% \mathop{\mathrm{End}}\nolimits V$

is also a nilpotent endomorphism.

## Proof.

Suppose that

 $X^{k}=0$

for some $k\in\mathbb{N}$. We will show that

 $\mathop{\mathrm{ad}}\nolimits(X)^{2k-1}=0.$

Note that

 $\mathop{\mathrm{ad}}\nolimits(X)=l(X)-r(X),$

where

 $l(X),r(X):\mathop{\mathrm{End}}\nolimits V\rightarrow\mathop{\mathrm{End}}% \nolimits V,$

are the endomorphisms corresponding, respectively, to left and right multiplication by $X$. These two endomorphisms commute, and hence we can use the binomial formula to write

 $\mathop{\mathrm{ad}}\nolimits(X)^{2k-1}=\sum_{i=0}^{2k-1}(-1)^{i}\,l(X)^{2k-1-% i}\,r(X)^{i}.$

Each of terms in the above sum vanishes because

 $l(X)^{k}=r(X)^{k}=0.$

QED

###### Lemma 2

Let $\mathfrak{g}$ be as in the theorem, and suppose, in addition, that $\mathfrak{g}$ is a nilpotent Lie algebra. Then the joint kernel,

 $\ker\mathfrak{g}=\bigcap_{a\in\mathfrak{g}}\ker a,$

is non-trivial.

## Proof.

We proceed by induction on the dimension of $\mathfrak{g}$. The claim is true for dimension 1, because then $\mathfrak{g}$ is generated by a single nilpotent transformation, and all nilpotent transformations are singular.

Suppose then that the claim is true for all Lie algebras of dimension less than $n=\dim\mathfrak{g}$. We note that $\mathcal{D}_{1}\mathfrak{g}$ fits the hypotheses of the lemma, and has dimension less than $n$, because $\mathfrak{g}$ is nilpotent. Hence, by the induction hypothesis

 $V_{0}=\ker\mathcal{D}_{1}\mathfrak{g}$

is non-trivial. Now, if we restrict all actions to $V_{0}$, we obtain a representation of $\mathfrak{g}$ by abelian transformations. This is because for all $a,b\in\mathfrak{g}$ and $v\in V_{0}$ we have

 $abv-bav=[a,b]v=0.$

Now a finite number of mutually commuting linear endomorphisms admits a mutual eigenspace decomposition. In particular, if all of the commuting endomorphisms are singular, their joint kernel will be non-trivial. We apply this result to a basis of $\mathfrak{g}/\mathcal{D}_{1}\mathfrak{g}$ acting on $V_{0}$, and the desired conclusion follows. QED

## Proof of the theorem.

We proceed by induction on the dimension of $\mathfrak{g}$. The theorem is true in dimension 1, because in that circumstance $\mathcal{D}_{1}\mathfrak{g}$ is trivial.

Next, suppose that the theorem holds for all Lie algebras of dimension less than $n=\dim\mathfrak{g}$. Let $\mathfrak{h}\subset\mathfrak{g}$ be a properly contained subalgebra of minimum codimension. We claim that there exists an $a\in\mathfrak{g}$ but not in $\mathfrak{h}$ such that $[a,\mathfrak{h}]\subset\mathfrak{h}$.

By the induction hypothesis, $\mathfrak{h}$ is nilpotent. To prove the claim consider the isotropy representation of $\mathfrak{h}$ on $\mathfrak{g}/\mathfrak{h}$. By Lemma 1, the action of each $a\in\mathfrak{h}$ on $\mathfrak{g}/\mathfrak{h}$ is a nilpotent endomorphism. Hence, we can apply Lemma 2 to deduce that the joint kernel of all these actions is non-trivial, i.e. there exists a $a\in\mathfrak{g}$ but not in $\mathfrak{h}$ such that

 $[b,a]\equiv 0\mod\mathfrak{h},$

for all $b\in\mathfrak{h}$. Equivalently, $[\mathfrak{h},a]\subset\mathfrak{h}$ and the claim is proved.

Evidently then, the span of $a$ and $\mathfrak{h}$ is a subalgebra of $\mathfrak{g}$. Since $\mathfrak{h}$ has minimum codimension, we infer that $\mathfrak{h}$ and $a$ span all of $\mathfrak{g}$, and that

 $\mathcal{D}_{1}\mathfrak{g}\subset\mathfrak{h}.$ (1)

Next, we claim that all the $\mathcal{D}_{k}\mathfrak{h}$ are ideals of $\mathfrak{g}$. It is enough to show that

 $[a,\mathcal{D}_{k}\mathfrak{h}]\subset\mathcal{D}_{k}\mathfrak{h}.$

We argue by induction on $k$. Suppose the claim is true for some $k$. Let $b\in\mathfrak{h},c\in\mathcal{D}_{k}\mathfrak{h}$ be given. By the Jacobi identity

 $[a,[b,c]]=[[a,b],c]+[b,[a,c]].$

The first term on the right hand-side in $\mathcal{D}_{k+1}\mathfrak{h}$ because $[a,b]\in\mathfrak{h}$. The second term is in $\mathcal{D}_{k+1}\mathfrak{h}$ by the induction hypothesis. In this way the claim is established.

Now $a$ is nilpotent, and hence by Lemma 1,

 $\mathop{\mathrm{ad}}\nolimits(a)^{n}=0$ (2)

for some $n\in\mathbb{N}$. We now claim that

 $\mathcal{D}_{n+1}\mathfrak{g}\subset\mathcal{D}_{1}\mathfrak{h}.$

By (1) it suffices to show that

 $[\overbrace{\mathfrak{g},[\ldots[\mathfrak{g}}^{n\text{ times}},\mathfrak{h}]% \ldots]]\subset\mathcal{D}_{1}\mathfrak{h}.$

Putting

 $\mathfrak{g}_{1}=\mathfrak{g}/\mathcal{D}_{1}\mathfrak{h},\quad\mathfrak{h}_{1% }=\mathfrak{h}/\mathcal{D}_{1}\mathfrak{h},$

this is equivalent to

 $[\overbrace{\mathfrak{g}_{1},[\ldots[\mathfrak{g}_{1}}^{n\text{ times}},% \mathfrak{h}_{1}]\ldots]]=0.$

However, $\mathfrak{h}_{1}$ is abelian, and hence, the above follows directly from (2).

Adapting this argument in the obvious fashion we can show that

 $\mathcal{D}_{kn+1}\mathfrak{g}\subset\mathcal{D}_{k}\mathfrak{h}.$

Since $\mathfrak{h}$ is nilpotent, $\mathfrak{g}$ must be nilpotent as well. QED

## Historical remark.

In the traditional formulation of Engel’s theorem, the hypotheses are the same, but the conclusion is that there exists a basis $B$ of $V$, such that all elements of $\mathfrak{g}$ are represented by nilpotent matrices relative to $B$.

Let us put this another way. The vector space of nilpotent matrices $\mathop{\mathrm{Nil}}\nolimits$, is a nilpotent Lie algebra, and indeed all subalgebras of $\mathop{\mathrm{Nil}}\nolimits$ are nilpotent Lie algebras. Engel’s theorem asserts that the converse holds, i.e. if all elements of a Lie algebra $\mathfrak{g}$ are nilpotent transformations, then $\mathfrak{g}$ is isomorphic to a subalgebra of $\mathop{\mathrm{Nil}}\nolimits$.

The classical result follows straightforwardly from our version of the Theorem and from Lemma 2. Indeed, let $V_{1}$ be the joint kernel $\mathfrak{g}$. We then let $U_{2}$ be the joint kernel of $\mathfrak{g}$ acting on $V/V_{0}$, and let $V_{2}\subset V$ be the subspace obtained by pulling $U_{2}x$ back to $V$. We do this a finite number of times and obtain a flag of subspaces

 $0=V_{0}\subset V_{1}\subset V_{2}\subset\ldots\subset V_{n}=V,$

such that

 $\mathfrak{g}V_{k+1}=V_{k}$

for all $k$. The choose an adapted basis relative to this flag, and we’re done.

Title Engel’s theorem EngelsTheorem 2013-03-22 12:42:25 2013-03-22 12:42:25 rmilson (146) rmilson (146) 5 rmilson (146) Theorem msc 17B30 msc 15A57