Today we will solve a problem from Wisconsin’s 2012 Algebra Qualifying Exam. The main tools used in the problem are Sylow Theory and group actions; we will take these tools for granted, but for great introductions to each topic I would recommend Keith Conrad’s notes here and here. The solution as written is not the cleanest; instead I have tried to add in some of my thoughts when solving the problem to highlight why the solution is (somewhat) natural to stumble upon. The problem is in three parts:
Problem: Let be a finite group of order .

Show that has a subgroup of order .

Show that has a subgroup of order whose normalizer has index dividing .

Conclude that is not simple.
Let’s do this. A natural place to start the first part is with Sylow Theory. Since , we might try to show that a Sylow11 subgroup is unique and hence normal, and then form a product subgroup with an element of order to get the desired subgroup.
Thus let be the number of Sylow11 subgroups. By the third Sylow Theorem, and divides . By simply checking all possibilities, we conclude . If , then there is a unique Sylow11 subgroup which is normal. We may then take a subgroup of size (which exists by Cauchy’s theorem or by the extended first Sylow Theorem), form a product subgroup, and create a subgroup of size . Otherwise, we have . In this case we note that (miraculously) . Furthermore, the second Sylow Theorem tells us that all Sylow11 subgroups are conjugate to one another; hence we consider the action of on a Sylow11 subgroup via conjugation. The orbit of this action has size , and thus the OrbitStabilizer Theorem tells us that the stabilizer subgroup has size as desired. Thus in either case we have found a subgroup of size 77, and we move to part 2. (NB: The fact that there are possibly Sylow11 subgroups and that is really just a happy accident that makes for a good test problem; it would be hard to plan for this from the outset, and hence this is somewhat of a “black magic” solution; however, it does follow quite organically from the consideration of Sylow Theory (and highlights the power of the theory) which is partly why I like this problem so much).
For part two, we need a subgroup of size with a large normalizer. Recall the definition of the normalizer of a subgroup as
Given that we just found a subgroup of size , the obvious choice is to explore this subgroup a little bit more. Now has size ; a simple application of the third Sylow Theorem shows that has a unique Sylow7 subgroup , and hence is normal in (in fact, one may show that must be cyclic of order by pursuing the Sylow Theory a little bit more, but that is irrelevant to the current problem). So we have a chain of subgroups
.
Thus we already know divides . Now is a group of size . Again by Sylow Theory we know that must be contained in a Sylow7 subgroup, say . Now ; a simple fact of groups is that groups of order are abelian, and hence must also normalize (in fact it centralizes it, but again, not important). So we also have . So is divisible by both and , and hence divides . Looking at indices, we get divides , as desired.
For the last part we are supposed to conclude the is not simple. To do this, we must find a normal subgroup contained in . The easiest normal subgroups to find in groups are kernels of homomorphisms; given that this is the last part of a problem, we should try to use the previous part to construct some sort of homomorphism with a kernel and we will be done.
Let us take the normalizer subgroup which we found above. There are two natural homomorphisms we can consider when given a subgroup such as , both arising from natural group actions. One is given by conjugation on , and the other is the multiplication action on cosets of . Since we have no idea about the number of subgroups which are conjugate to , but we do know that there are relatively few cosets of , let’s try the action on the cosets. Say that , so that there are cosets of (the proof with fewer cosets is identical). Then the action via left multiplication of on the set of cosets gives a map , the symmetric group on symbols. If this map was injective then we could identify with a subgroup of . But divides , and does not divide ; hence we would arrive at a contradiction. So this map cannot be injective, and hence must have a nontrivial kernel. Thus we are done, right? Well, not quite. It could be that the entire homomorphism was trivial, and hence the kernel is itself. But this means that must be equal to (why?). In particular, the normalizer of our original subgroup is all of , and hence itself is normal. So in any case has a proper normal subgroup, and hence cannot be simple.
This concludes this problem, which I hope exhibits the strength of the Sylow Theorems and also the importance of keeping natural group actions in the back of your head when tackling problems of this type. As always, thanks for reading!