The foundations of mathematics may seem to be a topic curiously disconnected from the the rest of mathematics. This is not quite so. This page lists examples where questions of foundations do affect questions and problems in “real world” mathematics.
See also Wikipedia's list of statements undecidable in ZFC.
The following question is called the Whitehead problem
Every free abelian group satisfies . Is, conversely, every abelian group that satisfies a free abelian group?
When formalizing group theory within ETCS as a set theory, then this is question is undecidable. (It is still undecidable in ZFC, equivalently in ETCS + Collection.) In some models it is true, while other models have counterexamples.
It was conjectured by Harvey Friedman that all theorems involving finitary objects published in Annals of Mathematics (for argument’s sake) can be proved in Elementary Function Arithmetic (EFA), a weak fragment of Peano arithmetic?. One implication is that Fermat’s last theorem is provable in PA. There is a current program of research by Angus MacIntyre to show this last fact directly. From a category theoretic point of view, Colin McLarty has shown that all of the material in SGA relies only on quite weak foundations, namely MacLane set theory with one universe (weaker than the theory considered in ZFC)
McLarty comments extensively on the possibility of proving Fermat’s last theorem, and more generally the Modularity theorem, in PA in the article