Motivation for different mathematics foundations I've been studying an introductory book on set theory that uses the ZFC set of axioms, and it's been really exciting to construct and define numbers and operations in terms of sets. But now I've seen that there are other alternatives to this approach to construct mathematics like New Foundations (NF), category theory and others. 

My question is, what was the motivation to look for an alternative for ZFC? Does ZFC have any weakneses that served as motivation? I know there are some concepts that don't exist in ZFC like the universal set or a set that belongs to itself. So I thought that maybe some branches of mathematics needed these concepts in order to develop them further, is that true?
Well. That depends on whom you might ask this.


Set theory might be inconsistent. In particular $\sf ZFC$ and its extension by large cardinal axioms. It's a nontrivial thing, to feel safe with these theories, and it takes a lot of practice and time until you understand that $\sf ZFC$ is self-evident to some extent, and [some] large cardinal axioms are somewhat self-evident as well.

But of course, we might be tricked. Crooks have been known to seem honest, until you're left with a big heap of nothing in your hands. That is why Nigerian royalty is going to have a very hard time emailing people around the world.

If that happens, we need to ask ourselves where the problem lies. Is it in one of the axioms, or maybe specifically in the existence of infinite sets? Will a slightly weaker set theory (e.g. $\sf ZC$) work better, or maybe we have to resort to arithmetic theories to fix things?
Our grasp of things is usually largely inconsistent,1 but we do like to think in "types". So working inside different categories when needed, or working with some type theory or another. A lot, and I mean a lot, of people will twitch when you ask them what are the elements of $\pi$. Or whether or not $\frac13$ is a subset of $e$.

Of course, those who have a firm understanding of this know that this is a question of implementation, and this is like asking whether or not the machine code of one implementation of an algorithm is the same or different of another. But people don't think about it this way, although in a way they kinda do.

Instead, people focus on their math, and they just remember (or more accurately: they don't) that you can formalize this in terms of set inside set theory.
$\sf ZFC$ is incomplete. That doesn't sound like a big deal. But it is, in a deep sense. We might half expect a foundational theory to be able and decide all sort of things about the mathematical universe. Enough so most, or any, questions we might have can be answered there. Again, those who study foundations long enough should realize this is not the goal of a foundational theory, but it is somewhat expected.

And some people feel very awkward with this incompleteness. They will also have a problem with arithmetic based foundations, since that is an incomplete foundation also. But it's true that in $\sf ZFC$ those incompleteness phenomenon are far more gaping, and as soon as you leave the term "countable" and enter the "uncountable", more or less everything becomes open and independent.

This poses an actual problem. Imagine that half the functional analysts would work in one set theory (say $V=L$), and the other half in another (say $\sf ZFC+PFA$). It would create some strange discrepancies which will eventually tear the field up. And the same goes to everything else, except set theory, whose main occupation is these different axioms.


But that's just counting three strikes against $\sf ZFC$. And I cannot, in good conscience finish my post like this. So let's balance things as to why $\sf ZFC$ is a good foundation.


Set theory, and in particular $\sf ZFC$ is quite self evident. When I was a starting masters student, I attended a course by one of the generation's greatest mathematician and asked him about the axioms of $\sf ZFC$ and he said that a good foundational theory is one whose axioms you don't feel you're using. It might be that we're trained to think in "set theory ways", but it's also true that the axioms give you exactly the expressive power to talk about everything you care about and not care about one specific implementation over another (looking your way, Replacement axioms!), which is quite great.

Can you imagine a mathematical universe where the reals constructed with Dedekind cuts and with Cauchy sequences are different? (Those mathematical worlds exists in other foundations, by the way, and at least to me, that is weird.)
The previous point leads us exactly to this argument. Humans take an abstract algorithm, implement it in various languages, on various processors, using various data structures and types. But the code and data all turn into electronic signals. So working with high level objects like $\Bbb N$ and $\Bbb R$, and function spaces and so on is our algorithms. And we can turn that into electronic signals, or sets and first-order structures in this case.
We like sets more than we are willing to admit. One of the problems of second-order logic is that the logic is incomplete.2 And if you have an algorithm for a list of inference rules, to verify a proof, then there will be statements which are true (even valid) that these inference rules cannot prove. So your proof theory is quite insufficient in this aspect. Because at least the valid sentences should be provable.

So we can, instead, use a first-order based foundation to fix this. Instead of proving something about $\Bbb R$, we prove that $\sf ZFC$ proves that thing about $\Bbb R$; and this we can verify mechanically. So one option is to resort to some arithmetic foundation. But this causes a different problem. We cannot "take objects" anymore. We don't have sets of reals, or rings, or groups. So when you prove something about groups, you can't say "Let $G$ be an abelian group, then bla bla bla" anymore. You have to say "The theory of abelian groups proves that bla bla bla".

And this is quite a big issue, since we think about mathematics in some material sense. The objects exists somewhere. They are not just axiomatic consequences. And set theory, in particular $\sf ZFC$ with its implementation agnosticism, provides us with the means to do exactly this.




Footnotes.

(1) Is $\Bbb N\subseteq\Bbb R$? Often the answer is yes, many other times the answer is no. And it really depends on what you want to do, or how you mostly use these two objects. But since we know that either method can be replaced by the other we're not worried about it too much. But it is a concrete question that has convenient answer either way, and we exploit it. And that, in a nutshell, a "global inconsistency" in our thinking.

You can argue about this, but it's going to be besides the point.

(2) Incomplete here means in the sense of the completeness theorem. Something true in every model need not be provable. This is a different kind of incompleteness than the one referred to earlier, where $\sf ZFC$ is incomplete in the sense that it does not prove or refute every statement.
Yes. Category theory is one example of a theory than transcends ZFC. Some other examples include Hyperreals and Surreals. There are many more.