How does one refute this ultrafinitist argument? From Wikipedia: 


  Edward Nelson criticized the classical conception of natural numbers because of the circularity of its definition. In classical mathematics the natural numbers are defined as $0$ and numbers obtained by the iterative applications of the successor function to $0$. But the concept of natural number is already assumed for the iteration. In other words, to obtain a number like $2\uparrow\uparrow\uparrow 6$ one needs to perform the successor function iteratively, in fact exactly $2\uparrow\uparrow\uparrow 6$ times to $0$.


Like most people, I am not an ultrafinitist. However, I do not quite know how to refute this argument. What are some criticisms of it?
I am agnostic about most questions of the philosophy of mathematics, but in this case, I think Nelson's argument is incoherent: a primitive shepherd who doesn't have any abstract concept of number can count sheep being herded into a fold by making marks on a stick and count the sheep out again by crossing the marks off (and this is how our human notions of cardinal numbers probably developed). There is no circularity involved.

To talk meaningfully about notations like $2 \uparrow\uparrow\uparrow 6$, you must have admitted a notion of definition by recursion that (as an ultrafinitist) Nelson can't accept. It seems to be me to be incoherent for an ultrafinitist to say anything more than "I can't accept that a primitive shepherd could ever have enough sticks".
Nelson's critique is groundless. Peano's axioms are just a list of the essential properties of the set of natural numbers, essential in the sense that all other known properties can, in theory, be derived from these few properties. There is nothing &quot;circular&quot; about them. So successful have they been in this regard that these essential properties have come to define the set of natural numbers $\mathbb{N}$.
The natural numbers and many of their properties have been known for millennia across diverse cultures. Can any reasonable person deny that:

(1) $0$ is a natural number.
(2) Every natural number has a unique successor.
(3) Different natural numbers cannot have the same successor.
(4) No natural number has a successor that is $0$.

Now, some might quibble about the 5th axiom, the principle of mathematical induction:

(5) For all subsets $P \subset\mathbb{N}$, if (a) $0\in P$, and (b) for all $x\in P$, we also have the successor of $x$ being in $P$, then $P=\mathbb{N}$.

With only a bit more reflection, this, too, passes for a reasonable property of the set of natural numbers. There is nothing weird or &quot;unnatural&quot; about it. In fact, induction can be shown to hold on some subset of any non-empty set $X$.
Proof sketch: Let $X$ be an arbitrary, non-empty set. There exists at least one function $f: X \to X$, namely the identity function. Given any function $f:X\to X$ and $x_0\in X$, we can construct a subset $N\subset X$ such that $N=\{x_0, f(x_0) f(f(x_0)), \cdots \}$ and show that induction holds on $N$, namely that:

For all subsets $P \subset N$, if (a) $0\in P$, and (b) for all $x\in P$, we also have the successor of $x$ being in $P$, then $P=N$.

See my formal proof at http://www.dcproof.com/InductionMinRequirementsV2.htm