Can there be a well-defined set with no membership criteria? Prime numbers are a well-defined set with specific membership criteria. Can the same be said about "numbers"? Aren't numbers (that is, all numbers) a well defined set but without membership criteria? Anybody can say, given a particular object, whether that belongs in the set of numbers or not. But it may not be possible to give any criteria for this inclusion. 
We may want to say that the set of all numbers has a criterion and that is that only numbers shall get into the set and all non-numbers shall stay out of it. But then, this is merely a definition and not a criterion for inclusion.
Therefore my question: Can there be a well-defined set with no membership criteria?

A set must have membership criteria. Cantor defined sets by (taken from wikipedia)

A set is a gathering together into a whole of definite, distinct objects of our perception [Anschauung] and of our thought – which are called elements of the set.

As the elements in a set therefore are definite, we can describe them in a way - which is a membership critria. There can happen funny things (Take a look at Axiom of Choice and Russels Paradox, the latter showing why Cantors definition isn't enough and ZFC is needed (I sadly can't link to it now) )
In your example: The set of all numbers is the set containing all objects that we characterize as numbers. This quickly gets a bit meta-mathematical ("What is a number?", "two" and "2" are just the representatives / "shadows" of the thing we call two), you should take a look at how the natural numbers are defined.
