Can this be taken as a definition of compactness for metric spaces? For metric spaces, the following theorem is true:

Let $X$ be any metric space, and $K$ be a compact subset of $X$. Then,
  for any infinite subset $E$ of $K$, $E$ has a limit point in $K$.

My question is: Is the converse true? If yes, can we take this as a definition of compactness for metric spaces?

The answer is yes.

If a metric space X, any of its infinite subsets has a limit point, X is compact.

The proof is a bit involving, but you should be able to find it easily on the web.
In terms of definition, since this is equivalent (true for both directions), logically it has no problem to be a definition for metric space.
However, this equivalent relation might not be true for general topological space, but the concept of compactness is still important there, thus I would not suggest to use this as the definition for compactness. 
Plus, original definition of compact set is more intuitive to me as it captures the nature more closely that a compact set is "small" and "finite" in some sense, and it is more general so that we do not need a metric exist to define compact sets.
One more point is that you probably noticed a big part of Analysis is wrestling with "infinite sets". If the set is finite, it is very easy to deal with (e.g. we could just take the max of elements, we know it is bounded etc.). And compactness extend the finite sets into a type of important sets that is infinite but is the closest to "finite sets" - the best thing if we cannot have a finite set. And the original definition of compact set directly reveal this "finite cover" feature of a compact set 
