How to solve this least square problem effectively? I want to solve the least square problem,
$\min\|Ax-b\|_2$, but the condition number of $A'*A$ is very large, How can I solve this problem effectively?
You should try regularization. Several common types are Tikhonov regularization and $\ell^1$ regularization. See this book for a machine learning perspective. 

Many common regularizer are of the form $f(x, \lambda)$ where $f(x,\lambda)$ is a non-negative function with tuning parameter $\lambda\geq 0$ to which you solve $\arg\min_x ||Ax-b||_2^2 + f(x,\lambda)$. If $ f=0$, you get the original (unregularized) problem. If $f(x,\lambda) = \lambda ||x||_2$ you get Tikhonov regularization (which can be efficiently implemented via say, the SVD, by applying a certain function to the singular values) (also known as ridge regression). If you have $f(x,\lambda) = \lambda ||x||_1$, this will make solutions of $x$ to be sparser (you may see this under the name LASSO in machine learning-- this is $\ell^1$ regularization). 

The Tikhonov regularization problem has a nice form: $x = (A^T A + \lambda I)^{-1} A^T b$, or equivalently, if $A=USV^T$ is the SVD of $A$ with singular values $\{\sigma_i\}$, estimate $x$ by  $V D U^T b$ where $D$ has diagonal $\{ \sigma_i/(\sigma_i^2 + \lambda^2)\}$. 

The LASSO problem typically doesn't have a nice solution (the orthogonal case in the original paper is the only one I know of)but can easily done with common convex programming packages , e.g, cvx, or with soft thresholding (see the original paper for details, or the papers on ISTA/FISTA (FISTA=Fast Iterative Shrinkage Thresholding Algorithm)).
Hint: Check Moore–Penrose pseudoinverse.