The large-$N$ limit of eigenvalues of matrices with non-diagonal elements scaling as $1/N$ Define a series of matrices$$H_N=
\begin{bmatrix}
1&1/N&1/N&\cdots&1/N\\
1/N&2&1/N&\cdots&1/N\\
1/N&1/N&3&\cdots&1/N\\
\vdots&\vdots&\vdots&&\vdots\\
1/N&1/N&1/N&\cdots&N
\end{bmatrix}$$
My question is, when $N\to+\infty$, would the eigenvalues of $H$ be different from $\{1,\ldots,N\}$ ?
The answer is not obvious, as,  for the matrix series
$$G_N=\begin{bmatrix}
1&1/N&1/N&\cdots&1/N\\
1/N&1&1/N&\cdots&1/N\\
1/N&1/N&1&\cdots&1/N\\
\vdots&\vdots&\vdots&&\vdots\\
1/N&1/N&1/N&\cdots&1
\end{bmatrix}$$
You can verify that $G_N$ has an eigenvalue of $2$.

Nearly a year passed and now I finally got the answer for my problem:  In the limit $N\to\infty$, the eigenvalues of $H_N$ won't be different from $\{1,2,\ldots,N\}$.
Proof: The matrix element of $H_N$ is given by $H_{ij}=j\delta_{ij}+1/N$, where we have added an unimportant $1/N$ for the diagonal elements (and in the following, we won't write the subindex $N$ explicitly). We can directly solve the eigenvalue equation $H_{ij}\psi_j=\lambda\psi_i$ in this case, giving
$$i \psi_i+\frac{1}{N}\sum^N_{j=1}\psi_j=\lambda\psi_i~~\Rightarrow~~  \psi_i=-\frac{1}{N(i-\lambda)}\sum^N_{j=1}\psi_j,$$ summing over $i$ on both sides, we get $$\sum^N_{i=1}\psi_i=-\sum^{N}_{i=1}\frac{1}{N(i-\lambda)}\sum^N_{j=1}\psi_j$$
If $\sum^N_{j=1}\psi_j=0$, then we have $(i-\lambda)\psi_i=0$ for all $i$, which is not possible for a nonzero eigenvector. It follows that
$$\sum^{N}_{i=1}\frac{1}{(\lambda-i)}=N.$$
It is easy to prove that this equation has $N$ distinct roots $1<\lambda_1<2<\lambda_2<3<\ldots <N<\lambda_N<N+1$. We now prove that $\lambda_j\to j$ in the limit $N\to+\infty$. Let $\lambda_j=j+\Delta_j$, if $$\lim_{N\to\infty}\Delta_j=\Delta>0,$$ then there exist $N_0$ such that for all $N>N_0$, $\Delta/2<\Delta_j<1$, which lead to
$$\sum^N_{i=1}\frac{1}{j+\Delta_j-i}<\sum^j_{i=1}\frac{1}{j+\Delta_j-i}=\sum^{j-1}_{i=0}\frac{1}{\Delta_j+i}<\frac{1}{\Delta/2}+\frac{1}{\Delta/2+1}+\ln N,$$
which is obviously much smaller than $N$ for sufficiently large $N$, a contradiction. Thus 
$$\lim_{N\to\infty}\Delta_j=0~~\Leftrightarrow \lim_{N\to\infty}\lambda_j=j$$
for all $j\in N_+$.
