Convergence of a double sum 
Let $(a_i)_{i=1}^\infty$ be a sequence of positive numbers such that $\sum_1^\infty a_i < \infty$. What can we say about the double series 
  $$\sum_{i, j=1}^\infty a_{i+ j}^p\ ?$$
  Can we find some values of $p$ for which it converges? I'm especially interested in $p=2$. 

Intuitively I'm inclined to think that the series converges for $p \ge 2$. This intuition comes from the continuum analog $f(x)= x^a, \quad x>1$:  if $a<-1$ we have 
$$\int_1^\infty f(x)\ dx < \infty$$
and $F(x, y)=f(x+y)$ is $p$-integrable on $(1, \infty) \times (1, \infty)$ for $p \ge 2$. 

To sum up, the result is false in general (see first part of the post below), trivially false for nonincreasing sequences $(a_n)$ if $p<2$ (consider $a_n=n^{-2/p}$) and true for nonincreasing sequences $(a_n)$ if $p\ge2$ (see second part of the post below).

Rearranging terms, one sees that the double series converges if and only if the simple series $\sum\limits_{n=1}^{+\infty}na_n^p$ does. But this does not need to be the case. 
To be specific, choose a positive real number $r$ and let $a_n=0$ for every $n$ not the $p$th power of an integer (see notes) and $a_{i^p}=i^{-(1+r)}$ for every positive $i$. Then $\sum\limits_{n=1}^{+\infty}a_n$ converges because $\sum\limits_{i=1}^{+\infty}i^{-(1+r)}$ does but $na_{n}^p=i^{-pr}$ for $n=i^p$ hence $\sum\limits_{n=1}^{+\infty}na_n^p$ diverges for small enough $r$.
Notes:

(1) If $p$ is not an integer, read $\lfloor i^p\rfloor$ instead of $i^p$.

(2) If the fact that some $a_n$ are zero is a problem, replace these by positive values which do not change the convergence/divergence of the series considered, for example add $2^{-n}$ to every $a_n$.

To deal with the specific case when $(a_n)$ is nonincreasing, assume without loss of generality that $a_n\le1$ for every $n$ and introduce the integer valued sequence $(t_i)$ defined by 
$$
a_n\ge2^{-i} \iff n\le t_i.
$$
In other words,
$$
t_i=\sup\{n\mid a_n\ge2^{-i}\}.
$$
Then $\sum\limits_{n=1}^{+\infty}a_n\ge u$ and $\sum\limits_{n=1}^{+\infty}na_n^p\le v$ with
$$
u=\sum\limits_{i=0}^{+\infty}2^{-i}(t_i-t_{i-1}),\quad v=\sum\limits_{i=0}^{+\infty}2^{-ip-1}(t_{i+1}^2-t_i^2).
$$
Now, $u$ is finite if and only if $\sum\limits_{i=0}^{+\infty}2^{-i}t_i$ converges and $v$ is finite if and only if $\sum\limits_{i=0}^{+\infty}2^{-ip}t_i^2$ does. For every $p\ge2$, one sees that $2^{-ip}t_i^2\le(2^{-i}t_i)^2$, and $\ell^1\subset\ell^2$, hence $u$ finite implies $v$ finite.
