Relation between $\zeta(s), \ Re(s) < 1$ and the summation $\sum_{k=1}^\infty k^{-s}$ First thing I want to mention is that this is not a topic about why $1+2+3+... = -1/12$ but rather the connection between this summation and $\zeta$.
I perfectly understand that the definition using the summation $\sum_{k=1}^\infty k^{-s}$ of the zeta function is only valid for $Re(s) > 1$ and that the function is then extrapolated through analytic continuation in the whole complex plan.
However some details bother me : Why can we manipulate the sum and still obtain correct final answer.
$$
S_1 = 1-1+1-1+1-1+... = 1-(1-1+1-1+1-...)= 1-S_1 \implies S_1 = \frac{1}{2} \\
S_2 = 1-2+3-4+5-... \implies S_2 - S_1 = 0-1+2-3+4-5... = -S_2 \implies S_2 = \frac{1}{4} \\
S = 1+2+3+4+5+... \implies S-S_2 = 4(1+2+3+4+...) = 4S \implies S = -\frac{1}{12} \\
S "=" \zeta(-1)
$$
Clearly these manipulations are not legal since we're dealing with infinite non-converging sums. But it works ! Why ?
Is there a real connection between the analytic continuation which yields the "true" value $\zeta(-1) = -1/12$ and these "forbidden manipulations" ? Could we somehow consider these manipulations as "continuation of non-converging sums" ? If so, is there a well-defined framework with defined rules because it is clear that we must be careful when playing with non-converging sums if we don't want to break the mathematics ! (For example  Riemann rearrangement theorem)
And since it seems that these illegal operations can be used to compute some value of zeta in the extended domain $Re(s) < 1$, are there other examples of such derivations, for example $0 = \zeta(-2) "=" 1^2 + 2^2 + 3^2 + 4^2 + ...$ ?
Hopefully this is not an umpteenth vague question about zeta and $1+2+3+4...$ I did some research about it but couldn't find any satisfying answer. Thanks !

They "work" because the manipulations you present work where the original definition was valid.  Notice that:
$$\eta(s)=\sum_{n=1}^\infty\frac{(-1)^{n+1}}{n^s}$$
This function can be made to converge for all $s:$
$$\eta(s)=\lim_{r\to1^{-1}}\sum_{n=1}^\infty\frac{(-1)^{n+1}}{n^s}r^n$$
Whereupon we find that for $s=0$,
$$\begin{align}S_{1r}&=r-r^2+r^3-\dots\\&=r-r(r-r-r^2+r^3-\dots)\\&=r-rS_{1r}\\\implies S_{1r}&=\frac r{1+r}\end{align}$$
Then take $r\to1$ to get $S_1=1/2$.  Notice the similarities and differences between this and your method.
In the same manner,
$$\begin{align}S_{2r}&=r-2r^2+3r^3-\dots\\&=(r-r^2+r^3-\dots)-r(r-2r^2+3r^3-\dots)\\&=S_{1r}-rS_{2r}\\\implies S_{2r}&=\frac r{(1+r)^2}\end{align}$$
Letting $r\to1$, we get $S_2=1/4$.
Notice that in each of the steps above, if we replace $r$ with $1$, we get the methods you present, despite that they don't really make sense in that way.

Also notice that, by manipulation of the original definitions,
$$\begin{align}\zeta(s)-\eta(s)&=2\left(\frac1{2^s}+\frac1{4^s}+\frac1{6^s}+\dots\right)\\&=2^{1-s}\left(\frac1{1^s}+\frac1{2^s}+\frac1{3^s}+\dots\right)\\&=2^{1-s}\zeta(s)\end{align}$$
$$\zeta(s)=\frac1{1-2^{1-s}}\eta(s)=\frac1{1-2^{1-s}}\lim_{r\to1^{-1}}\sum_{n=1}^\infty\frac{(-1)^{n+1}}{n^s}r^n$$
In this way, everything you have presented makes sense in the context of $\Re(s)>1$, so it holds for all $s$ by analytic continuation.
Notice things like the Riemann rearrangement theorem does not come in to play due to absolute convergence for $\Re(s)>1$, where we derive these formulas.  Also, $S_r$ converges absolutely for any $|r|<1$.  Also notice that what determines where a certain set of parentheses are allowed comes from these convergent scenarios.
From all of the above, I note that instead of using algebra, derivatives may be used to give

$$\zeta(-s)=\frac1{1-2^{1+s}}\lim_{r\to1}\left[\underbrace{r\frac r{dr}r\frac r{dr}\dots r\frac r{dr}}_s\frac{-1}{1+r}\right]$$

For whole numbers $s>0$.

The rules that allow this to work is to work only with convergent sums that are analytic continuations of the original sums.  Then, everything becomes normal and there are not so many worries.
