I know that $G\_t = R\_{t+1} + G\_{t+1}$.

Suppose $\gamma = 0.9$ and the reward sequence is $R\_1 = 2$ followed by an infinite sequence of $7$s. What is the value of $G\_0$?

As it's infinite, how can we deduce the value of $G\_0$? I don't see the solution. It's just $G\_0 = 5 + 0.9\*G\_1$. And we don't know $G\_1$ value, and we don't know $R\_2, R\_3, R\_4, ...$
['https://ai.stackexchange.com/questions/22243', 'https://ai.stackexchange.com', 'https://ai.stackexchange.com/users/38264/']
You know all the rewards. They're 5, 7, 7, 7, and 7s forever. The problem now boils down to essentially a geometric series computation.

$$ 
G\_0 = R\_0 + \gamma G\_1
$$

$$ 
G\_0 = 5 + \gamma\sum\_{k=0}^\infty 7\gamma^k 
$$

$$ 
G\_0 = 5 + 7\gamma\sum\_{k=0}^\infty\gamma^k 
$$

$$ 
G\_0 = 5 + \frac{7\gamma}{1-\gamma} = \frac{5 + 2\gamma}{1-\gamma}
$$
There are a few ways to resolve values of infinite sums. In this case, we can use a simple technique of self-reference to create a solvable equation.

I will show how to do it for the generic case here of an MDP with same reward $r$ on each timestep:

$$G\_t = \sum\_{k=0}^{\infty} \gamma^k r$$

We can "pop off" the first item:

$$G\_t = r + \sum\_{k=1}^{\infty} \gamma^k r$$

Then we can note that the second term is just $\gamma$ times the orginal term:

$$G\_t = r + \gamma G\_t$$

(There are situations where this won't work, such as when $\gamma \ge 1$ - essentially we are taking advantage that the high order terms are arbitrarily close to zero, so can be ignored)

Re-arrange it again:

$$G\_t = \frac{r}{1 - \gamma}$$

This means that you can get the value for a return for the discounted sum of repeating rewards. Which allows you to calculate your $G\_1$. I will leave that last part as an execrise for you, as you already figured the first part out.