Definition of derivative of real function in baby Rudin Let $f$ be defined (and real-valued) on $[a,b]$. For any $x\in [a,b]$ form a quotient $$\phi(t)=\dfrac{f(t)-f(x)}{t-x} \quad (a<t<b, t\neq x),$$ and define $$f'(x)=\lim \limits_{t\to x}\phi(t),$$ provided this limit exists in accordance with Defintion 4.1. 
I have one question. Why Rudin considers $t\in (a,b)$? What would be if $t\in [a,b]?$

To answer your question directly, it doesn't matter.
Since we need the limit $t \to x$, we only really need to worry about $t$ close to (but different from) $x$. If $x$ is closer to $a$ instead of $b$ (but different from $a$), then we can worry about
$$|x - t| < 1/2 |x - a| \le 1/2 | x - b |.$$
This means $t \neq a$ and $t \neq b$.
You can think of something similar for $x = a$ or $x = b$.
