Is any extension ring $S \supset R$ an $R$-algebra? Is any extension ring $S \supset R$ an $R$-algebra?
In our lecture note an $R$-algebra $A$  is defined as follows $:$
An $R$-algebra $A$ is a ring $A$, which is also an $R$-module satisfying the condition
$$a(xy)=(ax)y=x(ay),\ a \in R,\ x,y \in A.$$
It is clear that $S$ is a ring along with an $R$-module with the usual addition and scalar multiplication. But how does the scalar multiplication become bilinear unless $R \subset Z(S)$ where $Z(S)$ denotes the center of $S$. 
I think the assertion made in our lecture note is false. Please check it.
Thank you in advance.

No: take for example $\mathbb C \subseteq \mathbb H$, the complex numbers in the quaternions. (This is an example if what Pedro was getting at in the comments.)
The axioms of an $R$ algebra $A$ require that $R$ is contained in the center of $A$, which is not the case for this example.
