Derivative is just speed of change? In school we've been told that derivative of $x^2$ is $2x$.
Also I've read that derivative is simply a speed of value change.
So if
$$f(x)=x^2$$
then, using simple explanation, derivative of that function would be
$$f'(x)=f(x+1)-f(x).$$
Now if we take derivative when $x=2$ we will get
$$f'(2)=f(2+1)-f(2)=3^2-2^2=9-4=5.$$
But if we will take conversion rule (from school) which says that $[x^2]'$ is $2x$ then
$$f'(x)=2x$$
and if we will put the same point here we will get
$$f'(2)=2*2=4$$
so first result gives me $5$ and second result gives me $4$. And this problem seems to be appearing for every number. Number calculated by simplified interpretation is always bigger by $1$.
And my only guess is that simplified explanation missing something. Or, maybe, I made a mistake somewhere.
Can you, please, help me figure it out?
Update:
I spend 2 days trying to figure this out! Thanks to all of you, guys!!! Now i got it!))))

You are right, the derivative is a speed (or rate) of change.  People are often familiar with the idea that the slope of a line is it's rate(speed) of change.
I'd just like to add to Marco's answer and tell you to pay particular attention to his very first equation: 
$gradient =\frac{f(x + h) - f(x)}{(x+h) - x}$.
What he calls a gradient, is more commonly known as a slope and what you are looking at is a slope for a line that passes through the two points $(x, f(x))$ and $(x+h, f(x+h))$.  When h is a very small quantity that line is very, very close to the tangent line at $(x, f(x))$ and the slope of the tangent line is the speed of change for that function when it is at the point $(x, f(x))$.  The problem with finding the slope of the tangent line, and thus the instantaneous speed of change, is that we only know 1 point on the tangent line, $(x, f(x))$.
By also looking at the point $((x+h), f(x+h))$ and making h be very small, we are finding the slope of a line that is very close to the tangent line. It's slope is an approximation for the tangent's slope.  As we make h be smaller and smaller, we are making our lines get closer to the tangent line and getting better approximations of its slope.
When we take the limit of our slope formula as h approaches $0$, the limit of our lines' slope will be the tangent line slope.  That is why the derivative is defined as the limit of the slope of our lines as h approaches $0$: $$derivative = lim_{h \to 0}{\frac{f(x+h) - f(x)}{(x+h) - x}}$$
