How did the rule of addition come to be and why does it give the correct answer when compared empirically? I'm still a high school student and very interested in maths but none of my school books describe these kind of things, and only say how to do it not the whys. My question is very simple, for example:

        19
       +25
      = 44


because the one from adding 9 and 5 goes on to add with 1 and two. How did this rule of addition come to be?

Here's a bit of explanation that can be useful(sorry if it is frustrating):
Suppose we are a 3 year old child and no one teaches us how to add and we recognize 1 is our index and 5 all palm fingers held up. Someone gives us the problem add: 1+5 so we hold 'em up, right? and again someone gives us to add 8564+2345 so we can't lift 'em up. So we try to device a rule but we don't recognize 6+4= 10 in which 0 stays and one jumps neither can we say that only the digits from rightmost positions are to be added. This is what i meant.
Well essentially what you're doing is this:

$$19+25 = (10 + 9) + (20 + 5) = \underbrace{(9 + 5)}_{\text{ones}} + \underbrace{(10 + 20)}_{\text{tens}}$$

Then you'll get:

$$14 + (10 + 20)$$

So for the ones digit you get $4$ so let's subtract that from $14$ to get $10$. For the tens digit, since you still have $10$ left over from adding the ones digits you have to "carry over" $10$ into $(10+20)$:

$$10\,\text{(carried over digit)}+(10+20) = 20+20 = 40$$

Adding the result from the ones digit gives:
$$40 + 4 = 44$$

The "rule" of carrying over from one place value into another is just that, except that you do it vertically.
Though my answer is not quite what I want as well, this is what I could conclude seeing all the views here and it's not an algebric verification. If it will be possible I'll come back here and post an answer delving deeper into the history: