Is there a notion in mathematics saying that, in a sense, all finite dimensions are actually infinite dimensional? So then every ordered pair or triplet and so on would be actually represented by an infinite sequence of numbers, and what we think of as 3 dimensions would mean that the point has an infinite number of zeros in its sequence starting with the 4th term. Only the first 3 terms of the sequence are not necessarily 0. For instance the point $(1,2,3)$ could be written instead as $(1,2,3,0,0,0...)$
In other words, in n dimensions, the first n terms would correspond to the standard notion of a coordinate in $\mathbb R^n$, and the rest are zero. Does anyone use this, or something similar, in mainstream mathematics? I feel as though I've heard of it somewhere. 

It is possible to force this perspective, but it would be incorrect to say that the mathematics "actually" is this way.
To be more specific, yes, you can choose to consider the inclusions $j_n\colon \mathbb{R}^n\longrightarrow\mathbb{R}^\infty$ defined by
$$j_n(x_1,\ldots,x_n)=(x_1,\ldots,x_n,0,0\ldots)$$
However, the objects $\mathbb{R}^n$ have their own independent existence - in no way are they "really" subspaces of $\mathbb{R}^\infty$, and the maps $j_n$ aren't special.
