Understanding the fundamental theorem of calculus 
I'm having problems understanding why $$ \frac{d}{dx} \int_{a}^{x} f(t)\,dt = f(x)$$
I'm somewhat weirded out by the fact that there is a $dt$ at the end of $$F(x) = \int_a^x f(t)\,dt$$ too.
We are differentiating with respect to $x$...I understand that $ \frac{d}{dx} \int f(x) \, dx = f(x)$ but why is the $t$ in the definition?

You're encountering two standard abuses of notation in calculus:


*

*The integral sign $\int$ is used to denote both "definite" integrals (limits of Riemann sums) and "indefinite" integrals (a.k.a., antiderivatives, with $\int f(x)\, dx$ denoting an arbitrary function $F$ whose derivative is $f$).

*Mathematically, a function is an association of a unique output to each input. Calculus books often write $y = f(x)$ and speak of $f(x)$ as "the function". While this usage is compact and practically convenient, it's potentially misleading (and technically incorrect).
Concretely, suppose $f(x) = x^2$ for all real $x$. The "function" here (the object that can gets differentiated and integrated in calculus) is the squaring operator. The symbol $x$ is merely a dummy variable: In the absence of external restrictions on the meanings of symbols, the equations $f(t) = t^2$, $f(u) = u^2$, and $f(\xi) = \xi^2$ define exactly the same function.

The "indefinite" usage of the integral sign allows us to write, with $C$ denoting a real number,
$$
\int x^2\, dx = \frac{x^3}{3} + C.
$$
Note that "each side is a function of $x$". However, with equivalent meaning we could write: $\int t^2\, dt = \frac{1}{3}t^3 + C$ , $\int u^2\, du = \frac{1}{3}u^3 + C$, or $\int \xi^2\, d\xi = \frac{1}{3}\xi^3 + C$.
The "$dx$" plays a couple of roles, one being to specify the name of the input variable to $f$. Writing $\int x^2\, dx$ means "antidifferentiate the squaring function". Writing (say) $\int x^2\, dt$ is ambiguous; if $x$ denotes a real number, then $\int x^2\, dt = x^2t + C$, but if $x$ denotes a function of $t$ there is no general simplification. Writing $\int x^2$ (with no differential) is therefore a recipe for disaster.

The "definite" usage of the integral sign behaves a little differently. The result of integrating the squaring function over some interval $[a, x]$ can be written
$$
\int_a^x t^2\, dt = \frac{x^3}{3} - \frac{a^3}{3}.
$$
The integral on the left depends on $x$ (and on $a$), but $t$ makes no sense outside the integral, and can be replaced by any letter other than $x$ or $a$ without changing the left-hand side.
If we denote by $F$ the function defined by
$$
F(x) = \int_a^x t^2\, dt = \int_a^x f(t)\, dt,
$$
viewing $a$ as a fixed real number, then indeed $F'(x) = x^2 = f(x)$. Again, we wouldn't expect a $t$ to appear, since nothing in the preceding equation depends on $t$.

Trouble occurs when these notations and conventions about dummy variables get mixed or otherwise abused. The following are tempting to write, but likely to confuse (at best):
$$
\int_a^x f(x)\, dx;\qquad
\frac{d}{dx} \int_a^b f(x)\, dx;\qquad
\int f(x).
$$
