$F[a] \subseteq F(a)?$ I think this is probably an easy question, but I'd just like to check that I'm looking at it the right way.

Let $F$ be a field, and let $f(x) \in F[x]$ have a zero $a$ in some extension field $E$ of $F$. Define $F[a] = \left\{ f(a)\ |\ f(x) \in F[x] \right\}$. Then $F[a]\subseteq F(a)$.

The way I see this is that $F(a)$ contains all elements of the form $c_0 + c_1a + c_2a^2 + \cdots + c_na^n + \cdots$ ($c_i \in F$), hence it contains $F[a]$. Is that the "obvious" reason $F[a]$ is in $F(a)$?
And by the way, is $F[a]$ standard notation for the set just defined?

(1) Yes, you are correct. Note that $F(a)=\{\frac{f(a)}{g(a)}:f,g\in F[x], g(a)\neq 0\}$; in other words, $F(a)$ is the field of fractions of $F[a]$ and therefore certainly contains $F[a]$. 
(2) Yes, the notation $F[a]$ is standard for the set you described.
Exercise 1: Prove that if $a$ is algebraic over $F$, then $F[a]=F(a)$. (Hint: prove first that $\frac{1}{a}\in F[a]$ (if $a\neq 0$) using an algeraic equation of minimal degree of $a$ over $F$.)
Exercise 2: Prove that if $a$ is transcendental over $F$, then $F[a]\neq F(a)$. (Hint: Prove that $F[a]\cong F[x]$ where $F[x]$ denotes the polynomial ring in the variable $x$ over $F$. Note that $F[x]$ is never a field if $F$ is a field.)
