What do you call a space whose only compact sets are finite? What do you call a topological space where a subset is compact iff it's finite? Is there a technical name?
For example, take the discrete topology, or the countable complement topology.

As others have noted, the term that you want is anticompact. This is an example of the kind of property studied by Paul Bankston in The total negation of a topological property, Illinois J. of Math., Vol. 23, Nr. 2 (1979), 241-252. I quote:

Let $K$ be a topological class. The spectrum $\operatorname{Spec}(K)$ of $K$ is the class of cardinal numbers $\kappa$ such that any topology on a set of power $\kappa$ lies in $K$. For example, any topology on a finite set must be compact; and any infinite set supports noncompact topologies. Thus $\operatorname{Spec}(\{\text{compact spaces}\})=\omega$. Other spectra can be computed quite readily, such as $\operatorname{Spec}(\{\text{connected spaces}\})=2(=\{0,1\})$, and $\operatorname{Spec}(\{\text{perfect spaces}\})=1$.
Now let $K$ be a class of spaces and define $\operatorname{Anti}(K)$ to be the class of spaces $X$ such that whenever $Y\subset X$, $Y\in K$ iff $|Y|\in\operatorname{Spec}(K)$. Thus $X\in\operatorname{Anti}(K)$ iff the only subspaces of $X$ which are in $K$ are those which “have to be” on account of their cardinalities. Clearly $\operatorname{Anti}(K)$ is always hereditary.

