If the complement of any singeton is finite, the set is finite If for any singleton set, the complement is finite then the set is finite right? This seems very intuitive. If the set was infinite, then removing a single element would still be infinite. But what if the statement only says it is true for a specific singleton set? Would the set still be finite?

I think you are confused because of the quantification. 'for any' expresses the universal property of the quantified object, whereas 'there is' only notifies the existence of such an object. Therefore, we should utilize only the existence of the set in solving your problem.
Let $X$ be a set and suppose there is a subset $A$ of $X$ such that $|A| = 1$ and $|X\setminus A| < \infty$ where $|\cdot|$ represents the cardinality of a set. Since $A$ and $X\setminus A$ are disjoint, we have $|X| = |A| + |X\setminus A|$. Therefore,  $|X|< \infty$ and this conclusion is not dependent on what $A$ is actually is. Hence, we can say that $X$ is finite.
