What is the probability that 4 cards drawn randomly from a deck are all hearts, given that there are at least three hearts? Here is the question:

$4$ cards are drawn at random from a standard $52$-card deck. At least
  $3$ of them are hearts. What is the probability that they are all
  hearts?

I was able to obtain the correct solution as follows:
$\frac{ \binom{13}{4}}{\binom{13}{4} + 39 \binom{13}{3}} = \frac{5}{83}$.
since there are $\binom{13}{3} \cdot 39$ ways to choose a heart and a non-heart.
I want to solve this using the Principle of Inclusion-Exclusion, but I'm having difficulty arriving at the same answer. To compute the number of 4-card hands with at least three hearts, I tried the following:
$ \binom {4}{3} \binom{13}{3}49 - 3\binom{13}{4}$
but this is giving me the wrong answer.
Reasoning:
$\binom {4}{3}$ is to choose which three cards out of four are hearts, $\binom{13}{3}$ is to count which ranks of hearts are selected, and then there are 49 choices for the last (fourth) card. Since we have over-counted four-heart hands, we subtract $3\binom{13}{4}.$

I think you made a typo in your answer it should be 
$\frac{ \binom{13}{4}}{\binom{13}{4} + 39 \binom{13}{3}}$. Now about your other approach of finding number of ways of choosing 4 cards with at least 3 hearts note that it will be $\binom{13}{3}*49 - 3 \binom{13}{4}$, which is same as above.  The reasoning goes as follows, first note that order does not matter, now there are $\binom{13}{3}*49$ ways of choosing at least three hearts with double counting of cases which have all hearts. The cases with all hearts are counted 4 times so we subtract those to get the right answer.   
