Ten books are placed in a random order. Find the probability of three given books being side by side As i understand it, this question should be answered by N(A)/N.

I take N to be 10!, im not sure if this is correct.

How do you calculate N(A)?
We can use your $10!$. There certainly are $10!$ equally likely ways to arrange the books.

Now we count the number of orderings in which our $3$ books are together. The leftmost of our $3$ books can be in any of $8$ places, $1$ to $8$. (Write the numbers $!$ to $10$ in a row, and look.)  Once this place is chosen, implicitly the other locations are chosen. Then the $3$ books can be put in the places in $3!$ orders. And for each such arrangement, the other books can be chosen in $7!$ ways, for a total of $(8)(3!)(7!)$. Now divide by $10!$. We get probability $\dfrac{(8)(3!)(7!)}{10!}$. There is a lot of cancellation. 

The following is perhaps easier. The set of positions our $3$ books will occupy can be chosen in $\dbinom{10}{3}$ equally likely ways.
Note that we are not placing the books in these places yet, we are just choosing a set of locations.

Of these ways, $8$ have our books in consecutive order. So the probability is $\dfrac{8}{\binom{10}{3}}$.
Well, for that you need to find all possible orders where $3$ books stay together and their own possible orders. so your $N(A)$ is a multiplication of those above which is $8!3!$ ($8$ is for the other $7$ books and, lets say, those $3$ tied together). So your final probability will look like $\frac{8!3!}{10!}= 0.0\overline6$.