An argument for showing no simple group of order 504 in $S_7$ In this example, the first argument is that suppose the simple subgroup $H$ has order $504=2^3 3^2 7$, then $H\subset A_7 $, is there a more general theorem for this result? Because the example didn't make any argument here which made it looks trivial.

For me, I have to make the following argument: if $H$ is not in $A_7$ then $HA_7 =S_7$ and the third (diamond) Isomorphism Theorem would imply that $|H : H\cap A_7|=2$ and thus $H$ is not simple, a contradiction.

And is there a more general statement about this result?  Given an
 arbitrary group and its two simple subgroups of index say $2$ and $10$.
 Can we say anything about these two subgroups like containment?
Any subgroup of $S_{n}$ either contains all even permutation or half odd half even permutation. Consider $H \cap A_{7}$. Now If in H the latter case happens, then the even permutations of $H$ form a normal subgroup of $H$ , a contradiction to the simplicity of $H$. Hence the result.

A more general result can be likethis

If G is a simple group, and if there exists a subgroup of index $m$ in $G$ say $H$, that is , $[G:H]=m$, then $G$ can be embedded inside $A_{m}$.
You already showed that $G$ must be a subgroup of $A_7$. Since $|A_7|=2520=5\cdot504$, you want to consider the action of $A_7$ on the $5$ cosets of $G$ in $A_7$: Clearly, $A_7$ acts non-trivially, so $A_7/K$ must be isomorphic to a non-trivial subgroup of $A_5$, where $K$ is the kernel of the operation. But $A_7$ is simple, so $K$ must be $1$ which is an obvious contradiction.