Q:

Why is (>>) not defined as (*>)?

GHC currently implements >> as
(>>) :: m a -> m b -> m b
m >> k = m >>= \_ -> k

Why not do the following instead?
(>>) :: m a -> m b -> m b
m >> k = m *> k

Right now, I am thinking >>= does something *> doesn't.
But everything works out grammatically (as in, type-wise), so it's really hard to reason about why it wouldn't work.
Perhaps the monad instance does some computation that the applicative instance doesn't, but I think this would break the semantics of the type.
Update I only get to pick one SO answer as accepted, but the answer by dfeuer is very insightful (specially, for people who, like me, are relatively inexperienced in Haskell).

A:

According to a comment in the source code, it's to prevent people from accidentally creating a recursive binding if they override *> as >> in their Applicative instance.
(>>)        :: forall a b. m a -> m b -> m b
m >> k = m >>= \_ -> k -- See Note [Recursive bindings for Applicative/Monad]

The note says:

Note: Recursive bindings for Applicative/Monad
The original Applicative/Monad proposal stated that after implementation, the designated implementation of (>>) would become
(>>) :: forall a b. m a -> m b -> m b
(>>) = (*>)

by default. You might be inclined to change this to reflect the stated proposal, but you really shouldn't! Why? Because people tend to
  define such instances the other way around: in particular, it is
  perfectly legitimate to define an instance of Applicative (*>) in
  terms of (>>), which would lead to an infinite loop for the default
  implementation of Monad! And people do this in the wild.
This turned into a nasty bug that was tricky to track down, and rather
  than eliminate it everywhere upstream, it's easier to just retain the
  original default.

A:

4castle's answer is right, of course, but there's another thing to consider. Not every Monad instance supports an Applicative instance more efficient than liftA2 = liftM2. That is,
liftA2 f xs ys = xs >>= \x -> ys >>= \y -> pure (f x y)

Using the default (*>) = liftA2 (flip const) gives
xs *> ys = xs >>= \ _ -> ys >>= \y -> pure y

On the other hand, the default definition of (>>) gives
xs >> ys = xs >>= \ _ -> ys

As you can see, this uses only one bind, where the other uses two. By a monad identity law, they're equivalent, but the compiler doesn't know the monad laws. So the approach you suggest will probably make the optimizer work harder, and may even prevent it from producing the best code in some cases.

