Etiquette and guidelines for asking someone if they might be doing work potentially similar to yours? So, say, if I want to start on a new research project but want to be sure that someone else isn't doing the same thing that I'm doing, how should I phrase an email to them, if at all?

Oftentimes, this could lead to a collaboration between the two. But in a few cases, this could also be a case where someone might be competing with you for the same research finding, and where they might want to "take the glory" so to speak.
I'd recommend against sending such an e-mail to someone you don't know well.  If you've got serious reason to fear there may be competition that neither of you wants, then perhaps a mutual friend could help sort it out; if you don't, then I wouldn't worry about it.

One danger is that the e-mail can come across as aggressively staking out territory.  Even just asking whether someone is working on this topic can be read as "speak now or forever hold your peace", with the implication that unless they announce it now, you'll be seriously offended if they end up competing with you on this topic.  This is not a fair position to put someone in, since research can be unpredictable and their work may end up having consequences for these topics that they do not yet foresee.

Collaboration can also be a touchy issue.  It's sometimes a mutually beneficial way to resolve this sort of situation, but not always.  It can be awkward when someone you don't know well suggests a collaboration, since no matter what excuses you offer, it's hard to avoid the appearance that declining a collaboration request is a negative judgement on the person making the request.

If you know someone well, you may be able to avoid these difficulties, but I wouldn't try it otherwise.

If you're a graduate student, then your advisor should help with situations like this.  For example, they should make sure potential competitors are aware of your thesis project (at least in mathematics, it's considered extremely bad form to compete with a grad student, and part of the advisor's role is to minimize the chances of accidental competition).