Q:

Is this a viable approach to resolving multiple matches in a lexer?

I'm writing a lexer in JavaScript. It's pretty typical - rules are specified with regular expressions and produce a token.
I am unsure of the best way to handle when multiple rules are matched. The existing lexers I've looked at handle this by picking the rule with the longest match.
It also seems like a viable strategy, however, to simply use the first rule that matches. This is my current strategy.
Here is an example:
Input:
:=
Rules:
: -> COLON
= -> EQUALS
:= -> ASSIGN

The longest-match rule would return ASSIGN, where the first-match rule would return COLON and then EQUALS.
Obviously this is not desirable, so under my implementation I would just reorder the rules to put the ASSIGN rule first as follows:
:= -> ASSIGN
: -> COLON
= -> EQUALS

Is this a viable approach to just use the first matching rule? What are the advantages and disadvantages of each approach?

A:

No, that is not a viable strategy -- at least not if you want to have identifiers as well as keywords in your language.
Let's say you have two rules: one for identifiers that consist of one or more arbitrary letters and one for some keyword that consists of letters. Let's take the keyword var for example. Now let's consider the input string var variant. If the identifier rule comes first in your lexer definition, this will be tokenized as two identifiers. Clearly that's not what we want, so let's put the keyword rule first. But now it's tokenized as two var keywords, followed by the identifier iant. That's also not what we want.
So there is no way to order the rules, such that we can get what we want here. However the maximal munch rule would give us exactly what we want in cases like this. Therefore it is preferable.

A:

I wouldn't view it as "first" match, but having the lexer have defined priorities is not uncommon. The advantage of parsing colon first is on a miss, it eliminates two paths. If you miss assignment, you still need to check colon.

