This was an exam question for my course and I am struggling to actually answer it in a way that is not fluff.Here is my current answer:CFGs describe how non-terminal symbols are converted into terminal symbols via a scanner. However, a parser defines what those terminal symbols convert to in terms of lexical tokens. CFGs are grammatical descriptions of a language instead of simply defining what tokens should be parsed out from an input string.What is the correct way to answer this?
Why do we not use CFGs to describe the structure of lexical tokens?
context free;formal grammars;regular expressions;lexical analysis