1st stage of compilation： turns source code into tokens.
(e.g., var a = 2; -> var , a , = , 2 , ; )
the difference between tokenizing and lexing is：
Lexer is basically a tokenizer, but it usually attaches extra context to the tokens (this token is a number, that token is a string literal, etc.)
Osman
