Add CHDIR token to lexer

This CL changes the grammar defined in lexer.in.cc and propagates
the changes through all the affected files:

* src/depfile_parse.cc
* src/lexer.cc
* src/lexer.h
* src/lexer.in.cc (of course)

This adds a 'chdir' token which can be used after a 'subninja':

<subninja> path
<indent>  <chdir> <equals> path

However, this CL does not make the changes to the parser to
support the 'chdir' token. That is left for a future CL.

Note that https://github.com/ninja-build/ninja/pull/1578 is the
origin of this sequence of CLs. The github pull request does not
add a CHDIR token to the lexer, and uses a string compare with
"chdir". This CL corrects that and uses the lexer in the proper
way.

Change-Id: Iaac81cbcc387da44f7ace1298a0f7e388f414908
4 files changed