Changes between Version 20 and Version 21 of PIRCDevelopment

Show
Ignore:
Timestamp:
08/06/09 17:49:41 (5 years ago)
Author:
kjs
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • PIRCDevelopment

    v20 v21  
    176176The lexer starts out in the {{{INITIAL}}} state by default (as per Flex specification). When reading input such as {{{<<'EOS'}}}, the rule on [source:/trunk/compilers/pirc/src/hdocprep.l#L306 line 306] is activated. The actual string ("EOS") is stored in the field {{{state->delimiter}}}, and an escaped {{{newline}}} character is stored in the {{{heredoc}}} buffer. 
    177177 
    178 Since the preprocessor does not build a data structure representing the input, but instead writes the output directly (to a file), the "rest of the line" needs to be stored somewhere. This is because the {{{<<'EOS'}}} heredoc token is basically a placeholder for the actual (heredoc) string contents. Hence, the [source:/trunk/compilers/pirc/src/hdocprep.l#L318 activation of {{{SAVE_REST_OF_LINE state}}}]. 
    179  
    180 The state {{{SAVE_REST_OF_LINE}}} has only one function, and that is to SAVE the REST OF the LINE :-). It will match all the text after the {{{<<'EOS'}}} heredoc marker up to and include the end-of-line character. This, including an additional "\n" character is stored in the {{{linebuffer}}} field, which always contains the "rest of the line". 
     178Since the preprocessor does not build a data structure representing the input, but instead writes the output directly (to a file), the "rest of the line" needs to be stored somewhere. This is because the {{{<<'EOS'}}} heredoc token is basically a placeholder for the actual (heredoc) string contents. Hence, the [source:/trunk/compilers/pirc/src/hdocprep.l#L318 activation of SAVE_REST_OF_LINE state]. 
     179 
     180The state {{{SAVE_REST_OF_LINE}}} has only one function, and that is to SAVE the REST OF the LINE :-). It will match all the text after the {{{<<'EOS'}}} heredoc marker up to and include the end-of-line character. This, including an additional "\n" character is stored in the {{{linebuffer}}} field, which always contains the "rest of the line". As you can see, in this scenario there is no "rest of the line", except for the end-of-line character ("\n", or "\r\n" on Windows). 
     181 
     182After the heredoc marker the actual heredoc string must be scanned, hence the activation of the HEREDOC_STRING state on [source:/trunk/compilers/pirc/src/hdocprep.l#L331 line 331]. 
    181183 
    182184