From 600f1bda2366b82d1492eb04a7419d12f199349b Mon Sep 17 00:00:00 2001 From: Kaz Kylheku Date: Tue, 6 Dec 2016 21:56:25 -0800 Subject: parser: fix problems at EOF involving #; syntax. This patch addresses a problem whereby if a TXR Lisp file ends with an erased object notation such as #;(a b c), there is a syntax error. The strategy is to simplify the grammar so that a single yyparse primed with SECRET_ESCAPE_E or SECRET_ESCAPE_I will read either an object, or just one instance of the #; notation. If #;OBJ is read, then the parse tree is returned as the nao value. The caller knows that #;OBJ must have occurred because there are no errors and the parser isn't at EOF, yet there is no parse tree. Then in lisp_parse we can loop on this situation, and make adjustments elsewhere also. So that iread continues to work, we must separate the parser_eof condition from the lookahead token. Under iread, we were clearing the token in prime_parser_post, but that was having the side effect of making the parser look like it is in EOF. We now preserve the EOF indication in a flag, so we can manipulate the token. * parser.h (struct parser): new member, eof. * parser.c (parser_common_init): Initialize new eof flag in parser structure to zero. (prime_parser_post): Set the eof flag if the parser's most recent token is zero. (lisp_parse_impl): Call the parser repeatedly while there are no errors, and no EOF, yet no object has been produced. This indicates that a #; erasure has been processed. (read_eval_stream): Restructure the logic here for clarity. Do not break the loop if error_val was returned from the parser, but there are no errors, and the parser isn't at EOF. This is behavior is probably redundant with respect to the loop in lisp_parse_impl. (read_eval_ret_last): Bugfixes here. Pass an error indicating value down to lisp_parse, like in read_eval_stream and make the logic similar. (parser_eof): Just return an indication based no the eof flag. * parser.y (hash_semis_n_expr, hash_semis_i_expr, ignored_i_exprs, ignored_n_exprs): Grammar rules removed. (hash_semi_or_n_expr, hash_semi_or_i_expr): New grammar rules. (spec): Retarget SECRET_ESCAPE_E and SECRET_ESCAPE_I cases to new rules. (parse): Clear eof flag to zero. --- parser.h | 1 + 1 file changed, 1 insertion(+) (limited to 'parser.h') diff --git a/parser.h b/parser.h index 6980e0f4..658178c7 100644 --- a/parser.h +++ b/parser.h @@ -50,6 +50,7 @@ struct parser { val parser; cnum lineno; int errors; + int eof; val stream; val name; val prepared_msg; -- cgit v1.2.3