Revamp to fix POSIX incompatibilities, to count columns correctly, and
to check for invalid inputs.
Use mbsnwidth to count columns correctly. Account for tabs, too.
Include mbswidth.h.
(YY_USER_ACTION): Invoke extend_location rather than LOCATION_COLUMNS.
(extend_location): New function.
(YY_LINES): Remove.
Handle CRLF in C code rather than in Lex code.
(YY_INPUT): New macro.
(no_cr_read): New function.
Scan UCNs, even though we don't fully handle them yet.
(convert_ucn_to_byte): New function.
Handle backslash-newline correctly in C code.
(SC_LINE_COMMENT, SC_YACC_COMMENT): New states.
(eols, blanks): Remove. YY_USER_ACTION now counts newlines etc.;
all uses changed.
(tag, splice): New EREs. Do not allow NUL or newline in tags.
Use {splice} wherever C allows backslash-newline.
YY_STEP after space, newline, vertical-tab.
("/*"): BEGIN SC_YACC_COMMENT, not yy_push_state (SC_COMMENT).
(letter, id): Don't assume ASCII; e.g., spell out a-z.
({int}, handle_action_dollar, handle_action_at): Check for integer
overflow.
(YY_STEP): Omit trailing semicolon, so that it's more like C.
(<SC_ESCAPED_STRING,SC_ESCAPED_CHARACTER>): Allow \0 and \00
as well as \000. Check for UCHAR_MAX, not 255.
Allow \x with an arbitrary positive number of digits, as in C.
Check for overflow here.
Allow \? and UCNs, for compatibility with C.
(handle_symbol_code_dollar): Use quote_n slot 1 to avoid collision
with quote slot used by complain_at.