SQLiteLexer
class SQLiteLexer extends Lexer
| internal |
Lex SQLite schema output with the statement boundaries of sqlite3_complete().
Constants
| protected HIGH_BYTES |
Bytes of multibyte characters, which unquoted identifiers may contain. |
| protected LETTERS |
|
| protected DIGITS |
|
| protected WHITESPACE |
|
| protected COMMENT_STARTS |
|
| private WINDOW_LENGTH |
The most source that runs are matched against at once. |
| private WINDOW_CUTS |
The bytes a window can be cut before, as no unquoted token contains them. |
| private IDENTIFIER_BYTES |
|
| private SPACE |
Whitespace, line comments and byte order marks before a token in a run. Repetitions in the native patterns are bounded so one match stays within PCRE's limits; longer input stops the run and is read by the scanner. |
| private TOKEN |
The tokens a run reads, as the scanner splits them: words that do not run into a string, numbers, quoted
names whose closing quote does not begin a doubled one, strings, and any other byte as a symbol. Block comments, signs and blob literals are left to the scanner. |
| private RUN |
A run of tokens read in one native call, through a semicolon or up to a token that needs the scanner. |
| private NORMAL |
A statement that cannot be a trigger. |
| private CREATE |
The CREATE [TEMP] TRIGGER prefix is still possible. |
| private TRIGGER |
A trigger body, where semicolons do not end the statement. |
| private TRIGGER_SEMICOLON |
A semicolon inside a trigger body. |
| private TRIGGER_END |
END directly after a semicolon inside a trigger body. |
Properties
| protected int | $length | from Lexer | |
| protected int | $position | from Lexer | |
| protected int | $statementStart | from Lexer | |
| protected int | $tokenStart | The start of the token or terminator most recently scanned. |
from Lexer |
| protected bool | $ended | from Lexer | |
| protected int | $windowStart | from Lexer |
Methods
Begin the next non-empty statement, or return null at the end of the input.
Append a run of regular tokens, or one token that needs the scanner.
Get the type of a token appended without one, which follows from its upper-case text.
Match a run of tokens at the position, or return null where the scanner must read the next token.
Skip the rest of the current statement and return the offsets of its terminator.
Skip to the parenthesis that closes the given number of open groups and return its start.
Scan the token starting at the current position, or a terminator as End.
Get the length of a statement terminator at the current position, or zero.
Reset the trigger completion state.
Skip a comment, or a UTF-8 byte order mark, which SQLite reads as whitespace before any token.
Scan a quoted run whose opening quote is at the position, escaped by doubling and optionally backslashes.
Details
in
Lexer at line 67
__construct(string $sql)
Create a lexer over the complete SQL input.
in
Lexer at line 77
TokenStream|null
nextStatement()
Begin the next non-empty statement, or return null at the end of the input.
at line 65
void
read(array $types, array $starts, array $ends, array $texts)
Append a run of regular tokens, or one token that needs the scanner.
at line 102
TokenType
type(string $text)
Get the type of a token appended without one, which follows from its upper-case text.
in
Lexer at line 149
protected array|null
matchRun(string $pattern)
Match a run of tokens at the position, or return null where the scanner must read the next token.
Dialects read runs of ordinary tokens with one native match each, so only tokens that need boundary handling go through the scanner. A statement's first token is matched alone, as parsers skip most statements after it.
in
Lexer at line 194
array
skipStatement()
Skip the rest of the current statement and return the offsets of its terminator.
in
Lexer at line 208
int|null
skipGroup(int $depth)
Skip to the parenthesis that closes the given number of open groups and return its start.
Returns null when the statement ends first.
in
Lexer at line 226
string
value(TokenType $type, int $start, int $end)
Decode a token's value.
in
Lexer at line 239
int
line(int $offset)
Get the one-based line containing a source offset.
in
Lexer at line 247
ParseException
error(string $message, int $offset)
Create a parse error located at a source offset.
in
Lexer at line 255
protected TokenType
scan()
Scan one token or the statement terminator.
in
Lexer at line 272
protected void
finishStatement()
Mark the current statement as ended at its terminator.
at line 120
protected TokenType
scanToken()
Scan the token starting at the current position, or a terminator as End.
in
Lexer at line 286
protected void
skip()
Scan to the end of the current statement.
in
Lexer at line 294
protected int
terminator()
Get the length of a statement terminator at the current position, or zero.
in
Lexer at line 302
protected bool
skipCommand()
Consume a client command between statements.
at line 211
protected void
startStatement()
Reset the trigger completion state.
in
Lexer at line 317
protected void
endStatement()
Handle input that belongs to a statement after its terminator.
in
Lexer at line 324
protected void
skipWhitespace()
Skip whitespace and comments.
at line 197
protected bool
skipComment()
Skip a comment, or a UTF-8 byte order mark, which SQLite reads as whitespace before any token.
in
Lexer at line 363
protected void
skipLine()
Skip to the start of the next line.
in
Lexer at line 373
protected void
scanQuoted(string $quote, bool $backslashes, string $description)
Scan a quoted run whose opening quote is at the position, escaped by doubling and optionally backslashes.
in
Lexer at line 402
protected string
word(int $start, int $end)
Normalize an unquoted identifier or keyword.
in
Lexer at line 410
protected string
identifier(int $start, int $end)
Decode a quoted identifier.
in
Lexer at line 421
protected string
string(TokenType $type, int $start, int $end)
Decode a string literal whose quotes are escaped by doubling.