1. class
  2. Hypervel
  3. \
  4. SchemaParser
  5. \
  6. Parsing
  7. \
  8. MySqlLexer

MySqlLexer

class MySqlLexer extends Lexer

internal  
 

Lex mysqldump and mariadb-dump output as the mysql client splits it.

Executable comments are SQL, so their markers are read as whitespace and a delimiter inside one still ends the statement. MariaDB-only fragments are read only for MariaDB, as MySQL servers ignore them.

Constants

protected HIGH_BYTES

Bytes of multibyte characters, which unquoted identifiers may contain.
protected LETTERS

protected DIGITS

protected WHITESPACE

protected COMMENT_STARTS

Comment starts, and the * that closes an executable comment.
private WINDOW_LENGTH

The most source that runs are matched against at once.
private WINDOW_CUTS

The bytes a window can be cut before, as no unquoted token contains them.
private IDENTIFIER_BYTES

private SPACE

Whitespace and line comments before a token in a run.

Repetitions in the native patterns are bounded so one match stays within PCRE's limits; longer input stops the run and is read by the scanner.
private TOKEN

The tokens every run reads: words that are not string prefixes, integers that are not the start of a name or decimal, backquoted names whose closing quote does not begin a doubled one, and punctuation.

Comment and executable comment markers, dots and other literals are left to the scanner.
private ESCAPED_STRING

private STANDARD_STRING

private ESCAPED_DOUBLE_STRING

private STANDARD_DOUBLE_QUOTED

A double-quoted string where backslashes are ordinary characters, or an ANSI_QUOTES name; both escape quotes by doubling.
private ESCAPED_RUN

A run of tokens read in one native call, through the default delimiter or up to a token that needs the scanner.
private STANDARD_RUN

A run where backslashes are ordinary characters in strings.
private ANSI_RUN

A run where double quotes delimit names and backslashes are escapes in strings.

Properties

protected int $length from  Lexer
protected int $position from  Lexer
protected int $statementStart from  Lexer
protected int $tokenStart

The start of the token or terminator most recently scanned.

 from  Lexer
protected bool $ended from  Lexer
protected int $windowStart from  Lexer
bool $noBackslashEscapes

Whether backslashes in string literals are ordinary characters.
bool $ansiQuotes

Whether double quotes delimit identifiers rather than strings.

Methods

__construct(string $sql, bool $mariaDb = false)

Create a lexer, reading MariaDB executable comments when requested.

TokenStream|null
nextStatement()

Begin the next non-empty statement, or return null at the end of the input.

from  Lexer
void
read(array $types, array $starts, array $ends, array $texts)

Append a run of regular tokens, or one token that needs the scanner.

TokenType
type(string $text)

Get the type of a token appended without one, which follows from its upper-case text.

array|null
matchRun(string $pattern)

Match a run of tokens at the position, or return null where the scanner must read the next token.

from  Lexer
array
skipStatement()

Skip the rest of the current statement and return the offsets of its terminator.

from  Lexer
int|null
skipGroup(int $depth)

Skip to the parenthesis that closes the given number of open groups and return its start.

from  Lexer
string
value(TokenType $type, int $start, int $end)

Decode a token's value.

from  Lexer
int
line(int $offset)

Get the one-based line containing a source offset.

from  Lexer
ParseException
error(string $message, int $offset)

Create a parse error located at a source offset.

from  Lexer
TokenType
scan()

Scan one token or the statement terminator.

from  Lexer
void
finishStatement()

Mark the current statement as ended at its terminator.

from  Lexer
TokenType
scanToken()

Scan the token starting at the current position, or a terminator as End.

void
skip()

Scan to the end of the current statement.

from  Lexer
int
terminator()

Get the length of the delimiter at the current position, or zero.

bool
skipCommand()

Apply a mysql client DELIMITER command between statements.

void
startStatement()

Reset the qualification state at a statement boundary.

void
endStatement()

Handle input that belongs to a statement after its terminator.

from  Lexer
void
skipWhitespace()

Skip whitespace and comments, failing at an unterminated executable comment.

bool
skipComment()

Skip comments and executable comment markers.

void
skipLine()

Skip to the start of the next line.

from  Lexer
void
scanQuoted(string $quote, bool $backslashes, string $description)

Scan a quoted run whose opening quote is at the position, escaped by doubling and optionally backslashes.

from  Lexer
string
word(int $start, int $end)

Normalize an unquoted identifier or keyword.

from  Lexer
string
identifier(int $start, int $end)

Decode a quoted identifier.

from  Lexer
string
string(TokenType $type, int $start, int $end)

Decode a string literal, including backslash escapes and prefixes.

Details

at line 79
__construct(string $sql, bool $mariaDb = false)

Create a lexer, reading MariaDB executable comments when requested.

Parameters

string $sql
bool $mariaDb

in Lexer at line 77
TokenStream|null nextStatement()

Begin the next non-empty statement, or return null at the end of the input.

Return Value

TokenStream|null

at line 87
void read(array $types, array $starts, array $ends, array $texts)

Append a run of regular tokens, or one token that needs the scanner.

Parameters

array $types
array $starts
array $ends
array $texts

Return Value

void

at line 133
TokenType type(string $text)

Get the type of a token appended without one, which follows from its upper-case text.

Parameters

string $text

Return Value

TokenType

in Lexer at line 149
protected array|null matchRun(string $pattern)

Match a run of tokens at the position, or return null where the scanner must read the next token.

Dialects read runs of ordinary tokens with one native match each, so only tokens that need boundary handling go through the scanner. A statement's first token is matched alone, as parsers skip most statements after it.

Parameters

string $pattern

Return Value

array|null

the upper-case texts and their offsets from $windowStart

in Lexer at line 194
array skipStatement()

Skip the rest of the current statement and return the offsets of its terminator.

Return Value

array

in Lexer at line 208
int|null skipGroup(int $depth)

Skip to the parenthesis that closes the given number of open groups and return its start.

Returns null when the statement ends first.

Parameters

int $depth

Return Value

int|null

in Lexer at line 226
string value(TokenType $type, int $start, int $end)

Decode a token's value.

Parameters

TokenType $type
int $start
int $end

Return Value

string

in Lexer at line 239
int line(int $offset)

Get the one-based line containing a source offset.

Parameters

int $offset

Return Value

int

in Lexer at line 247
ParseException error(string $message, int $offset)

Create a parse error located at a source offset.

Parameters

string $message
int $offset

Return Value

ParseException

in Lexer at line 255
protected TokenType scan()

Scan one token or the statement terminator.

Return Value

TokenType

in Lexer at line 272
protected void finishStatement()

Mark the current statement as ended at its terminator.

Return Value

void

at line 152
protected TokenType scanToken()

Scan the token starting at the current position, or a terminator as End.

Return Value

TokenType

in Lexer at line 286
protected void skip()

Scan to the end of the current statement.

Return Value

void

at line 217
protected int terminator()

Get the length of the delimiter at the current position, or zero.

Return Value

int

at line 227
protected bool skipCommand()

Apply a mysql client DELIMITER command between statements.

Return Value

bool

at line 243
protected void startStatement()

Reset the qualification state at a statement boundary.

Return Value

void

in Lexer at line 317
protected void endStatement()

Handle input that belongs to a statement after its terminator.

Return Value

void

at line 252
protected void skipWhitespace()

Skip whitespace and comments, failing at an unterminated executable comment.

Return Value

void

at line 264
protected bool skipComment()

Skip comments and executable comment markers.

Return Value

bool

in Lexer at line 363
protected void skipLine()

Skip to the start of the next line.

Return Value

void

in Lexer at line 373
protected void scanQuoted(string $quote, bool $backslashes, string $description)

Scan a quoted run whose opening quote is at the position, escaped by doubling and optionally backslashes.

Parameters

string $quote
bool $backslashes
string $description

Return Value

void

in Lexer at line 402
protected string word(int $start, int $end)

Normalize an unquoted identifier or keyword.

Parameters

int $start
int $end

Return Value

string

in Lexer at line 410
protected string identifier(int $start, int $end)

Decode a quoted identifier.

Parameters

int $start
int $end

Return Value

string

at line 325
protected string string(TokenType $type, int $start, int $end)

Decode a string literal, including backslash escapes and prefixes.

Parameters

TokenType $type
int $start
int $end

Return Value

string