2 Lexing and Parsing

This library provides facilities for lexing and parsing JavaScript source. It can be required via:

(require (planet dherman/javascript:9:1/parse))

2.1 Input Sources

An input-source is either a string, a path, or an input port.

(input-source? x) → boolean?

x : any

Determines whether x is an input-source.

(input-source->input-port in) → input-port?

in : input-source?

Produces an input port for reading from in.

2.2 Source Representation

(struct position (offset line col))

offset : exact-positive-integer?

line : exact-positive-integer?

col : exact-nonnegative-integer?

Re-exported from the PLT Scheme collection parser-tools/lex.

(struct region (source start end))

source : any

start : position?

end : position?

Represents a region of text from start (inclusive) to end (exclusive).

(struct regexp-contents (pattern global? case-insensitive?))

pattern : string

global? : boolean?

case-insensitive? : boolean?

The source to a regular expression token, with regular expression pattern pattern and flags global?, representing the /g option, and case-insensitive?, representing the /i option.

(struct token (type contents location))

type : symbol?

contents   :   (optional/c  (or/c string? number? symbol? regexp-contents?))

location : region?

A single token of input.

(region->string rgn) → string?

rgn : region?

Produces a string representation of a region, convenient for debugging and error reporting.

2.3 Syntax Errors

(struct (exn:fail:syntax exn:fail) (source location text))

source : (implementation?/c lexer<%>)

location : region?

text : any

2.4 Lexer Objects

lexer<%> : interface?

A JavaScript lexical scanner.

(send a-lexer fail fmt arg ...) → any

fmt : string?

arg : any

Raises an exn:fail:syntax exception with error message (format fmt arg ...).

(send a-lexer fail/loc loc text fmt arg ...) → any

loc : region?

text : any

fmt : string?

arg : any

Raises an exn:fail:syntax exception with source location loc and error message (format fmt arg ...).

(send a-lexer done?) → boolean?

Determines whether the end of input has been reached.

(send a-lexer current-token) → token?

Produces the current token in the lexer’s input stream.

(send a-lexer match type) → (optional token?)

type : symbol?

Attempts to read a token of type type, producing the token on success and #f on failure.

(send a-lexer must-match type) → token?

type : symbol?

Attempts to read a token of type type, produce the token on success and raising an exn:fail:syntax exception on failure.

(send a-lexer peek-token [skip]) → token?

skip : natural-number/c = 0

Produces the next token (after skipping skip tokens) in the input stream without changing the current position in the input stream.

(send a-lexer peek-token/infix-operator [skip]) → token?

skip : natural-number/c = 0

Similar to peek-token, but assumes the lexer is being used in a parsing state that expects an infix operator.

(send a-lexer peek-token/same-line) → token?

Similar to peek-token, but does not lex past end-of-line sequences.

(send a-lexer read-token [skip]) → token?

skip : natural-number/c = 0

Produces the next token (after skipping skip tokens) in the input stream and updates the current position in the input stream.

(send a-lexer read-token/same-line) → token?

Similar to read-token, but does not lex past end-of-line sequences.

(send a-lexer unread-token) → any

Rewinds the current position in the input stream by one token.

(send a-lexer skip-whitespace) → any

Skips past any whitespace in the underlying input port.

lexer% : class?

superclass: object%

extends:  lexer<%>

An implementation of lexer<%>, a JavaScript lexical scanner.

(new lexer% [port port] [[name name]]) → (is-a?/c lexer%)

port : input-port?

name : any = (object-name port)

Constructs a new lexer% which reads tokens from port. The optional name argument is used for source location information and error reporting.

2.5 Lexing Functions

(lex in) → (-> token?)

in : input-source?

Convenience function for producing a functional lexer from an input source.

2.6 Parser Objects

parser<%> : interface?

A JavaScript parser.

(send a-parser parse-source-element) → SourceElement?

Parses a single source element.

(send a-parser parse-source-elements)

→ (listof SourceElement?)

Parses a sequence of source elements.

(send a-parser parse-expression) → Expression?

Parses a single expression.

(send a-parser skip-empty-tokens) → any

Skips past meaningless whitespace tokens.

parser% : class?

superclass: object%

extends:  parser<%>

An implementation of parser<%>, a JavaScript parser.

(new parser% [lexer lexer]) → (is-a?/c parser%)

lexer : (implementation/c lexer<%>)

Constructs a new parser% which receives tokens from lexer.

(input-source->parser in) → (is-a?/c parser<%>)

in : input-source?

Produces a JavaScript parser for the input from in.

2.7 Parsing Functions

(parse-program-unit in) → (listof SourceElement?)

in : input-source?

Parses a JavaScript program unit from in.

(parse-expression in) → Expression?

in : input-source?

Parses a JavaScript expression from in.

(parse-function-constructor args body) → FunctionExpression?

args : string?

body : string?

Uses the arguments constructed from the JavaScript Function constructor to parse a function expression. The args string represents the comma-separated formal parameter list, and the body string represents the function body (not including the surrounding braces).

(parse-source-element in) → SourceElement?

in : input-source?

Parses a JavaScript source element from in.