Module: lib/bundler/scanner

Source-position primitives shared by every part of the bundler.

Why Avenx scans rather than parses

Everything else in this compiler reads source with a purpose-built scanner — the HTML tokenizer, the declaration reader, the Atlas reference scanner — and the bundler follows the same rule for the same reason: a dependency-free build is a property of the project, not an accident of what was convenient.

The bundler does not need a full ECMAScript parser. It needs to know three things about a module: which import/export declarations it contains, where each top-level statement begins and ends, and which top-level names it declares. All three are answerable from a scanner that knows where source text is not code.

What "is not code" means here

String literals, template literals (including nested ${} expressions), regular-expression literals, line comments and block comments. Every one of them can contain the characters this file searches for — a { inside a string would otherwise corrupt every depth calculation downstream, and the word import inside a comment would otherwise become a phantom dependency.

Distinguishing / as division from / as the start of a regex is the one genuinely ambiguous case in JavaScript lexing. regexAllowedAfter resolves it the way every hand-written JS lexer does: by looking at the last significant token.

Source:

Classes

CodeMask

Members

(inner, constant) COMMENT :number

Mask value for a comment: not code, and its whitespace carries no meaning.

Type:
  • number
Source:

(inner, constant) KEYWORDS_BEFORE_REGEX :Set:.<string:>

Keywords after which a / begins a regular expression rather than division.

Type:
  • Set:.<string:>
Source:

(inner, constant) LITERAL :number

Mask value for a string, template or regex literal: not code, and its text is data that must survive verbatim.

Type:
  • number
Source:

(inner, constant) PUNCTUATORS_BEFORE_REGEX :Set:.<string:>

Characters that can legally precede a regular-expression literal.

After any of these, a / starts a regex; after an identifier, a number, or a closing bracket it is division. The exceptions to "closing bracket means division" are keywords such as return and typeof, handled by KEYWORDS_BEFORE_REGEX.

Type:
  • Set:.<string:>
Source:

Methods

(static) matchBracket(source, mask, index) → {number}

Finds the offset just past a balanced bracket run starting at index.

Parameters:
Name Type Description
source string

The source text.
mask CodeMask

Mask for the same source.
index number

Offset of the opening bracket.
Source:
Returns:

Offset one past the matching close, or source.length.

Type
number

(static) regexAllowedAfter(source, index) → {boolean}

Decides whether a / at index opens a regular-expression literal.

Parameters:
Name Type Description
source string

The full source text.
index number

Offset of the /.
Source:
Returns:

True when a regex literal starts here.

Type
boolean

(static) scanRegions(source) → {Array:.<Region:>}

Walks a source and reports every non-code region.

Template literals are reported as a single region spanning the whole literal, including any ${} substitutions. Code inside a substitution is therefore not scanned at all — which is correct for this bundler's purposes, because an import declaration cannot appear inside an expression and a top-level statement boundary cannot fall inside a template literal.

Parameters:
Name Type Description
source string

The source text.
Source:
Returns:

Regions in ascending order of start.

Type
Array:.<Region:>

(static) statementEnd(source, mask, start) → {number}

Finds the end of the top-level statement that begins at start.

"End" means the offset one past its terminating semicolon, or one past the closing brace of a block-bodied declaration, or the end of the line for an ASI-terminated statement. Only bracket depth at code positions is consulted, so a ; inside a string or an object literal never terminates anything.

Parameters:
Name Type Description
source string

The source text.
mask CodeMask

Mask for the same source.
start number

Offset of the statement's first character.
Source:
Returns:

Exclusive end offset.

Type
number

Type Definitions

Region

A single lexical region the scanner recognises.

Type:
  • object
Properties:
Name Type Description
kind 'code' | 'string' | 'template' | 'regex' | 'line-comment' | 'block-comment'

What the region is.
start number

Inclusive start offset.
end number

Exclusive end offset.
Source: