Token

A syntactic token.

Tokens describe how a span of source text should be presented (for syntax highlighting, say), which is a different question from what it is -- Node answers that. A token that names a declaration reports it as an opaque NodeId in node, resolvable with Unit.node.

Example

for token = unit.tokens()
  echo $token.kind $token.span

Fields

context @ (:CALL: | nil)

The syntactic context the token appears in: :CALL: for a callee position, or nil.

kind @ TokenKind

The token kind.

node @ (NodeId | nil)

The ID of the declaration this token names, or nil when it names none.

span @ Span

The span of source text the token covers.