upstream-sqlc/internal/sql/ast/CLAUDE.md

AST Package - Claude Code Guide

This package defines the Abstract Syntax Tree (AST) nodes used by sqlc to represent SQL statements across all supported databases (PostgreSQL, MySQL, SQLite).

Key Concepts

Node Interface

All AST nodes implement the Node interface with:

• Pos() int - returns the source position
• Format(buf *TrackedBuffer, d format.Dialect) - formats the node back to SQL

TrackedBuffer

The TrackedBuffer type (print.go) handles SQL formatting with dialect-specific behavior:

• astFormat(node Node) - formats any AST node
• join(list *List, sep string) - joins list items with separator
• WriteString(s string) - writes raw SQL
• QuoteIdent(name string) - quotes identifiers (dialect-specific)
• TypeName(ns, name string) - formats type names (dialect-specific)

Pretty printing

The doc renderer lives in ast/printer (printer.Buffer), which knows nothing about AST nodes; TrackedBuffer embeds it and adds the AST-facing layer (astFormat, comment emission, anchors). TrackedBuffer records a Wadler-style document (the model behind Prettier and ruff): Format methods emit text plus layout tokens, and the renderer decides which break opportunities become newlines.

• Line() - a space when the group fits on one line, a newline otherwise
• Softline() - nothing when the group fits, a newline otherwise
• Group() / EndGroup() - a region laid out flat when its width fits
• Indent() / EndIndent() - one level deeper (2 spaces) after breaks
• Breaker() - forces every enclosing group to break (measures as infinitely wide)
• joinComma(list) - joins items with , + Line()
• condition(node) - clause-level AND/OR chain without outer parentheses, one branch per line when broken

ast.Format(n, d) renders on a single line (all breaks collapse); ast.Pretty(n, d, width) breaks lines to fit width columns. Statement Format methods open a group and put Line() before each clause keyword (FROM, WHERE, ...), so a statement that fits stays on one line and a long one breaks at clause boundaries. When adding a Format method, write tokens so the flat rendering is correct SQL; layout tokens are optional.

The fmt command does not pass a real width: like gofmt it never rewraps on its own, so it prints with an effectively infinite width and defers to the author's line breaks instead. AttachComments records, for each emission-point marker, whether its neighbouring printed nodes sat on different source lines; when they did, boundary() emits a Breaker() there on the real print, so the group the author broke stays broken and everything else stays flat.

Comments

ast.File{Stmts, Comments} is what a comment-surfacing parser returns (SQLite via meyer's ParseFile). AttachComments(raw, d, comments, src) classifies each comment once, against a dry-run of the printer: source positions and line numbers decide trailing (same line as the code before) vs leading, and each comment is attached to the emission point — node or clause/list boundary — where the printer will reach it. From then on positions are never consulted: PrettyWithComments(n, d, width, table) emits by node identity, which is what lets edited or synthetic trees print their comments correctly (the dave/dst model; each record also keeps the node the comment followed, for future rewriting tools). A line comment forces every enclosing group to break — hardline and breaker tokens measure as infinitely wide — so commented statements format instead of collapsing. Emission points (beforeClause, boundary in joinComma/condition) double as classification markers on the dry run, guaranteeing attach-time decisions and print-time emission agree.

Dialect Interface

Dialect-specific formatting is handled via the Dialect interface:

type Dialect interface {
    QuoteIdent(string) string
    TypeName(ns, name string) string
    Param(int) string      // $1 for PostgreSQL, ? for MySQL
    NamedParam(string) string // @name for PostgreSQL, :name for SQLite
    Cast(string) string
}

Adding New AST Nodes

When adding a new AST node type:

1. Create the node file (e.g., variable_expr.go):

package ast

type VariableExpr struct {
    Name     string
    Location int
}

func (n *VariableExpr) Pos() int {
    return n.Location
}

func (n *VariableExpr) Format(buf *TrackedBuffer) {
    if n == nil {
        return
    }
    buf.WriteString("@")
    buf.WriteString(n.Name)
}

2. Add to astutils/walk.go - Add a case in the Walk function:

case *ast.VariableExpr:
    // Leaf node - no children to traverse

3. Add to astutils/rewrite.go - Add a case in the Apply function:

case *ast.VariableExpr:
    // Leaf node - no children to traverse

4. Update the parser/converter - In the relevant engine (e.g., dolphin/convert.go for MySQL)

Helper Functions for Format Methods

• set(node Node) bool - returns true if node is non-nil and not an empty List
• items(list *List) bool - returns true if list has items
• todo(node) Node - placeholder for unimplemented conversions (returns nil)

Common Node Types

Statements

• SelectStmt - SELECT queries with FromClause, WhereClause, etc.
• InsertStmt - INSERT with Relation, Cols, SelectStmt, OnConflictClause
• UpdateStmt - UPDATE with Relations, TargetList, WhereClause
• DeleteStmt - DELETE with Relations, FromClause (for JOINs), Targets

Expressions

• A_Expr - General expression with operator (e.g., a + b, @param)
• ColumnRef - Column reference with Fields list
• FuncCall - Function call with Func, Args, aggregation options
• TypeCast - Type cast with Arg and TypeName
• ParenExpr - Parenthesized expression
• VariableExpr - MySQL user variable (e.g., @user_id)

Table References

• RangeVar - Table reference with schema, name, alias
• JoinExpr - JOIN with Larg, Rarg, Jointype, Quals/UsingClause

MySQL-Specific Nodes

• VariableExpr - User variables (@var), distinct from sqlc's @param syntax
• IntervalExpr - INTERVAL expressions
• OnDuplicateKeyUpdate - MySQL's ON DUPLICATE KEY UPDATE clause
• ParenExpr - Explicit parentheses (TiDB parser wraps expressions)

Important Distinctions

MySQL @variable vs sqlc @param

• sqlc named parameters (@param) never reach the AST: internal/sql/preprocess replaces them with the dialect's native placeholder before parsing, so engines only ever see a ParamRef
• @name is only sqlc syntax where the preprocessor's dialect says so. MySQL is excluded, so @user_id there stays a user variable (VariableExpr)

Type Modifiers

• TypeName.Typmods holds type modifiers like varchar(255)
• For MySQL, only populate Typmods for types where length is user-specified:
  • VARCHAR, CHAR, VARBINARY, BINARY - need length
  • DATETIME, TIMESTAMP, DATE - internal flen should NOT be output