Tree-sitter Explained: Parsing, Queries and How Code Tools Use It

If you have used syntax highlighting in Neovim, Helix or Zed, browsed code on GitHub, or run a structural search tool like ast-grep, you have probably used tree-sitter without knowing it. It is a parser generator and incremental parsing library, originally created by Max Brunsfeld at GitHub for the Atom editor, and it has quietly become the default way developer tools turn source code into something they can reason about.

This post explains what tree-sitter is, how its parsing model works, how to write tree-sitter queries (the part most people actually need), how editors and code tools use it, and where its limits are.

Key takeaways

• Tree-sitter generates fast parsers from a grammar written in JavaScript, and produces a concrete syntax tree for a source file.
• It is incremental (re-parsing after an edit reuses the old tree) and error tolerant (broken code still produces a useful tree with ERROR nodes).
• Grammars exist for well over a hundred languages, and the same API works for all of them, which is why multi-language tools love it.
• The query language uses S-expression patterns with captures (@name), fields (name:), predicates (#eq?, #match?), anchors, quantifiers and alternations.
• Editors use queries for highlighting (highlights.scm), scoping (locals.scm), folding and text objects; code navigation tools use tags.scm to find definitions and references.
• Tree-sitter is syntax only. It does not know types, resolve imports, or tell you which function a call refers to. Tools build that on top.

What tree-sitter is

Tree-sitter has three parts:

1. A grammar DSL. You describe a language's syntax in a grammar.js file using functions like seq, choice, repeat, optional, field and prec.
2. A parser generator. tree-sitter generate turns that grammar into a parser written in C (src/parser.c) with no runtime dependencies.
3. A runtime library with bindings for C, Rust, JavaScript (Node and WebAssembly), Python, Go, Swift and others, which loads those parsers, parses text, and lets you walk and query the resulting tree.

The parsers it generates are LR(1) with GLR support, which means they can handle ambiguous parts of a grammar by exploring several parses at once and keeping the one that works. In practice this is what lets tree-sitter cope with the messier corners of languages like C++ and JavaScript.

A concrete syntax tree, with named nodes

Parse this JavaScript:

function add(a, b) { return a + b }

and tree-sitter's CLI prints the tree as an S-expression:

(program
  (function_declaration
    name: (identifier)
    parameters: (formal_parameters (identifier) (identifier))
    body: (statement_block
      (return_statement
        (binary_expression left: (identifier) right: (identifier))))))

(The real output also includes a start and end position for each node, like [0, 0] - [0, 36].)

A few things to notice. Nodes have types (function_declaration, identifier). Some children are labeled with fields (name:, parameters:, body:), which is what makes queries precise. And the printed tree shows only named nodes; punctuation and keywords like (, { and return exist as anonymous nodes and can be matched in queries by writing them as strings, such as "return".

Incremental parsing

Editors parse on every keystroke. Re-parsing a 10,000-line file from scratch each time would be too slow, so tree-sitter lets you tell it what changed and reuses everything else:

tree.edit({
  startIndex: 13, oldEndIndex: 13, newEndIndex: 16,
  startPosition: { row: 0, column: 13 },
  oldEndPosition: { row: 0, column: 13 },
  newEndPosition: { row: 0, column: 16 },
})
const newTree = parser.parse(newSource, tree)

Only the parts of the tree affected by the edit are rebuilt. tree.getChangedRanges(newTree) then tells the editor which regions need re-highlighting.

Error tolerance

Code in an editor is broken most of the time: half-typed expressions, missing braces. Tree-sitter recovers from errors locally, inserting ERROR nodes for text it cannot place and MISSING nodes for tokens it expected but did not find. The rest of the tree stays intact, so highlighting and navigation keep working in the parts of the file that are valid. Code-analysis tools benefit too: a repository with one file that does not compile can still be indexed.

Trying it from the command line

Install the CLI, then point it at grammars on disk:

npm install -g tree-sitter-cli     # or: cargo install tree-sitter-cli, or brew install tree-sitter

tree-sitter init-config            # creates a config listing parser directories
git clone https://github.com/tree-sitter/tree-sitter-javascript ~/src/tree-sitter-javascript

tree-sitter parse example.js       # print the syntax tree
tree-sitter query calls.scm example.js   # run a query and print captures

The config file lists parser-directories where the CLI looks for tree-sitter-* grammar folders; put the clone under one of them. tree-sitter playground starts a local web page where you can paste code and see the tree and query results live, which is the fastest way to learn a grammar's node names.

The query language

Queries are where tree-sitter becomes useful for building tools. A query is a set of patterns, written as S-expressions that mirror the tree structure, with captures that name the nodes you want back.

Basic patterns and captures

Find every function declaration and capture its name:

(function_declaration
  name: (identifier) @function.name)

Find calls to plain functions and to methods:

(call_expression
  function: (identifier) @call.name)

(call_expression
  function: (member_expression
    property: (property_identifier) @method.name))

Find import sources in modern versions of the JavaScript grammar:

(import_statement
  source: (string (string_fragment) @import.source))

A capture name can be anything; dots are just a naming convention. A pattern matches when the structure matches, and children you do not mention are ignored, so (function_declaration name: (identifier) @n) matches regardless of what the parameters and body contain.

Wildcards, alternations and quantifiers

; Any node as the value of a variable declarator
(variable_declarator name: (identifier) @var value: (_) @value)

; Either a function declaration or a method definition
[
  (function_declaration name: (identifier) @fn)
  (method_definition name: (property_identifier) @fn)
]

; One or more comments immediately followed by a function: doc comments
(
  (comment)+ @doc
  .
  (function_declaration name: (identifier) @fn)
)

(_) matches any named node and _ matches any node, named or anonymous. Square brackets are alternation. +, * and ? are quantifiers. Parentheses around several patterns group siblings.

Anchors

The . anchor constrains position. Between two sibling patterns it means "immediately adjacent". At the start or end of a child list it means "first" or "last":

; The first identifier in an array literal
(array . (identifier) @first)

; The last statement in a block
(statement_block (_) @last .)

Negated fields

!field matches nodes that do not have that field. In the TypeScript grammar:

; Classes without generic type parameters
(class_declaration
  name: (type_identifier) @class
  !type_parameters)

Predicates

Predicates filter matches by text. They go inside the pattern's outer parentheses:

; SCREAMING_CASE identifiers, likely constants
((identifier) @constant
  (#match? @constant "^[A-Z][A-Z0-9_]+$"))

; Calls to console.log, console.warn, console.error
((call_expression
   function: (member_expression
     object: (identifier) @obj
     property: (property_identifier) @method))
  (#eq? @obj "console")
  (#any-of? @method "log" "warn" "error"))

#eq?, #match? and #any-of? (plus their #not- variants) are widely supported. Predicates are evaluated by the host binding rather than the core C library, so check your binding's documentation before relying on less common ones. Editors such as Neovim add their own predicates and directives on top.

Using queries from code

Here is a complete Node script that lists functions and the calls inside them:

// list-calls.js: node list-calls.js file.js
const fs = require('node:fs')
const Parser = require('tree-sitter')
const JavaScript = require('tree-sitter-javascript')

const parser = new Parser()
parser.setLanguage(JavaScript)

const source = fs.readFileSync(process.argv[2], 'utf8')
const tree = parser.parse(source)

const query = new Parser.Query(JavaScript, `
  (function_declaration
    name: (identifier) @fn
    body: (statement_block) @body)
`)
const callQuery = new Parser.Query(JavaScript, `
  (call_expression function: [
    (identifier) @callee
    (member_expression property: (property_identifier) @callee)
  ])
`)

for (const match of query.matches(tree.rootNode)) {
  const fn = match.captures.find((c) => c.name === 'fn').node
  const body = match.captures.find((c) => c.name === 'body').node
  const callees = callQuery.captures(body).map((c) => c.node.text)
  console.log(`${fn.text} (line ${fn.startPosition.row + 1}) calls: ${[...new Set(callees)].join(', ')}`)
}

matches returns each pattern match with its captures grouped; captures returns a flat list of captured nodes in document order. Every node exposes its type, text, start and end positions, parent, children and named fields, so once you have a capture you can walk anywhere in the tree.

In the browser, the web-tree-sitter package provides the same model over WebAssembly. You load the runtime and a .wasm file for each grammar (built with tree-sitter build --wasm), then parse and query as above. The exact import style changed between releases, so follow the README for the version you install.

How tools use tree-sitter

Syntax highlighting. Grammars ship a queries/highlights.scm file mapping nodes to capture names like @function, @string, @keyword. The editor maps those names to theme colors. Because highlighting is based on the tree, a function name is colored as a function whether it appears in a declaration or a call.

Scopes and local variables. locals.scm marks scopes (@local.scope), definitions (@local.definition) and references (@local.reference) so editors can highlight a variable consistently and jump to a local definition.

Code navigation. tags.scm captures definitions and references using a standard naming scheme:

(function_declaration name: (identifier) @name) @definition.function
(call_expression function: (identifier) @name) @reference.call

The tree-sitter tags command runs these and prints a symbol index. This is the same idea behind search-based "go to definition" on GitHub, and behind the repository maps some AI coding tools build to show a model the shape of a codebase.

Editor features. Neovim, Helix, Zed and Emacs (via the built-in treesit package since Emacs 29) use tree-sitter for highlighting, indentation, folding, and structural text objects such as "select this function".

Structural search and diff. ast-grep lets you search and rewrite code using patterns that compile to tree-sitter matches. Difftastic uses tree-sitter to diff syntax trees instead of lines, so moving a function or reformatting a block produces a readable diff.

Code graphs. Multi-language code intelligence tools parse every file with tree-sitter, extract definitions, imports and calls with queries, and connect them into a graph of the repository. That graph is what powers dependency graphs, call graphs, and the structural context that helps AI agents, discussed in code knowledge graph vs embeddings.

Limits

Tree-sitter is a parser, and it is honest about it. Things it does not do:

• No semantic analysis. It does not know types, does not resolve import { x } from './y' to a file, and cannot tell which send method a call refers to. Tools add name resolution on top, either heuristically or with systems like GitHub's stack graphs, or they defer to a language server.
• Grammar quality varies. Grammars for mainstream languages are mature. Less common languages, and languages with heavy preprocessing or macros (C macros, Rust macros, templating languages), produce trees that are less faithful.
• Node names differ by grammar. A function is function_declaration in JavaScript, function_definition in Python, function_item in Rust and method_declaration in Go and Java. Queries are written per language.
• Embedded languages need injections. SQL in a string or JavaScript in an HTML script tag requires an injections.scm query and a second parser.
• Grammar changes can break queries. When a grammar renames a node or field, queries that depend on it stop matching. Pin grammar versions in tools.

Writing a grammar, briefly

If you need a grammar for an internal DSL, the workflow is short: tree-sitter init scaffolds a project, you write rules in grammar.js, tree-sitter generate builds the parser, and tree-sitter test runs corpus tests in test/corpus/*.txt, each of which pairs a code snippet with its expected tree. Start from an existing grammar for a similar language; the official ones under the tree-sitter GitHub organization are the best references.

Tree-sitter in the browser

Because grammars compile to WebAssembly, tree-sitter can run anywhere JavaScript runs, without a server. OpenVisio uses this: when you paste a repository into Explore, it parses the files with tree-sitter WASM inside a web worker in your browser, extracts definitions, imports and calls, and builds the architecture map and code graph locally, so the source never leaves your machine. The same kind of graph is what coding agents query over MCP through tools like find_symbol and trace_calls. If you want to see what a tree-sitter-derived map of a real codebase looks like before building your own tooling, that is a quick way to try it.