indexion-kgf

indexion kgf

Inspect and debug KGF language specs by viewing tokens, parse events, and extracted edges.

When to Use

• User wants to debug how indexion processes a specific file
• User is developing or modifying a KGF spec
• User asks "how does indexion parse this file?"
• Verifying that tokenization/parsing works correctly for a language
• Debugging grep patterns: when a grep pattern doesn't match, use kgf tokens to see the actual token kinds

Subcommands

indexion kgf list — List Installed Specs

indexion kgf list

indexion kgf update — Update All Specs

Download the latest specs from GitHub.

indexion kgf update

indexion kgf add — Install a Single Spec

indexion kgf add <spec-name>

indexion kgf inspect — Full Inspection

Show tokens, events, and edges all at once.

indexion kgf tokens — Tokenization Only

Show how a file is tokenized.

indexion kgf events — Parse Events Only

Show parse events generated from tokens.

indexion kgf edges — Extracted Edges Only

Show the dependency edges extracted from a file.

Options

Relationship to grep

indexion grep uses KGF tokenization under the hood. Pattern aliases (pub → KW_pub) are derived from the === lex section of KGF specs.

When a grep pattern doesn't match as expected:

Common token kinds (MoonBit):

• KW_pub, KW_fn, KW_struct, KW_enum, KW_type, KW_trait, KW_let, KW_for
• Ident (lowercase identifiers), TypeIdent (PascalCase type names)
• LPAREN, RPAREN, LBRACE, RBRACE, LBRACKET, RBRACKET
• NL (newline), SKIP (whitespace — filtered from grep patterns)
• DocComment, DocLine, DocSection, LineComment, BlockComment
• String, Number, Char

Workflow

1. Run indexion kgf inspect <file> to see the full processing pipeline
2. If something looks wrong, drill down with tokens, events, or edges
3. Compare with the KGF spec file (kgfs/<lang>.kgf) to diagnose issues

KGF Development Pitfalls

Common bugs found when writing or modifying KGF specs:

PEG Item Ordering (first-match-wins)

KGF uses PEG parsing. In Item -> A / B / C, if A matches, B and C are never tried. DocComment as a standalone alternative before declaration rules will consume doc comments that should be attached to declarations.

NL Between Doc and Keyword

Source code has newlines between doc comments and declarations. Without NL? or NL*, the optional doc capture fails silently:

Bottom-Up Event Order (bind/scope)

Events fire bottom-up: child rules before parent rules. If ExportDecl wraps FunctionDecl, FunctionDecl fires first. Use bind/$scope to pass data from child to parent:

Token Priority Conflicts

Tokens defined earlier take priority. A generic Operator /[=+\-*]+/ before EQ /=/ will consume = as Operator. Define specific tokens first:

Verifying Doc Extraction

After modifying a KGF, always verify doc appears in declares edges: