Skip to content
Merged
11 changes: 6 additions & 5 deletions docs/spec/code-organization.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,11 +147,12 @@ compilation target nests as `ir/{dialect}/{target}/{category}/<name>.py`;
target-neutral abstractions stay at `ir/{dialect}/{category}/`. For
example the whole MMA surface is target-owned — the `Mma` op, the
`MmaOpSpec` / `MmaAtom` descriptors, the CUDA SM80 instruction spec, and its
fragment layouts all live under `ir/tir/cuda/nn/` (`mma.py` + `mma_atom.py`),
and the HIR per-shape `Mma_SM80_*` / `Wgmma_SM90_*` ops under
`ir/hir/cuda/nn/mma.py` — because an MMA instruction fixes a concrete hardware
op. (`codegen/` and `runtime/` are **target-first** instead — their primary
axis is the target — so each tree is organized by its own primary axis.)
fragment layouts all live under `ir/tir/cuda/nn/` (`mma.py` + `mma_atom.py`).
The backend-bound construction stays in TIR: HIR is the checking reference
side, and carrying the instruction name in that reference would make two GPU
targets require different HIR references. (`codegen/` and `runtime/` are
**target-first** instead — their primary axis is the target — so each tree is
organized by its own primary axis.)

**Rule 2 — one (node, target) codegen = one file.** Each handler
lives at `codegen/<target>/tir/<category>/<name>.py`. Stmt emitters,
Expand Down
92 changes: 4 additions & 88 deletions docs/spec/hir.md
Original file line number Diff line number Diff line change
Expand Up @@ -507,9 +507,10 @@ class Binary(Op):
- constraints:
- Values follow torch pointwise semantics; dtypes do not promote. Both operands
MUST already carry the same `dtype`, and typeinfer MUST reject a mismatch. A
Python float scalar is given the other operand's float dtype by the authoring
surface, before it is an operand at all ([parser §2.1](./parser.md#21-syntax)); a Python
integer is not.
Python literal is an ordinary operand of the dtype it is written with — `f32`
for a float, `i64` for an integer — and the authoring surface MUST NOT adapt
it to its peer. The rejection MUST name `Cast` as the remedy, so the dtype a
value carries is the one the author wrote.
- The elementwise `min` / `max` kinds are also surfaced as `minimum` / `maximum`.
- Equal plain layouts, or one plain layout paired with `layout=None`, pass
through only when that layout describes the broadcast result. Otherwise two
Expand Down Expand Up @@ -1405,91 +1406,6 @@ class RoPE(Op):
- A non-`sum` Partial, multiple value-carrying Partials, or a Partial on a
secondary cache/index input MUST be rejected with a `Reshard` remedy.

##### CUDA matrix multiply-accumulate family

```python
class Mma(Op):
"""Provide the marker base for HIR CUDA matrix operations."""


class Mma_SM80_16x8x16(Mma):
"""Produce an SM80 16-by-8 accumulator fragment.

Attributes:
a: input; Left 16-by-16 fragment.
b: input; Right 16-by-8 fragment.
dtype_a: attribute; Left operand dtype.
dtype_b: attribute; Right operand dtype.
dtype_acc: attribute; Accumulator dtype.
a_layout: attribute; Left matrix orientation.
b_layout: attribute; Right matrix orientation.
"""

a: Tensor
b: Tensor
dtype_a: DType
dtype_b: DType
dtype_acc: DType
a_layout: str = "T"
b_layout: str = "N"


class Wgmma_SM90_64x128x16(Mma):
"""Produce an SM90 64-by-128 accumulator fragment.

Attributes:
a: input; Left fragment.
b: input; Right fragment.
dtype_a: attribute; Left operand dtype.
dtype_b: attribute; Right operand dtype.
dtype_acc: attribute; Accumulator dtype.
a_layout: attribute; Left matrix orientation.
b_layout: attribute; Right matrix orientation.
"""

a: Tensor
b: Tensor
dtype_a: DType
dtype_b: DType
dtype_acc: DType
a_layout: str = "T"
b_layout: str = "N"
```

- constraints:
- `Mma` is a marker base used for family dispatch and has no independent
callable parameter surface.
- HIR MMA is the two-input value `a @ b`; it has no accumulator input. The
evaluator converts the rounded operands to `dtype_acc` before multiplying.
The independent handwritten TIR MMA surface has an in-place accumulator,
but there is no HIR-to-TIR compile route between these contracts.
- `Mma_SM80_16x8x16` requires `a.shape == (16, 16)` and
`b.shape == (16, 8)`, and returns `(16, 8)`. `Wgmma_SM90_64x128x16`
requires `(64, 16)` and `(16, 128)`, and returns `(64, 128)`.
- Each operand dtype MUST equal its declared `dtype_a` / `dtype_b`. Supported
`(dtype_a, dtype_b, dtype_acc)` combinations are `(f16, f16, f16)`,
`(bf16, bf16, bf16)`, `(f16, f16, f32)`, `(bf16, bf16, f32)`, and
`(f32, f32, f32)`. Each orientation MUST be `N` or `T`; orientation names
the hardware encoding and does not transpose either logical HIR input.
- Plain logical input Types remain valid for evaluation and Analyze. Their
result has a fresh row-major logical layout when either operand has a plain
layout. Fully `Broadcast` `ShardLayout` inputs carry no real ownership and
therefore pin no result mesh.
- Only the known SM80 BF16/BF16/F32 TN A/B fragment pair in RMEM derives the
known C fragment. Its A/B layouts, `Split` bindings, thread topology, and
mesh shape MUST match. Mesh coordinate names do not participate in
identity; the derived C layout uses `a`'s physically compatible mesh. Any
other genuine SM80 shard claim or fragment in non-RMEM storage MUST fail
with an explicit `Reshard` / materialize-to-RMEM remedy.
- WGMMA has no representable fragment contract yet. A genuine WGMMA shard
claim MUST fail with an explicit `Reshard` remedy, while plain logical
WGMMA remains evaluable and analyzable.
- Cost is `2*M*N*K` in the operand dtype with exactly three traffic slots:
A read, B read, and result write. Byte residency and topology projection
come only from the selected operand/result Types.
- `HirToTirPass` MUST reject both concrete HIR MMA Ops by name before
emitting TIR; see [passes §7.1](./passes.md#71-hirtotirpass).

#### `ir/hir/sharding/`

`ShardLayout` and `Mesh` are type-system constructs, not Expr inputs
Expand Down
84 changes: 60 additions & 24 deletions docs/spec/parser.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,14 +100,14 @@ type-annotation ::= tensor
| scalar-type
signature ::= (name ':' type-annotation (',' name ':' type-annotation)*)?
return-type ::= type-annotation
loop-iterator ::= 'tile'
| 'range'
loop-iterator ::= 'tile' '(' expression ',' expression ')'
| 'range' '(' (expression | expression ',' expression | expression ','
expression ',' expression) ')'
loop-carry-statement ::= expression '=' expression
| 'for' name 'in' expression ':' loop-carry
| statement
loop-carry ::= (loop-carry-statement (newline loop-carry-statement)*)?
loop-header ::= 'for' identifier 'in' loop-iterator '(' (expression | name '=' expression)
(',' (expression | name '=' expression))* ')' ':' loop-carry
loop-header ::= 'for' identifier 'in' loop-iterator ':' loop-carry
loop-body ::= (statement (newline statement)*)?
for ::= 'for' name 'in' expression ':' loop-body
mesh-context ::= ('Mesh' | primary '.' identifier) '(' (expression | ('layout' | 'names')
Expand All @@ -132,9 +132,11 @@ subscript-index ::= '(' ((index-slice | index-endpoint) (',' (index-slice
| index-slice
| index-endpoint
subscript-expression ::= runtime-expression '[' subscript-index ']'
binary-expression ::= runtime-expression binary-op runtime-expression
| runtime-expression comparison-op runtime-expression
| runtime-expression boolean-op runtime-expression
matmul-expression ::= runtime-expression '@' runtime-expression
binary-expression ::= runtime-expression ('+' | '-' | '*' | '/' | '//' | '%') runtime-expression
| runtime-expression ('==' | '!=' | '<' | '<=' | '>' | '>=')
runtime-expression
| runtime-expression ('and' | 'or') runtime-expression
unary-expression ::= unary-op runtime-expression
name ::= identifier
constant ::= boolean-literal
Expand All @@ -144,6 +146,7 @@ tuple-expression ::= '(' (runtime-expression (',' runtime-expression)*)? ')
runtime-expression ::= op-call
| launch
| subscript-expression
| matmul-expression
| binary-expression
| unary-expression
| mesh-coordinate
Expand Down Expand Up @@ -173,39 +176,27 @@ function ::= 'def' name '(' signature ')' ('->' return-type)? ':' b
<!-- parser-constraints:start -->
| Owner | Situation | Rule | Statement | Source |
| --- | --- | --- | --- | --- |
| binary_expression | expression, slice_endpoint, subscript_index | CallBindingRule | A call must bind its arguments into a Call tuple. | src/tilefoundry/parser/pattern_nodes.py |
| binary_expression | expression, slice_endpoint, subscript_index | CallExpectedTypeRule | A call's inferred type must satisfy the expected expression type. | src/tilefoundry/parser/pattern_nodes.py |
| binary_expression | expression, slice_endpoint, subscript_index | CallTypeInferenceRule | A call's result type must be inferred from its binding. | src/tilefoundry/parser/pattern_nodes.py |
| binary_expression, matmul_expression, op_call, unary_expression | expression, slice_endpoint, subscript_index | CallBindingRule | A call must bind its arguments into a Call tuple. | src/tilefoundry/parser/pattern_nodes.py |
| binary_expression, matmul_expression, op_call, unary_expression | expression, slice_endpoint, subscript_index | CallExpectedTypeRule | A call's inferred type must satisfy the expected expression type. | src/tilefoundry/parser/pattern_nodes.py |
| binary_expression, matmul_expression, op_call, unary_expression | expression, slice_endpoint, subscript_index | CallTypeInferenceRule | A call's result type must be inferred from its binding. | src/tilefoundry/parser/pattern_nodes.py |
| dim_expr | dim_expr, layout_extent, layout_shape, tensor_dim_expr, tensor_optional_slot, tensor_shape | ShapeDimRule | A shape dimension must be an integer, DimVar, or expression. | src/tilefoundry/parser/ast_pattern.py |
| dtype | tensor_dtype | CanonicalDTypeRule | A dtype must resolve to a canonical DType. | src/tilefoundry/parser/ast_pattern.py |
| explicit_layout | tensor_optional_slot | LayoutPositionRule | A layout must be legal for its parser position. | src/tilefoundry/parser/ast_pattern.py |
| explicit_layout | tensor_optional_slot | LayoutShapeRule | A layout must have a valid non-boolean shape. | src/tilefoundry/parser/ast_pattern.py |
| explicit_layout, layout, placed_layout, plain_layout | layout_shape, tensor_optional_slot, tensor_shape | LayoutPositionRule | A layout must be legal for its parser position. | src/tilefoundry/parser/ast_pattern.py |
| explicit_layout, layout, placed_layout, plain_layout | layout_shape, tensor_optional_slot, tensor_shape | LayoutShapeRule | A layout must have a valid non-boolean shape. | src/tilefoundry/parser/ast_pattern.py |
| function | function | FunctionDialectRule | A function kind and constructed value must agree with the active dialect. | src/tilefoundry/parser/pattern_nodes.py |
| function | function | FunctionRegistrationRule | A validated function must be registered exactly once in its owning scope. | src/tilefoundry/parser/pattern_nodes.py |
| function | function | FunctionReturnRule | A HIR function body's inferred type must match its return type. | src/tilefoundry/parser/pattern_nodes.py |
| function | function | FunctionRoleValidationRule | A root, variant, or converter must satisfy its role before registration. | src/tilefoundry/parser/pattern_nodes.py |
| function | function | FunctionSignatureRule | A function must construct an ordered parameter tuple. | src/tilefoundry/parser/pattern_nodes.py |
| index_slice | subscript_index | TileWindowSliceBoundRule | A tile window cannot be used as a slice bound. | src/tilefoundry/parser/pattern_nodes.py |
| layout | tensor_optional_slot | LayoutPositionRule | A layout must be legal for its parser position. | src/tilefoundry/parser/ast_pattern.py |
| layout | tensor_optional_slot | LayoutShapeRule | A layout must have a valid non-boolean shape. | src/tilefoundry/parser/ast_pattern.py |
| module | module_finalization | ModuleFinalizationRule | A module declaration must contain valid unique members and a resolvable entry. | src/tilefoundry/parser/ast_pattern.py |
| module | module_function | ModuleFunctionRegistrationRule | A validated module function must be recorded in declaration order. | src/tilefoundry/parser/ast_pattern.py |
| module | module_function | ModuleFunctionValidationRule | A module function must satisfy its root, variant, or converter role before mutation. | src/tilefoundry/parser/ast_pattern.py |
| op_call | expression, slice_endpoint, subscript_index | CallBindingRule | A call must bind its arguments into a Call tuple. | src/tilefoundry/parser/pattern_nodes.py |
| op_call | expression, slice_endpoint, subscript_index | CallExpectedTypeRule | A call's inferred type must satisfy the expected expression type. | src/tilefoundry/parser/pattern_nodes.py |
| op_call | expression, slice_endpoint, subscript_index | CallTypeInferenceRule | A call's result type must be inferred from its binding. | src/tilefoundry/parser/pattern_nodes.py |
| op_call | expression, slice_endpoint, subscript_index | CallVariadicInputFormRule | A variadic call must use one explicit list, tuple, or supported static list comprehension. | src/tilefoundry/parser/pattern_nodes.py |
| placed_layout | layout_shape, tensor_optional_slot, tensor_shape | LayoutPositionRule | A layout must be legal for its parser position. | src/tilefoundry/parser/ast_pattern.py |
| placed_layout | layout_shape, tensor_optional_slot, tensor_shape | LayoutShapeRule | A layout must have a valid non-boolean shape. | src/tilefoundry/parser/ast_pattern.py |
| plain_layout | tensor_optional_slot | LayoutPositionRule | A layout must be legal for its parser position. | src/tilefoundry/parser/ast_pattern.py |
| plain_layout | tensor_optional_slot | LayoutShapeRule | A layout must have a valid non-boolean shape. | src/tilefoundry/parser/ast_pattern.py |
| shape | layout_shape, layout_strides, tensor_shape | ShapeTupleRule | A shape must construct a tuple of dimensions. | src/tilefoundry/parser/ast_pattern.py |
| storage | tensor_optional_slot | StorageValueRule | Storage must resolve to a StorageKind. | src/tilefoundry/parser/ast_pattern.py |
| tensor | annotation, expression, slice_endpoint, subscript_index, type_annotation | TensorLayoutStorageRule | A tensor type must contain compatible layout and storage values. | src/tilefoundry/parser/ast_pattern.py |
| tensor | annotation, expression, slice_endpoint, subscript_index, type_annotation | TensorPositionRule | A tensor type's storage must be legal for its dialect and position. | src/tilefoundry/parser/ast_pattern.py |
| unary_expression | expression, slice_endpoint, subscript_index | CallBindingRule | A call must bind its arguments into a Call tuple. | src/tilefoundry/parser/pattern_nodes.py |
| unary_expression | expression, slice_endpoint, subscript_index | CallExpectedTypeRule | A call's inferred type must satisfy the expected expression type. | src/tilefoundry/parser/pattern_nodes.py |
| unary_expression | expression, slice_endpoint, subscript_index | CallTypeInferenceRule | A call's result type must be inferred from its binding. | src/tilefoundry/parser/pattern_nodes.py |
<!-- parser-constraints:end -->

## 3. Implementation Overview
Expand All @@ -218,6 +209,7 @@ function ::= 'def' name '(' signature ')' ('->' return-type)? ':' b
| Ordered Rules | Validates and normalizes each owner value after construction. |
| Module Build | Lets Python execute the class body, collects declarations, resolves child Modules first, then parses Functions in source order and finalizes the Module. |
| Pattern Visitor | Traverses the same graph to render this section's generated grammar and constraints. |
| Refusal | Carries the reason from the pattern that claimed a node and then refused it, so a report names a cause rather than the absence of a match. |

```mermaid
classDiagram
Expand All @@ -227,6 +219,11 @@ classDiagram
Element o-- AstPattern
Element o-- AstRule
AstPattern --> AstMatch
AstPattern --> MatchFailure
MatchFailure <|.. PatternFailure
MatchFailure <|.. ChoiceFailure
ChoiceFailure o-- MatchFailure : causes
ParseError <.. MatchFailure
PatternVisitor ..> AstPattern
ParserAPI ..> ModuleBuild
```
Expand All @@ -247,6 +244,45 @@ flowchart TD
MODULE -->|no| RETURN["return standalone result"]
```

```mermaid
flowchart TD
TRY["alternative.match(node)"] --> OUT{"outcome"}
OUT -->|"AstMatch"| WIN["choice accepts it; pending refusals are discarded"]
OUT -->|"MatchFailure"| CLAIM["claimed the node and refused: reason recorded"]
OUT -->|"None"| PASS["did not recognize the node: nothing recorded"]
PASS --> NEXT["try the next alternative"]
CLAIM --> NEXT
NEXT --> DONE{"any refusal recorded?"}
DONE -->|no| SILENT["return None: no alternative recognized this node"]
DONE -->|yes| COLLECT["ChoiceFailure over the claimants"]
COLLECT --> UP["travels up unchanged; combinators add no wrapping"]
UP --> RENDER["render(): a sole claimant is the whole report"]
RENDER --> RAISE["ParseError with source location"]
SILENT --> RAISE
```

A pattern MUST establish that a node is its own before it refuses with a reason. That claim is
what makes the reason trustworthy: it says no remaining alternative can accept this node, so
the refusal is the author's mistake and not another pattern's turn. A callee resolving to an op
schema is such a claim, and a wrong argument count after it is an error. A callee that does not
resolve is not a claim: it may be a bare name or a foreign namespace that another alternative
owns, so the pattern returns `None` and says nothing. Reasons are never reconstructed from the
AST after the fact; an inspection outside the refusing pattern cannot see which step it failed
at, and becomes a second, divergent copy of that knowledge.

Being last in one choice is not a claim either, because that choice may itself be an
alternative in another. `parse_node` is the single place where no alternative remains, so it
is the only place that MAY describe a node from its shape rather than from a pattern's
statement, and it does so only when the shape says something worth reading.

`None` and a `MatchFailure` differ only for a choice; every other combinator returns either one
unchanged, so a refusal keeps the identity and the wording of the pattern that produced it all
the way to `parse_node`. Nothing is wrapped, filtered, or re-described on the way up. A
`ChoiceFailure` records only the alternatives that claimed the node, which is normally one, and
it renders as that sole claimant. Two claimants mean two patterns claim overlapping shapes; the
report states both rather than choosing between them, because the ambiguity is in the grammar
and not in the report.

Pattern combinators serve both runtime matching and Spec traversal. `AstMatch` separates syntax
matching from object construction, while each Rule reads the recursive context after its owner
value exists. Module class control flow remains Python execution; no Module AST grammar exists.
Loading
Loading