documentation for Enzyme Type Trees #2385
Conversation
|
|
||
| ## What are Type Trees? | ||
|
|
||
| Type trees in Enzyme are a way to represent the types of variables, including their activity (e.g., whether they are active, duplicated, or contain duplicated data) for automatic differentiation. They provide a structured way for Enzyme to understand how to handle different data types during the differentiation process. |
There was a problem hiding this comment.
Are there any docs or code which shows them including activity?
There was a problem hiding this comment.
my bad i saw this and got confused they both are in the same, later when i was going through enzyme codebase i realised they are different, fixing this
|
|
||
| Enzyme needs to understand the structure and properties of Rust types to perform automatic differentiation correctly. This is where type trees come in. They provide a detailed map of a type, including pointer indirections and the underlying concrete data types. | ||
|
|
||
| The `-enzyme-rust-type` flag in Enzyme helps in interpreting types more accurately in the context of Rust's memory layout and type system. |
There was a problem hiding this comment.
The flag just tells enzyme to parse (rust) debug "dwarf" information.
A lot of type information is not encoded in such debug metadata, and the flag hasn't been re-evaluated or used in years. It's good to mention it here (with the corrected description), but I wouldn't mention it in the following sections. We should generate typetrees based on Rust types even without debug metadata. But how to use debug metadata is something we'll also discuss in one of the meetings with oli.
There was a problem hiding this comment.
got this, the flag was there in most of test files, which were related to rust
|
|
||
| Consider a Rust reference to a 32-bit floating-point number, `&f32`. | ||
|
|
||
| In LLVM IR, this might be represented, for instance, as an `i8*` (a generic byte pointer) that is then `bitcast` to a `float*`. Consider the following LLVM IR function: |
There was a problem hiding this comment.
Ah, you took that from rustmutpointer.ll I guess? Unfortunately they are too outdated, I just realized.
Typed ptr were removed (see my comment about this flag being very outdated), so i8* isn't a thing anymore.
Instead, we now have ptr ("opaque pointers") in LLVM.
You can look for the PRs which introduced these tests a few years ago. They should have instructions on how to reproduce them, so you can re-generate newer tests once you have a working setup.
There was a problem hiding this comment.
yes,
found the PR, this will be a great reference and understanding of how things were done
|
|
||
| * **`{ ... }`**: This encloses the set of type information for the variable. | ||
| * **`[-1]:Pointer`**: | ||
| * `[-1]` is an index or path. In this context, `-1` often refers to the base memory location or the immediate value pointed to. |
There was a problem hiding this comment.
-1 has a slightly different meaning, something along the lines of everything accessible from here, without dereferencing a pointer.
So [f64;32] could be represented as [-1]:Float@double, or as [0]:Float@double, [8]:Float@double, ...
Afaik we usually prefer -1 in such cases since it's shorter, but IIRC there were some gotchas.
@wsmoses can you share the private youtube video with him (if you prefer in a zulip dm), such that he has more information on how to write these docs?
There was a problem hiding this comment.
Or e.g. x: *const [f64;32] could be represented as
[[-1]:Pointer, [-1:-1]Float@double], or as [[0]:Pointer, [0:0]:Float@double, [0:8]:Float@double, ...
or as [[0]:Pointer, [0:-1]:Float@double or as [[-1]:Pointer, [-1:0]:Float@double, [-1:8]:Float@double, ...
(I think)
That also all is under the assumption that we won't access e.g. x[34] which is out of bounds of the original array.
I am not 100% sure if there are cases where this could be valid, I think with raw pointers it might be valid to access other elements. E.g. struct { x: [f64;32], y: i32 }. I you derive a raw pointer to x, you might be able to use it to access y (legally), in which case -1:Float@double would be wrong. I'd need a refresher on pointer provenance and other things, hopefully Oli will know more about it. You can add this as an open question at the end.
jieyouxu
added
S-waiting-on-author
No description provided.