[ROCm] Flydsl decode backend

[avbokovoy]

As per request from #463 (review)

[meta-codesync]

@q10 has imported this pull request. If you are a Meta employee, you can view this in D116064789.

[bottler]

Seems odd that this code is in mslk/attention/fmha/flydsl but the recent backward flydsl code #467 is in mslk/attention/flydsl.

[cthi]

Yeah I agree with @bottler , can AMD align on a common directory structure for attn? e.g. where kernels should go, regardless of how we integrate it into fmha. cc @liligwu

[avbokovoy]

@bottler @cthi Rebased and restructured the branch to align with current Flash/CUTLASS backends (Triton uses different file structure). Could you verify if it complies with repo requirements?

[bottler]

This makes mslk/attention/flydsl contain two independent groups of files implementing different backends, should they be split into subdirectories? I think there was a tiny bit of shared code in the previous version of this PR, but there's none now.

[avbokovoy]

This makes mslk/attention/flydsl contain two independent groups of files implementing different backends, should they be split into subdirectories? I think there was a tiny bit of shared code in the previous version of this PR, but there's none now.

@bottler I don't think that we should take into consideration shared code while making the decision on subdirectories split. If you would ask me, I lean towards more radical repo-wide restructuring, however I just want to stick with the solution that will be clear and reasonable for this repo and doesn't break anything. So, we definitely can split the algorithms into subdirectories and I think it's reasonable, however I believe it should be a follow-up PR after this one is merged. Wdyt?

[bottler]

Let's do something about folder structure in this PR. It just seems unfriendly to have two unrelated groups of files in the same directory (unless their names were grouped I guess). Easiest change would be to make this PR put its file in a subdir called e.g. decode. Then there's no clash with #485.

[avbokovoy]

Let's do something about folder structure in this PR. It just seems unfriendly to have two unrelated groups of files in the same directory (unless their names were grouped I guess). Easiest change would be to make this PR put its file in a subdir called e.g. decode. Then there's no clash with #485.

Done restructuring in ae94844