Tracking issue: Incorrect Meta Strides / Turn On PyDispatcher in FakeTensor Mode

[eellison]

🐛 Describe the bug

Incorrect Strides can manifest in errors within torch.compile. Potentially what makes them trickier is that they only sometimes cause errors. An incorrect stride can lay dormant for a while and then cause a problem.

See, this discussion with @ezyang, @bdhirsh and myself about incorrect strides.

There are a number of known issues that yet unfixed. Some of them have outstanding prs, please check with the pr author before taking it on.

• full_like: inductor full_like decompositions give incorrect strides #144699
• _unsafe_index : incorrect _unsafe_index meta #139312
• _fft_r2c: [export] _fft_r2c does not support dynamic shapes #135087
• _constant_pad_nd: Wrong meta function for constant_pad_nd #144187

Additionally, there are a number of stride & other issues that have been exposed by enabling PyDispatcher in FakeTensorMode. This causes us to potentially route through different decompositions and metas. It is what we use in torch.compile, which means we lack coverage of this mode in our other tests.

Tests exposed by this turning this on:

• dropout
• MultiLabelMarginLoss

Fft tests as well, but that might be related to _fft_r2c in the existing issue.

Versions

master

cc @chauhang @penguinwu @SherlockNoMad @zou3519 @bdhirsh @yf225

[vadimkantorov]

Regarding torch.compile + _fft_r2c there is also an issue in:

• Avoid torch.jit.isinstance to support torch.compile(MFCC(), fullgraph=True) audio#3876