Convolution Data Layout NHWC vs. NCHW

[lgrothh]

Moin,

the tiled_conv_auto function from gemmini.h expects the inputs to be in NHWC and the weights to be in KhKwIO format, but the standard layout in PyTorch is NCHW for input/output tensors and OIKhKw for the weights.
Right now for testing I manually permute my input data to match the format expected by Gemmini which works, but it causes significant overhead so I was wondering how difficult it would be to make this a variable parameter?

I tried modifying the following lines in gemmini.h:

const elem_t * weights_slice = weights + (poch*kernel_dim*in_channels + kch*in_channels + krow_) * out_channels + kcol_;
[...]
const elem_t * in = input + (b*in_dim*in_dim + kch*in_channels  + ((irow+upad)>>input_dilated)) * in_dim  + ((icol+lpad)>>input_dilated);

But it does not work, I would appreciate any input on this
Best Regards

[hngenc]

The main problem is that switching from NHWC to NCHW changes the innermost dimension. Changing outer dimensions would just involve setting new strides in LoopConv.scala and tiled_conv (as you began doing), but changing the innermost dimension requires more intricate changes to how Gemmini expects the data to be laid out in its scratchpad.