Lookahead in GEMM is not good

[QingleiCao]

The default lookahead in GEMM is 2, which is too small if running on multiple nodes, especially for large scaling. It should be related to the process grid.

[bosilca]

So what is a good lookahead ? If we had a table of good/better lookahead per problem-size we could try to fit it into a formula that covers most cases.

[QingleiCao]

I implemented a SUMMA SYRK, where A (input) in INT8 and C (output) in FP32. It seems lookahead = P is not bad (a formula with some theory support? :)). This is what I tested on 256 Summit nodes.

[QingleiCao]

(2 processes per node)

[QingleiCao]

[bosilca]

Looks good, let's make it P then. I guess the only thing one need to do is replace the LOOK_AHEAD [ type = int ] line with LOOK_AHEAD [ type = int default = tP]

[QingleiCao]

Looks good, let's make it P then. I guess the only thing one need to do is replace the LOOK_AHEAD [ type = int ] line with LOOK_AHEAD [ type = int default = tP]

Yes. Do I need to make a PR for that?