Support __cuda_array_interface__ types without len()

[pentschev]

Support __cuda_array_interface__ types that do not implement the __len__ method. The distributed.utils.nbytes function that we currently use relies on determining the size of object using either the __len__ methods implemented by the type or the nbytes attribute, which are not required by types that implement __cuda_array_interface__, such as pylibcudf.gpumemoryview. This change attempts to use the existing distributed.utils.nbytes function, and if that does not succeed then proceeds to calculate the size based on the attributes of __cuda_array_interface__. Note that strides are not currently implemented.

[rjzamora]

Thanks @pentschev !

I'm eager to get this in, so I don't want to make you jump through any unnecessary hoops. With that said, I have one suggestion: Can we add a comment explaining why we are adding this patch? Ideally, the comment would explain what a "long-term" fix would look like (do we know what that looks like?)

[pentschev]

Ideally, the comment would explain what a "long-term" fix would look like (do we know what that looks like?)

Actually, maybe we'll be better off implementing __len__/nbytes for gpumemoryview. Let me give that a look tomorrow to evaluate how much work that would be, in trying to move fast I forgot to look at that.

[pentschev]

Superseded by rapidsai/cudf#18133 , closing.