Arrow select-K does not support null-odering #800
Description
Draft of #799 added support for select-k algorithm instead of fully sorting the result, when orderByFields and limit are supplied simultaneously. When adding tests, I unfortunately discovered that the select-k implementation by arrow (more precisely arrow acero) does not support null values, which are dropped instead. Therefore we cannot use the select-k implementation for now.
When resolving this issue, we will supply an alternate solution to the problem, which also does not require sorting the full set.
The following (stale) PR in arrow also resolved the issue. We will solve it similarly by providing a custom select-k node.