Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for keyword-only arguments #448

Merged
merged 1 commit into from
Mar 6, 2024
Merged

Conversation

oremanj
Copy link
Contributor

@oremanj oremanj commented Mar 5, 2024

I know this was in the "not going to support it" list, but I figured out how to do it with extremely minimal overhead, so I'm politely asking you to reconsider that position. :-) All ordering/etc checks happen at compile-time, the size of func_data doesn't change, and nb_func.o gets very slightly smaller on my laptop. The 'simple' function dispatcher is completely unaffected and the changes to the 'complex' one are very minimal.

@oremanj
Copy link
Contributor Author

oremanj commented Mar 5, 2024

I'm pretty sure the pypy failure is not my fault, as I see the same failure in #442.

Copy link
Owner

@wjakob wjakob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct that this is a feature that isn't banned in any way. I just thought that the overload dispatch loop in pybind11 was way overkill. Here is some first feedback.

This actually looks really good -- remarkably simple as you say. I do have a few comments and questions though.

docs/porting.rst Outdated Show resolved Hide resolved
@@ -54,22 +60,61 @@ NB_INLINE PyObject *func_create(Func &&func, Return (*)(Args...),
nargs = sizeof...(Args);

// Determine the number of nb::arg/nb::arg_v annotations
constexpr size_t nargs_provided =
static constexpr size_t nargs_provided =
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am curious -- is the addition of static for these constexpr parameters just general best practice, or do you think it has an actual impact here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is mostly a habit I have because of past experience with compilers thinking constexpr non-statics need to be captured in lambdas. If I remove the lambda then these definitely aren't necessary. I was confused why some of them were previously static and some not, though.

include/nanobind/nb_func.h Show resolved Hide resolved

// Avoid instantiating count_args_before_index unless the kw_only
// annotation is actually used
static constexpr size_t nargs_before_kw_only = []() constexpr {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a fan of this one, it seems kind of complicated. I am suspicious of what a compiler like MSVC will make out of this (I've seen it struggle with things as basic as inlining a 1-line lambda function, and it is generally bad at constexpr computation).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switched to a version that doesn't use a lambda.

/// for each of these.
uint16_t nargs;

/// Number of arguments to the C++ function that may be filled from
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to clarify: this includes nb::arg() (unnamed) and nb::arg("name")` (named) arguments?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct, as long as they appear before the kw_only annotation or variadic *args. I expanded on the comment a bit.

src/nb_func.cpp Show resolved Hide resolved
include/nanobind/nb_func.h Show resolved Hide resolved
src/nb_func.cpp Show resolved Hide resolved
src/nb_func.cpp Show resolved Hide resolved
@oremanj
Copy link
Contributor Author

oremanj commented Mar 5, 2024

Thank you for the prompt review! I made some updates in response to your comments, mostly better comments with a couple functional fixes (detecting double kw_only annotations, handling kw_only before **kwargs with no *args). Please take another look when you have a moment.

Copy link
Owner

@wjakob wjakob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect! Only a few minor comments at this point.

include/nanobind/nb_attr.h Show resolved Hide resolved
include/nanobind/nb_func.h Show resolved Hide resolved
// by 1 for methods. Note that nargs_before_kw_only is only used if
// a kw_only annotation exists (i.e., if explicit_kw_only is true);
// the conditional is just to save the compiler some effort otherwise.
constexpr size_t nargs_before_kw_only =
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stupid question: what's the difference between keeping this code here as constexpr size_t or just making a size_t variable and then potentially overwriting it in the constexpr block? If efficiency (avoiding the template expansion) is the reason, then it would seem to me that the compiler must in any case be smart enough to not expand templates in a if constexpr (false) block to avoid potential exponential instantiation of templates in recursive constexpr programs. Anyways, this way of writing things down seems somehow more natural (closer to the behavior of a regular non-constexpr program).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need the nargs_before_kw_only value available as a constexpr in order to static_assert about it in the explicit_kw_only block, and we also need it down at the bottom of the function to initialize nargs_pos. The latter doesn't need to be constexpr, so I guess we could do something like

size_t nargs_before_kw_only = nargs;
if constexpr (explicit_kw_only) {
    constexpr size_t cx_nargs_before_kw_only = is_method_det + count_args_before_index<...>(...);
    // static_assert about cx_nargs_before_kw_only
    nargs_before_kw_only = cx_nargs_before_kw_only;
}

but that seemed like way too much trouble to go to when there's not any reason to believe the instantiation of count_args_before_index is particularly expensive. I was only trying to avoid it on principle, but the cure seemed worse than the disease in the end.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Let's leave it like this then.

include/nanobind/nb_func.h Show resolved Hide resolved
src/nb_func.cpp Show resolved Hide resolved
@wjakob
Copy link
Owner

wjakob commented Mar 6, 2024

I think this ready to go in. Could you add a changelog entry and then squash the commits?

@oremanj
Copy link
Contributor Author

oremanj commented Mar 6, 2024

Should be all set!

@wjakob
Copy link
Owner

wjakob commented Mar 6, 2024

Thank you!

@wjakob wjakob merged commit bf46193 into wjakob:master Mar 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants