Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RSDK-8541: ProtoValue get_unchecked and visit API #279

Merged
merged 74 commits into from
Sep 20, 2024
Merged
Show file tree
Hide file tree
Changes from 64 commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
cdbd2e0
update paths to cpp files
lia-viam Jul 23, 2024
764cf2f
Merge branch 'main' of github.com:viamrobotics/viam-cpp-sdk
lia-viam Jul 26, 2024
ce05487
introduce proof of concept type erasure proto type
lia-viam Jul 26, 2024
3a24f90
add documentation and tests
lia-viam Jul 26, 2024
f72ebc1
nolint recursion
lia-viam Jul 26, 2024
0759604
make ctor helper an IILE and move the nolint
lia-viam Jul 26, 2024
a01ed14
double nolint
lia-viam Jul 26, 2024
7a41fa9
add template keyword for pmf
lia-viam Jul 29, 2024
b834098
add helpers to simplify test case invocation
lia-viam Jul 29, 2024
981becb
introduce coexisting pointer based converters
lia-viam Jul 31, 2024
4ae83d9
unfriend to_proto_value and remove model value version
lia-viam Jul 31, 2024
f391a30
ABI insulate proto value
lia-viam Jul 31, 2024
79a4c4d
add and test is_a and use static_cast instead of reinterpret_cast
lia-viam Jul 31, 2024
afcdae7
test copy construction and casting
lia-viam Jul 31, 2024
ada9e37
add holder class and test more copy and move and null
lia-viam Aug 1, 2024
19159fe
add missing namespace
lia-viam Aug 1, 2024
07b82ae
Merge branch 'main' of github.com:viamrobotics/viam-cpp-sdk
lia-viam Aug 1, 2024
41bd481
Merge branch 'main' into feature/type-erase-proto
lia-viam Aug 1, 2024
4c8e592
remove unneeded forward decl
lia-viam Aug 1, 2024
9705aa2
update comments
lia-viam Aug 1, 2024
0e009aa
use local stack storage for all types and test new move semantics
lia-viam Aug 2, 2024
8ab5d96
move small_size and remove unneeded default
lia-viam Aug 5, 2024
184f11f
Merge branch 'main' into feature/type-erase-proto
lia-viam Aug 5, 2024
a2b751a
add struct_to_map and map_to_struct
lia-viam Aug 6, 2024
4e3d985
use non ptr version
lia-viam Aug 6, 2024
5b31f86
construct with move or rvalue
lia-viam Aug 7, 2024
2483377
move list value
lia-viam Aug 7, 2024
26d7cc7
rename to vtable_t and remove noexcept in function pointers
lia-viam Aug 7, 2024
03bc715
s/model_t/model/g
lia-viam Aug 7, 2024
b3e2f09
s/storage_t/storage/g
lia-viam Aug 7, 2024
1f45217
remove _value in to_proto/from_proto
lia-viam Aug 7, 2024
c600450
move almost all function defs to source file
lia-viam Aug 7, 2024
dfa7575
update comments, add static_assert, and use aligned_storage_t
lia-viam Aug 7, 2024
1f07ecc
revert kind_t change
lia-viam Aug 7, 2024
a333406
update kind_t comment
lia-viam Aug 7, 2024
47bdd17
document type trait
lia-viam Aug 8, 2024
b0b0e15
rename ProtoT to ProtoValue
lia-viam Aug 8, 2024
875d968
use impl type trait for noexcept
lia-viam Aug 8, 2024
4b01a12
rename attrmap to protostruct
lia-viam Aug 8, 2024
ed70190
add doxygen comments
lia-viam Aug 8, 2024
2a8cb3a
use aligned_union_t for local storage and change static_asserts
lia-viam Aug 9, 2024
51e2b9a
revert impl-ing kind_t and add doc comment
lia-viam Aug 9, 2024
7ca50e6
rename files to proto_value
lia-viam Aug 9, 2024
83de916
silence recursion linting
lia-viam Aug 9, 2024
1516e81
add another nolint
lia-viam Aug 9, 2024
b659c76
rename member var
lia-viam Aug 9, 2024
c549e8d
Merge branch 'main' of github.com:viamrobotics/viam-cpp-sdk into feat…
lia-viam Aug 13, 2024
d5dc9cb
fix typo in move_may_throw
lia-viam Aug 13, 2024
fb48286
rename impl namespace to detail
lia-viam Aug 13, 2024
50338ab
rename dyn_cast to member get
lia-viam Aug 13, 2024
955eae4
rename namespace to fit pattern
lia-viam Aug 13, 2024
cbcdc4e
add is_null
lia-viam Aug 13, 2024
fcf88ac
static_assert nothrow dtor
lia-viam Aug 13, 2024
896480b
static_assert size and alignment match
lia-viam Aug 13, 2024
0536359
only compile proto_types.cpp into tests
lia-viam Aug 13, 2024
c3b27d1
add and test unchecked access
lia-viam Aug 13, 2024
f10ec1f
Merge branch 'main' of github.com:viamrobotics/viam-cpp-sdk into feat…
lia-viam Aug 13, 2024
e94a94b
move proto_value tests into own file
lia-viam Aug 14, 2024
95be658
make kind_t sfinae-able
lia-viam Aug 14, 2024
5bba955
remove unused include
lia-viam Aug 14, 2024
e68a3bd
add and test proto value visit
lia-viam Aug 14, 2024
cb35f2d
document visit api
lia-viam Aug 14, 2024
57553a7
change include order
lia-viam Aug 14, 2024
df74f74
Merge branch 'main' of github.com:viamrobotics/viam-cpp-sdk into feat…
lia-viam Aug 15, 2024
4b88dcf
code review: change include order
lia-viam Sep 16, 2024
682a270
code review: change include order
lia-viam Sep 16, 2024
1ab6ae5
code review: change include order
lia-viam Sep 16, 2024
4e12704
put extern template in header
lia-viam Sep 16, 2024
c6fd7b2
Merge branch 'main' of github.com:viamrobotics/viam-cpp-sdk into feat…
lia-viam Sep 16, 2024
47b70f9
Merge branch 'main' into feature/proto-value-visit
lia-viam Sep 19, 2024
f55d8a7
replace integer types with enums and no longer expose kind_t
lia-viam Sep 20, 2024
3c330eb
remove switch defaults
lia-viam Sep 20, 2024
31116da
alias protolist
lia-viam Sep 20, 2024
f5e94cf
use k_ for enum names and add explicit numerical values
lia-viam Sep 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions src/viam/sdk/common/proto_value.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,56 @@ bool ProtoValue::is_null() const {
return kind() == kind_t<std::nullptr_t>{};
}

template <typename T>
std::enable_if_t<std::is_scalar<T>{}, T&> ProtoValue::get_unchecked() {
assert(this->is_a<T>());
return *(this->self_.template get<T>());
}

template <typename T>
std::enable_if_t<std::is_scalar<T>{}, T> ProtoValue::get_unchecked() const {
assert(this->is_a<T>());
return *(this->self_.template get<T>());
}

template bool& ProtoValue::get_unchecked<bool>();
template int& ProtoValue::get_unchecked<int>();
template double& ProtoValue::get_unchecked<double>();

template bool ProtoValue::get_unchecked<bool>() const;
template int ProtoValue::get_unchecked<int>() const;
template double ProtoValue::get_unchecked<double>() const;
acmorrow marked this conversation as resolved.
Show resolved Hide resolved

template <typename T>
std::enable_if_t<!std::is_scalar<T>{}, T&> ProtoValue::get_unchecked() & {
assert(this->is_a<T>());
return *(this->self_.template get<T>());
}

template <typename T>
std::enable_if_t<!std::is_scalar<T>{}, T const&> ProtoValue::get_unchecked() const& {
assert(this->is_a<T>());
return *(this->self_.template get<T>());
}

template <typename T>
std::enable_if_t<!std::is_scalar<T>{}, T&&> ProtoValue::get_unchecked() && {
assert(this->is_a<T>());
return std::move(*(this->self_.template get<T>()));
}

template std::string& ProtoValue::get_unchecked<std::string>() &;
template std::vector<ProtoValue>& ProtoValue::get_unchecked<std::vector<ProtoValue>>() &;
template ProtoStruct& ProtoValue::get_unchecked<ProtoStruct>() &;

template std::string const& ProtoValue::get_unchecked<std::string>() const&;
template std::vector<ProtoValue> const& ProtoValue::get_unchecked<std::vector<ProtoValue>>() const&;
template ProtoStruct const& ProtoValue::get_unchecked<ProtoStruct>() const&;

template std::string&& ProtoValue::get_unchecked<std::string>() &&;
template std::vector<ProtoValue>&& ProtoValue::get_unchecked<std::vector<ProtoValue>>() &&;
template ProtoStruct&& ProtoValue::get_unchecked<ProtoStruct>() &&;
acmorrow marked this conversation as resolved.
Show resolved Hide resolved

// --- ProtoT::model<T> definitions --- //
template <typename T>
ProtoValue::model<T>::model(T t) noexcept(std::is_nothrow_move_constructible<T>{})
Expand Down
79 changes: 63 additions & 16 deletions src/viam/sdk/common/proto_value.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -110,16 +110,40 @@ class ProtoValue {
template <typename T>
T const* get() const;

/// @brief Return a reference to the underlying T, without checking.
/// @tparam T a bool, int, or double
template <typename T>
std::enable_if_t<std::is_scalar<T>{}, T&> get_unchecked();

/// @brief Return the underlying T by value, without checking.
/// @tparam T a bool, int, or double.
template <typename T>
std::enable_if_t<std::is_scalar<T>{}, T> get_unchecked() const;

/// @brief Return a mutable reference to the underlying T, without checking
/// @tparam T a std::string, std::vector<ProtoValue>, or ProtoStruct.
template <typename T>
std::enable_if_t<!std::is_scalar<T>{}, T&> get_unchecked() &;

/// @brief Return an immutable reference to the underlying T, without checking.
/// @tparam T a std::string, std::vector<ProtoValue>, or ProtoStruct.
template <typename T>
std::enable_if_t<!std::is_scalar<T>{}, T const&> get_unchecked() const&;

/// @brief Return an rvalue reference to the underlying T, without checking.
/// @tparam T a std::string, std::vector<ProtoValue>, or ProtoStruct.
template <typename T>
std::enable_if_t<!std::is_scalar<T>{}, T&&> get_unchecked() &&;

///@}

private:
// This struct is our implementation of a virtual table, similar to what is created by the
// compiler for polymorphic types. We can't use actual polymorphic types because the
// implementation uses aligned stack storage, so we DIY it insted.
// The vtable can be thought of as defining a concept or interface which our ProtoValue types
// must satisfy.
// The first void [const]* parameter in any of the pointers below is always the `this` or `self`
// pointer.
// The vtable can be thought of as defining a concept or interface which our ProtoValue
// types must satisfy. The first void [const]* parameter in any of the pointers below is
// always the `this` or `self` pointer.
struct vtable {
void (*dtor)(void*);
void (*copy)(void const*, void*);
Expand Down Expand Up @@ -273,33 +297,56 @@ Struct map_to_struct(const ProtoStruct& m) {
return s;
}

/// @brief ProtoValue RTTI type trait.
/// This type trait is used to implement the ProtoValue::kind method which provides the type
/// discriminator constant that is used in the value access API.
/// A ProtoValue can only be constructed from types for which this trait is well formed.
namespace proto_value_details {

template <typename T>
struct kind_t;
struct kind {};

template <>
struct kind_t<std::nullptr_t> : std::integral_constant<int, 0> {};
struct kind<std::nullptr_t> {
using type = std::integral_constant<int, 0>;
};

template <>
struct kind_t<bool> : std::integral_constant<int, 1> {};
struct kind<bool> {
using type = std::integral_constant<int, 1>;
};

template <>
struct kind_t<int> : std::integral_constant<int, 2> {};
struct kind<int> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just occurred to me that proto's struct doesn't actually have an integer type. Just double. Do we want to be in the business of defining that conversion, or should we push that to users, since we cannot always know their intent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been wondering about this too--proto just has a number type which promotes everything to double. There are some gotchas in the unit tests about this:
https://github.com/lia-viam/viam-cpp-sdk/blob/feature/proto-value-visit/src/viam/sdk/tests/test_proto_value.cpp#L43

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having mulled it over, I think it should not have an int case, or conversion functions from int. It gives the user the illusion that int is supported, but it really isn't, since it becomes a double. If anything, I'm almost tempted to say that we should =delete the ability to construct ProtoValue from integers, so that the caller must deal with the conversion themselves.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm in favor of getting rid of the separate int and double types and replacing them with a number type which is just a double underneath, as this would match what's in proto
https://protobuf.dev/reference/protobuf/google.protobuf/#value

Deleting the conversion from int entirely is tempting, but maybe this will cause us headaches when migrating from ProtoType? The variant underlying that class also has distinct int and double, which is arguably a mistake for the same reason, but there will definitely be instances of std::make_shared<ProtoType>(5)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, so several thoughts have come from this discussion:

  • Does ProtoValue really need a template constructor, or should it just have constructors for each type. As written, you will just get a funky probably-link-time error because it can't instantiate it if you try to, say, construct a ProtoValue of some unrelated type.

  • Should (some of the) ProtoValue constructors (templated or not) be explicit?

  • I suspect that if we eliminate the ability to hold integers, then the one numeric constructor should take double, but allow implicit conversions (so, not explicit). That'd let std::make_shared<ProtoType>(5) still work, I think. What I want to avoid is confusing situations where by accepting int as a constructor argument, we allow things with conversion to int to be passed, but then those things become doubles internally.

Copy link
Contributor Author

@lia-viam lia-viam Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good questions. My thoughts

  • I may as well just spell out a bunch of non-template constructors for each type, although I think I would still want to have them defer to a private template constructor. Whatever amount of duplication is present it would still be fewer lines than all the explicit instantiations I'm writing
  • I think yeah ideally we would want to follow the example of JSON libraries, where you have sort of easy ergonomic interfaces for building up objects like this, which you get because the constructors are not explicit.
  • Yeah I think this makes sense and we will just be deferring to standard C++ language rules about type promotion/argument overloading

All that said, maybe I should make a separate PR to do

  • have only number type
  • define non-template public constructors
  • make kind an enum and don't expose kind_t

and then rebase this one off that one?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, let's go forward with what you have now where int is a thing, and then defer the whole int removal and any associated constructor fussing into a followup PR. I don't think you need to do the rebase part.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK cool so just to confirm should I still do the kind_t removal and changing to an enum Kind for this PR too?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think the kind_t removal is pretty trivial, and I'd really like to see the default case go away if possible, so I think the enum thing makes more sense in this review than in a new one.

using type = std::integral_constant<int, 2>;
};

template <>
struct kind_t<double> : std::integral_constant<int, 3> {};
struct kind<double> {
using type = std::integral_constant<int, 3>;
};

template <>
struct kind_t<std::string> : std::integral_constant<int, 4> {};
struct kind<std::string> {
using type = std::integral_constant<int, 4>;
};

template <>
struct kind_t<std::vector<ProtoValue>> : std::integral_constant<int, 5> {};
struct kind<std::vector<ProtoValue>> {
using type = std::integral_constant<int, 5>;
};

template <>
struct kind_t<ProtoStruct> : std::integral_constant<int, 6> {};
struct kind<ProtoStruct> {
using type = std::integral_constant<int, 6>;
};

} // namespace proto_value_details

/// @brief ProtoValue RTTI type trait.
/// This type trait is used to implement the ProtoValue::kind method which provides the type
/// discriminator constant that is used in the value access API.
/// @remark A ProtoValue can only be constructed from types for which this trait is well formed.
/// This type trait can be used to induce substitution failure on non-ProtoValue constructible
/// types.
template <typename T>
using kind_t = typename proto_value_details::kind<T>::type;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need the _t?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose not, more just convention

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually just tried doing this change, this made the compiler complain because of the ambiguity with the kind() method, I think we might be better off keeping the _t, or possibly using capital Kind, actually not super clear what our convention is here as I think it varies a bit throughout the SDK

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, do you even need kind_t? Is there a reason not to just say proto_value_details::kind<T>::type where you otherwise would have said kind_t? I don't see that kind_t figures in the public API of ProtoValue anywhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point!


template <typename T>
bool ProtoValue::is_a() const {
Expand Down
95 changes: 95 additions & 0 deletions src/viam/sdk/common/proto_value_visit.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
#pragma once

#include <viam/sdk/common/proto_value.hpp>

#include <type_traits>
acmorrow marked this conversation as resolved.
Show resolved Hide resolved

namespace viam {
namespace sdk {

// It might seem that we could write these all with a single template, but this ends up requiring
// some annoying gymnastics with type traits that only saves around 15 lines. For more info see
// https://github.com/boostorg/json/issues/952 and the linked PR with the fix.

/// @defgroup ProtoValueVisit ProtoValue visit API
/// Invoke a function object with the contents of a ProtoValue. The function object must be callable
/// with all the possible types of ProtoValue. visit will inspect the ProtoValue to determine its
/// stored type, and then call the visitor on it with value category matching that with which the
/// ProtoValue was passed.
/// @{

template <typename Visitor>
auto visit(Visitor&& visitor, ProtoValue& value)
-> decltype(std::forward<Visitor>(visitor)(std::declval<std::nullptr_t&>())) {
switch (value.kind()) {
case kind_t<bool>{}:
return std::forward<Visitor>(visitor)(value.get_unchecked<bool>());
case kind_t<int>{}:
return std::forward<Visitor>(visitor)(value.get_unchecked<int>());
case kind_t<double>{}:
return std::forward<Visitor>(visitor)(value.get_unchecked<double>());
case kind_t<std::string>{}:
return std::forward<Visitor>(visitor)(value.get_unchecked<std::string>());
case kind_t<std::vector<ProtoValue>>{}:
return std::forward<Visitor>(visitor)(value.get_unchecked<std::vector<ProtoValue>>());
case kind_t<ProtoStruct>{}:
return std::forward<Visitor>(visitor)(value.get_unchecked<ProtoStruct>());
default:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is a default needed at all? Should it result in calling the nullptr case for visitor? I feel like it shouldn't be possible or should be treated as some sort of fatal error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's "needed" just to not make the compiler complain, but in theory it's not possible by the invariants we maintain on the proto value objects. We could do a BOOST_UNREACHABLE_RETURN or similar.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would having kind return an enum over the available types placate the compiler about not having a default?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that would work actually, I'll give it a shot

case kind_t<std::nullptr_t>{}: {
auto np = nullptr;
return std::forward<Visitor>(visitor)(np);
}
}
}

template <typename Visitor>
auto visit(Visitor&& visitor, const ProtoValue& value)
-> decltype(std::forward<Visitor>(visitor)(std::declval<const std::nullptr_t&>())) {
switch (value.kind()) {
case kind_t<bool>{}:
return std::forward<Visitor>(visitor)(value.get_unchecked<bool>());
case kind_t<int>{}:
return std::forward<Visitor>(visitor)(value.get_unchecked<int>());
case kind_t<double>{}:
return std::forward<Visitor>(visitor)(value.get_unchecked<double>());
case kind_t<std::string>{}:
return std::forward<Visitor>(visitor)(value.get_unchecked<std::string>());
case kind_t<std::vector<ProtoValue>>{}:
return std::forward<Visitor>(visitor)(value.get_unchecked<std::vector<ProtoValue>>());
case kind_t<ProtoStruct>{}:
return std::forward<Visitor>(visitor)(value.get_unchecked<ProtoStruct>());
default:
case kind_t<std::nullptr_t>{}: {
const auto np = nullptr;
return std::forward<Visitor>(visitor)(np);
}
}
}

template <typename Visitor>
auto visit(Visitor&& visitor, ProtoValue&& value)
-> decltype(std::forward<Visitor>(visitor)(std::declval<std::nullptr_t&&>())) {
switch (value.kind()) {
case kind_t<bool>{}:
return std::forward<Visitor>(visitor)(std::move(value.get_unchecked<bool>()));
case kind_t<int>{}:
return std::forward<Visitor>(visitor)(std::move(value.get_unchecked<int>()));
case kind_t<double>{}:
return std::forward<Visitor>(visitor)(std::move(value.get_unchecked<double>()));
case kind_t<std::string>{}:
return std::forward<Visitor>(visitor)(std::move(value.get_unchecked<std::string>()));
case kind_t<std::vector<ProtoValue>>{}:
return std::forward<Visitor>(visitor)(
std::move(value.get_unchecked<std::vector<ProtoValue>>()));
case kind_t<ProtoStruct>{}:
return std::forward<Visitor>(visitor)(std::move(value.get_unchecked<ProtoStruct>()));
default:
case kind_t<std::nullptr_t>{}:
return std::forward<Visitor>(visitor)(std::nullptr_t());
}
}

/// @}

} // namespace sdk
} // namespace viam
2 changes: 2 additions & 0 deletions src/viam/sdk/tests/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ viamcppsdk_add_boost_test(test_motion.cpp)
viamcppsdk_add_boost_test(test_movement_sensor.cpp)
viamcppsdk_add_boost_test(test_pose_tracker.cpp)
viamcppsdk_add_boost_test(test_power_sensor.cpp)
viamcppsdk_add_boost_test(test_proto_value.cpp)
viamcppsdk_add_boost_test(test_proto_value_visit.cpp)
viamcppsdk_add_boost_test(test_resource.cpp)
viamcppsdk_add_boost_test(test_sensor.cpp)
viamcppsdk_add_boost_test(test_servo.cpp)
Expand Down
Loading