Request for clarification on camera data and camera-based collaboration baseline

[openmacp3]

Hi,

When projecting the dataset's 3D annotations into the corresponding 2D camera images using the parameters provided in the YAML files, I have encountered problems in achieving the correct bbox alignment.

Could you please provide guidance, documentation, or ideally a code snippet showing the intended workflow for coordinate transformations and projection of the camera data?
Specifically, how to correctly use camera parameters like camera cords, intrinsic, and the object location, extent, and angle to map 3D world boxes to 2D image coordinates.

Additionally, any details regarding the camera-based collaboration baselines would be very helpful.

Thanks,

[Merealtea]

I met same problems too, I suggest authors could put forward a official tutorial for visualization and demostration of coordination of different sensors.

[zhz03]

Hi there, thank you for using our dataset! I think the issue that you're facing is because of the transformation matrix. Please check out our dataset tool repo for getting the correct transformation: https://github.com/zhz03/dataset_tool

The problem that you are encountering is mainly because our dataset follows the OPV2V format, which uses UNREAL engine (same as CARLA) . This coordinate frame is different from the normal coordinate frame.

You can also find some examples there: https://github.com/zhz03/dataset_tool/blob/main/data_examples/v2x-real/vehicle_1/output/000030_cam1_bbx_20251110_153609.jpg

Please let me know if there are any questions regarding our code!

[zhumorui]

Hi there, thank you for using our dataset! I think the issue that you're facing is because of the transformation matrix. Please check out our dataset tool repo for getting the correct transformation: https://github.com/zhz03/dataset_tool

The problem that you are encountering is mainly because our dataset follows the OPV2V format, which uses UNREAL engine (same as CARLA) . This coordinate frame is different from the normal coordinate frame.

You can also find some examples there: https://github.com/zhz03/dataset_tool/blob/main/data_examples/v2x-real/vehicle_1/output/000030_cam1_bbx_20251110_153609.jpg

Please let me know if there are any questions regarding our code!

Thanks for providing this repo, I checked some examples, looks like there are still some mismatch.
Could you figure out why this happen?

[Merealtea]

Hi there, thank you for using our dataset! I think the issue that you're facing is because of the transformation matrix. Please check out our dataset tool repo for getting the correct transformation: https://github.com/zhz03/dataset_tool
The problem that you are encountering is mainly because our dataset follows the OPV2V format, which uses UNREAL engine (same as CARLA) . This coordinate frame is different from the normal coordinate frame.
You can also find some examples there: https://github.com/zhz03/dataset_tool/blob/main/data_examples/v2x-real/vehicle_1/output/000030_cam1_bbx_20251110_153609.jpg
Please let me know if there are any questions regarding our code!

Thanks for providing this repo, I checked some examples, looks like there are still some mismatch. Could you figure out why this happen?

Yes! I met the same problem too even using their official code for visualization. I found the temporal misalignment between LiDAR data and camera data is the key problem. You can verify it by checking bounding boxes of the moving targets and static targets in the same frame. Maybe the data sample trigger is not synchronous for LiDAR and camera within a small enough time interval.

[Merealtea]

Hi there, thank you for using our dataset! I think the issue that you're facing is because of the transformation matrix. Please check out our dataset tool repo for getting the correct transformation: https://github.com/zhz03/dataset_tool
The problem that you are encountering is mainly because our dataset follows the OPV2V format, which uses UNREAL engine (same as CARLA) . This coordinate frame is different from the normal coordinate frame.
You can also find some examples there: https://github.com/zhz03/dataset_tool/blob/main/data_examples/v2x-real/vehicle_1/output/000030_cam1_bbx_20251110_153609.jpg
Please let me know if there are any questions regarding our code!

Thanks for providing this repo, I checked some examples, looks like there are still some mismatch. Could you figure out why this happen?

Even you can draw the LiDAR points on image to check it out

[zhumorui]

Hi there, thank you for using our dataset! I think the issue that you're facing is because of the transformation matrix. Please check out our dataset tool repo for getting the correct transformation: https://github.com/zhz03/dataset_tool
The problem that you are encountering is mainly because our dataset follows the OPV2V format, which uses UNREAL engine (same as CARLA) . This coordinate frame is different from the normal coordinate frame.
You can also find some examples there: https://github.com/zhz03/dataset_tool/blob/main/data_examples/v2x-real/vehicle_1/output/000030_cam1_bbx_20251110_153609.jpg
Please let me know if there are any questions regarding our code!

Thanks for providing this repo, I checked some examples, looks like there are still some mismatch. Could you figure out why this happen?

Even you can draw the LiDAR points on image to check it out

Very interesting observation! I just visualized it and found the misalignment between lidar and camera.

[zhumorui]

Hi there, thank you for using our dataset! I think the issue that you're facing is because of the transformation matrix. Please check out our dataset tool repo for getting the correct transformation: https://github.com/zhz03/dataset_tool
The problem that you are encountering is mainly because our dataset follows the OPV2V format, which uses UNREAL engine (same as CARLA) . This coordinate frame is different from the normal coordinate frame.
You can also find some examples there: https://github.com/zhz03/dataset_tool/blob/main/data_examples/v2x-real/vehicle_1/output/000030_cam1_bbx_20251110_153609.jpg
Please let me know if there are any questions regarding our code!

Thanks for providing this repo, I checked some examples, looks like there are still some mismatch. Could you figure out why this happen?

Yes! I met the same problem too even using their official code for visualization. I found the temporal misalignment between LiDAR data and camera data is the key problem. You can verify it by checking bounding boxes of the moving targets and static targets in the same frame. Maybe the data sample trigger is not synchronous for LiDAR and camera within a small enough time interval.

@zhz03 I would appreciate if you could resolve synchronization problem between LiDAR and camera data!