How can i use the yolov8 model as SGIE ?? It is not working.

[Yahiya-Mulla]

I tried using yolov8 as SGIE but it is not working. Sometimes the bounding boxes are correct and sometimes the bounding boxes overshoot i.e cross the PGIE BBOX.

How to get over this ??

[Yahiya-Mulla]

i get error

Deserialize yoloLayer plugin: yolo
Segmentation fault (core dumped)

[marcoslucianops]

Please use the new implementation with ONNX.

[Yahiya-Mulla]

I am getting this error with the implementation of ONNX model.

Error:
Deserialize yoloLayer plugin: yolo
Segmentation fault (core dumped)

I have tested this with Yolo v8.

When I convert the Yolo v8 model to ONNX and make it work as primary model then it works and when I add another Yolo v8 model to work as secondary detector on top of primary detectors output then I am getting the above error.

If I use older versions where in we use .cfg file and .wts file to create the engine file it works but all the secondary model outputs are expanding or are bigger than the primary models output.

I have also verified the original Yolo v8 .pt files from a simple python code where it works as expected but with deepstream it is a different story all to gather.

I just want to make the Yolo v8 model work as secondary model to detect the objects from cropped images of primary models output which deepstream automatically handles.

[marcoslucianops]

The ONNX model doesn't have serialized YOLO layer. This error shouldn't happen with ONNX files.

[marcoslucianops]

What's the pgie model?

[Yahiya-Mulla]

PGIE model is the base Yolo v8 model.

I was running the PGIE with older .cfg and .wts format. The bboxparser parser used was also the old one itself.

I’ll check the above error once with both working as ONNX model and update soon.

But for reference if I run only the PGIE which is currently generating engine files from .cfg and .wts files it works. When I add up the SGIE with the new ONNX files and bboxparser I get the above error.

[marcoslucianops]

Try to use both with the ONNX models and the updated code from the repo (for the lib and the exporter)

[Yahiya-Mulla]

Using both PGIE and SGIE as ONNX works but the output is similar as before.

Here is a scenario.
PGIE normal yolo model to detect cars.
SGIE to detect license place on top of that car.

Expected output.
PGIE detects car and SGIE detects license plate.

But the detection of SGIE is such that the detected bounding box of license plate crosses the bounding box of car. Sometimes it work as expected and most of the times license plate bounding box is out of the detected car bounding box and also sometimes the bounding box of SGIE is even bigger than that of the PGIE detection.

It is obvious that we may believe that SGIE model is not good but i generated the same pipeline with python using ultralytics and opencv. I do not find any such scenarios which i have mentioned above. It is only observed when working with deepstream.

I want to know how to overcome this and use yolo v8 model as SGIE. What changes needs to be done in the bboxparser so that it also works properly with SGIE ??

[marcoslucianops]

Add on the sgie config file (in the property section)

operate-on-gie-id=1
operate-on-class-ids=0

Set the class id you want to operate instead of 0. Make sure the gie-unique-id=1 on the pgie config file (property section).

[Yahiya-Mulla]

Yes I have done that. Also I have set process_mode=2 as well. The observations are post that.