On Thu, Aug 12, 2021 at 12:09 PM, shortcipher wrote:

I'm trying to get a tensorflow object detection model working, but I must be missing something.  I would appreciate some help. I'm using a model from the zoo: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1_detection_zoo.md#mobile-models This is the model. http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v3_small_coco_2020_01_14.tar.gz It is trained on COCO, so I'm using the labels here: https://raw.githubusercontent.com/amikelive/coco-labels/master/coco-labels-paper.txt My pipeline is as follows: gst-launch-1.0 v4l2src name=cam_src ! videoconvert ! videoscale ! video/x-raw,width=640,height=480,format=RGB,framerate=30/1 ! tee name=t t. ! queue leaky=2 max-size-buffers=2 ! videoscale ! video/x-raw,width=320,height=320,format=RGB ! tensor_converter ! tensor_transform mode=arithmetic option=typecast:uint8,add:0,div:1 ! tensor_filter framework=tensorflow2-lite model=model.tflite ! tensor_decoder mode=bounding_boxes option1=mobilenet-ssd-postprocess option2=bxr/coco-labels-paper.txt option3=3:1:2:0 option4=640:480

option2=bxr/coco-labels-paper.txt option3=0:1:2:3,50 option4=640:480

Please refer to "option3 of mobilenet-ssd-postprocess mode" section of the bound-box decoder description. 1. You are missing "threshold" value (50) 2. The output dimensions of your model is "41:1, 100:1:1:1, 100:1:1:1, 1:1:1:1", which implies that "location" is 0'th tensor and "number" is 3rd tensor while you have designated 3rd tensor as location (thus, gstcap cannot be negotiated) and 0th tensor as number (again, gstcap cannot be negotiated)

With this modification, it works well.

Cheers, MyungJoo

option5=320:320 ! compositor name=mix sink_0::zorder=2 sink_1::zorder=1 ! videoconvert ! ximagesink t. ! queue leaky=2 max-size-buffers=10 ! mix. The output is: ** Message: 19:24:39.631: accl = cpu

** (gst-launch-1.0:67): CRITICAL **: 19:24:39.689: bb_getOutCaps: assertion 'dim1[0] == 1' failed

** (gst-launch-1.0:67): CRITICAL **: 19:24:39.691: bb_getOutCaps: assertion 'dim1[0] == 1' failed Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock

(gst-launch-1.0:67): GLib-CRITICAL **: 19:24:39.870: g_string_insert_len: assertion 'len == 0 || val != NULL' failed (gst-launch-1.0:67): GLib-CRITICAL **: 19:24:39.896: g_string_insert_len: assertion 'len == 0 || val != NULL' failed (gst-launch-1.0:67): GLib-CRITICAL **: 19:24:39.907: g_string_insert_len: assertion 'len == 0 || val != NULL' failed (gst-launch-1.0:67): GLib-CRITICAL **: 19:24:39.917: g_string_insert_len: assertion 'len == 0 || val != NULL' failed (gst-launch-1.0:67): GLib-CRITICAL **: 19:24:39.923: g_string_insert_len: assertion 'len == 0 || val != NULL' failed (gst-launch-1.0:67): GLib-CRITICAL **: 19:24:39.926: g_string_insert_len: assertion 'len == 0 || val != NULL' failed ** (gst-launch-1.0:67): CRITICAL **: 19:24:39.927: bb_getOutCaps: assertion 'dim1[0] == 1' failed ** (gst-launch-1.0:67): CRITICAL **: 19:24:39.929: bb_getOutCaps: assertion 'dim1[0] == 1' failed ** (gst-launch-1.0:67): CRITICAL **: 19:24:39.930: bb_getOutCaps: assertion 'dim1[0] == 1' failed ** (gst-launch-1.0:67): CRITICAL **: 19:24:39.932: bb_getOutCaps: assertion 'dim1[0] == 1' failed ** (gst-launch-1.0:67): CRITICAL **: 19:24:40.395: bb_getOutCaps: assertion 'dim1[0] == 1' failed ** (gst-launch-1.0:67): CRITICAL **: 19:24:40.402: bb_getOutCaps: assertion 'dim1[0] == 1' failed Redistribute latency... ** (gst-launch-1.0:67): CRITICAL **: 19:24:40.439: bb_getOutCaps: assertion 'dim1[0] == 1' failed ** (gst-launch-1.0:67): CRITICAL **: 19:24:40.441: bb_getOutCaps: assertion 'dim1[0] == 1' failed ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:cam_src: Internal data stream error. Additional debug info: gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:cam_src: streaming stopped, reason not-negotiated (-4) Execution ended after 0:00:00.595088466 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ... I've double checked the output dimensions.  The error is from here: ext/nnstreamer/tensor_decoder/tensordec-boundingbox.c

Thanks for the quick response!  I tried that modification adding the threshold and permuting the output tensor indices and I'm still getting the same error.  (I also tried all other permutations of the indices 0:1:2:3,50 / 3:2:1:0,50 / 3:1:2:0 / etc) Did you make any other changes to get it to work well?

On Thu, Aug 12, 2021 at 01:41 PM, shortcipher wrote:

Thanks for the quick response!  I tried that modification adding the threshold and permuting the output tensor indices and I'm still getting the same error.  (I also tried all other permutations of the indices 0:1:2:3,50 / 3:2:1:0,50 / 3:1:2:0 / etc) Did you make any other changes to get it to work well?

Here is the gst pipeline description that I've tested:

gst-launch-1.0 v4l2src name=cam_src ! videoconvert ! videoscale ! video/x-raw,width=640,height=480,format=RGB,framerate=30/1 ! tee name=t t. ! queue leaky=2 max-size-buffers=2 ! videoscale ! video/x-raw,width=320,height=320,format=RGB ! tensor_converter ! tensor_transform mode=arithmetic option=typecast:uint8,add:0,div:1 ! tensor_filter framework=tensorflow2-lite model=model.tflite ! tensor_decoder mode=bounding_boxes option1=mobilenet-ssd-postprocess option2=coco-labels-paper.txt option3=0:1:2:3,50 option4=640:480 option5=320:320 ! compositor name=mix sink_0::zorder=2 sink_1::zorder=1 ! videoconvert ! ximagesink t. ! queue leaky=2 max-size-buffers=10 ! videoconvert ! mix.

It appears that the video format (RGB) from t might be not compatible with compositor: https://gstreamer.freedesktop.org/documentation/compositor/index.html?gi-language=c