Slow performance on 4Gb jetson nano

I’ve been running Deepstack on a 4Gb Jetson Nano dev kit for a while - and initially, I was getting ~280-350ms per image - which was great.

Now it seems that the image detection is returning 1s+ processing times:

[GIN] 2022/01/09 - 22:21:38 | 200 |  1.312451188s |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/09 - 22:21:40 | 200 |  452.552868ms |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/09 - 22:21:42 | 200 |  1.282738598s |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/09 - 22:21:44 | 200 |  1.255284078s |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/09 - 22:21:46 | 200 |  1.285991639s |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/09 - 22:22:06 | 200 |  1.338995478s |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/09 - 22:22:08 | 200 |  1.294107705s |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/09 - 22:22:10 | 200 |   1.27414304s |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/09 - 22:22:12 | 200 |  1.310547968s |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/09 - 22:22:14 | 200 |  1.298691609s |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/09 - 22:22:16 | 200 |  1.276047197s |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/09 - 22:22:18 | 200 |  1.272013826s |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/09 - 22:22:20 | 200 |  1.283279598s |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/09 - 22:22:22 | 200 |  1.257982366s |     172.31.1.89 | POST     /v1/vision/detection

No matter what I try and do, I can’t seem to get it back to the sub-400ms processing time range.

I’m using the following script to launch Deepstack:

#!/bin/bash

IMAGE="deepquestai/deepstack:jetpack"

docker pull ${IMAGE}
docker stop deepstack

docker run -d --rm --gpus all --cpus=3 \
	-e TZ=Australia/Melbourne \
	-e VISION-DETECTION=True \
	-e MODE=Medium \
	-p 5000:5000 \
	-v localstorage:/datastore \
	--name deepstack \
	${IMAGE}

Hi @CRCinAU , did you try restarting DeepStack to see if it might resolve this issue?

If you did and it persisted, any chance you are running another compute intensive application/program on the Jetson device?

There is a chance DeepStack might be throttled due to another program using a lot of memory on the Jetson Nano. Try to get compute usage on the device to see which programs are using most of the compute.

I think I may have found the issue… It seems the frame grabbing from the video stream sometimes had a delay which meant that it was possible to send multiple images in a short timeframe.

Although you can’t really tell from the logs, it seems that if you send through multiple images to be processed at once, things don’t go really well. After a while, it seems that even single images get the same delay.

I’ve worked on my image processing a little before I then submit the pictures to deepstack, and now I’m seeing response times like:

[GIN] 2022/01/16 - 11:42:47 | 200 |  405.998223ms |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/16 - 11:42:47 | 200 |  406.116767ms |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/16 - 11:42:49 | 200 |  377.855668ms |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/16 - 11:42:50 | 200 |  379.918786ms |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/16 - 11:42:52 | 200 |  392.628237ms |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/16 - 11:42:53 | 200 |   397.33579ms |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/16 - 11:42:55 | 200 |  392.760949ms |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/16 - 11:42:56 | 200 |  389.692651ms |     172.31.1.89 | POST     /v1/vision/detection
[GIN] 2022/01/16 - 11:42:58 | 200 |  388.998729ms |     172.31.1.89 | POST     /v1/vision/detection