Performence issue(?) when running 2 requests at the same time

Hi,

I’m using v3.4 GPU with RTX2060, today I started to use the 2nd camera to recognize faces,
It seems when I’m sending just 1 image per second it returns after less than 0.5 seconds,
After adding the second camera, first request returns after 0.5 second while the second request returns after ~0.9 seconds.

Details that might assist:
CPU is AMD 8350FX (8 cores), 7 are up to 5%, 1 at most getting to 40%, load:
1m-0.75, 5m-0.78, 15m-0.77

Memory: 4.8GB/16GB

GPU @6% processing, using 2.6GB/6GB

Images that being sent from both camera are in the same size (same model and configuration) - 1920*1080.

docker configuration:
VISION-FACE=TRUE
VISION-DETECTION=FALSE
MODE=HIGH
API-KEY and ADMIN-KEY available

Labels:
com.nvidia.cuda.version=9.0.176
com.nvidia.cudnn.version=7.4.2.24
com.nvidia.volumes.needed=nvidia_driver

Below logs:
[GIN] 2019/04/07 - 12:44:49 | 200 | 624.605729ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:48 | 200 | 464.737585ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:47 | 200 | 612.951007ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:46 | 200 | 466.986447ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:45 | 200 | 970.300599ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:44 | 200 | 464.818145ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:43 | 200 | 931.865943ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:42 | 200 | 478.910538ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:41 | 200 | 647.036718ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:40 | 200 | 470.077216ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:39 | 200 | 925.494124ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:38 | 200 | 466.878066ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:37 | 200 | 903.51132ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:36 | 200 | 464.104109ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:35 | 200 | 629.870196ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:34 | 200 | 460.986506ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:33 | 200 | 926.707444ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:32 | 200 | 479.420916ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/04/07 - 12:44:29 | 200 | 910.352136ms | 192.168.2.10 | POST /v1/vision/face/recognize

Hi, thanks for the detailed info, we shall investigate this further and get back to you.

Hello @elad.bar, did this issue occur in the previous version?

Not sure about GPU version, with previous CPU version it happend - 3.9 seconds to 12 seconds

Thanks for the info. I assume the performance issue you reported is with the latest cpu version?

Latest GPU version, before it happend with previous CPU version

Hello, thanks for your patience, I ran a number of tests with both the CPU and GPU Versions, the performance in your logs seem to be cpu performance.
I used a Core i7, 8GB Ram system and running with three different source of requests, each per second, the average cpu speed seems to be very regular at about 630ms.

On my GPU, an Nvidia GTX 1050 with 4GB dedicated memory, running requests every second from 5 sources comes at a speed of about 230ms per request.

I assume you might be running on cpu

I’m running the gpu package using nvidia runtime, CPU is less than 20% loaded, when I ran the CPU version CPU was 70% loaded, I’ll try to switch to CPU version to provide exact numbers.

The problem is when running 2 images requests at the same time, when adding additional image (3 at the same time) it multiples the time by 3… seems that it’s not running in parallel

Hello @elad.bar, apologies for the previous reply. The test was run using the default performance mode.
I have rerun the tests and observed some decrease in speed as the request multiplies.
We are working on fixing this.

Important things to note is that the default performance mode provides efficient parallelization on desktop GPUs. You will observe that processing many simultaneous requests in the default mode does not lead to such multiple decrease in speed. The High Mode is more computationally demanding and is more tailored towards more high end GPUs.

1 Like

Tried to switch to medium, there was no change

I’m using RTX 2060 with 6GB memory…

I changed a bit the HA custom component to first detect if person available and only then try to recognize faces, later I will release that version, it works with a flag - detect_first, default = False (requires enabling the endpoint of detect).

With flag turned on (detect before recognize):
2 requests to detect images in parallel takes about ~300 MSecs per each to return, running just 1, takes the same - performance issue as I see is just with the recognize.

With flag turned off (recognize only):
2 requests to recognize images in parallel takes about ~450 MSecs for one image, second image returns after ~900 MSec

After running it more time, it seems that the issue take place also for the detection endpoint,
not all the time:
2019/04/12 - 13:33:09 | 200 | 498.272999ms | 192.168.2.10 | POST /v1/vision/face/recognize
2019/04/12 - 13:33:09 | 200 | 313.771954ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:33:09 | 200 | 308.598806ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:33:07 | 200 | 510.082852ms | 192.168.2.10 | POST /v1/vision/face/recognize
2019/04/12 - 13:33:07 | 200 | 324.661506ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:33:07 | 200 | 327.045747ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:33:05 | 200 | 403.49396ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:33:05 | 200 | 211.703599ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:33:03 | 200 | 313.602209ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:33:03 | 200 | 316.983587ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:33:02 | 200 | 385.273973ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:33:02 | 200 | 212.217656ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:33:00 | 200 | 411.695139ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:33:00 | 200 | 210.444469ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:32:58 | 200 | 410.759305ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:32:58 | 200 | 209.583816ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:32:56 | 200 | 301.701467ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:32:56 | 200 | 303.811033ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:32:54 | 200 | 390.044484ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:32:54 | 200 | 213.229071ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:32:52 | 200 | 209.033766ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:32:52 | 200 | 205.84638ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:32:50 | 200 | 203.465957ms | 192.168.2.10 | POST /v1/vision/detection
2019/04/12 - 13:32:50 | 200 | 206.11315ms | 192.168.2.10 | POST /v1/vision/detection

Thanks for the extra details. This issue would be investigated in more detail. Generally, the speeds would be similar but are never 100% constant due to other factors and variables on the user’s system that are not coontrolled by DeepStack.

You mentioned that the speed for the Medium mode was the same as the High mode?
Can you share command you used while starting in Medium mode?

hi @john, I downloaded the latest version of GPU, the issue is still the same, or bit worst… before 2 camera returned response - first after ~210 msec, second after ~410.
Now both returns after ~300 msec with spikes to ~400 msec, running just one request at the time less than 200 msec.
GPU reaches to 7%, CPU is ~60%.

ENV
VISION-FACE True
VISION-DETECTION True
Mode Medium
TZ Asia/Jerusalem
PATH /usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
DEBIAN_FRONTEND teletype
LANG en_US.UTF-8
LANGUAGE en_US:en
LC_ALL en_US.UTF-8
CUDA_VERSION 9.0.176
CUDA_PKG_VERSION 9-0=9.0.176-1
LD_LIBRARY_PATH /usr/local/nvidia/lib:/usr/local/nvidia/lib64
NVIDIA_VISIBLE_DEVICES all
NVIDIA_DRIVER_CAPABILITIES compute,utility
NVIDIA_REQUIRE_CUDA cuda>=9.0
NCCL_VERSION 2.4.2
CUDNN_VERSION 7.4.2.24
BATCH_SIZE 32
SLEEP_TIME 0.001
API-KEY ******
ADMIN-KEY ******
Labels
com.nvidia.cuda.version 9.0.176
com.nvidia.cudnn.version 7.4.2.24
com.nvidia.volumes.needed nvidia_driver

2 request at the same time:
[GIN] 2019/04/28 - 19:29:27 | 200 | 298.624577ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:25 | 200 | 294.99364ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:24 | 200 | 293.923917ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:23 | 200 | 302.353645ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:20 | 200 | 392.118326ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:20 | 200 | 196.042031ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:18 | 200 | 293.452561ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:16 | 200 | 298.057304ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:14 | 200 | 290.944849ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:12 | 200 | 392.276629ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:12 | 200 | 192.796972ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:10 | 200 | 296.108966ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:08 | 200 | 296.276849ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:06 | 200 | 295.554151ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:04 | 200 | 294.399795ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:03 | 200 | 293.702576ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:02 | 200 | 374.338698ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:29:02 | 200 | 205.066721ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:28:59 | 200 | 298.251831ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:28:57 | 200 | 394.349118ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:28:57 | 200 | 194.807351ms | 192.168.2.10 | POST /v1/vision/detection

1 request:
[GIN] 2019/04/28 - 19:31:37 | 200 | 195.022803ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:31:35 | 200 | 197.0678ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:31:33 | 200 | 199.684812ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:31:32 | 200 | 194.055546ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:31:30 | 200 | 206.307469ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:31:28 | 200 | 199.736995ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:31:26 | 200 | 196.305268ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:31:24 | 200 | 188.705622ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:31:22 | 200 | 195.251724ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:31:20 | 200 | 190.566965ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:31:19 | 200 | 198.83528ms | 192.168.2.10 | POST /v1/vision/detection
[GIN] 2019/04/28 - 19:31:17 | 200 | 197.977337ms | 192.168.2.10 | POST /v1/vision/detection

Hello, I observed you are running object detection primarily.
I investigated this on a GTX 1050, and the speeds were consistently much higher.
I ran this tests using the Medium mode, same as your logs.

Running 1 request with a delay of 0.2 seconds between requests, the speeds are ~180 ms

v1/vision/detection
[GIN] 2019/04/29 - 08:22:20 | 200 | 174.194543ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:22:21 | 200 | 173.054744ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:22:21 | 200 | 178.314496ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:22:22 | 200 | 180.802638ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:22:22 | 200 | 149.72158ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:22:22 | 200 | 185.085107ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:22:23 | 200 | 173.896789ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:22:23 | 200 | 174.695929ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:22:23 | 200 | 168.653835ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:22:24 | 200 | 180.561788ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:22:24 | 200 | 172.569118ms | 172.17.0.1 | POST /v1/vision/detection

Running two requests simulataneously, each with a time delay of 0.2 seconds, gives about the same speed of ~ 180 ms per request. Logs below.

v1/vision/detection
[GIN] 2019/04/29 - 08:26:11 | 200 | 164.312369ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:12 | 200 | 180.823245ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:12 | 200 | 189.066542ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:12 | 200 | 171.295082ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:12 | 200 | 186.990257ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:12 | 200 | 162.517444ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:12 | 200 | 150.258987ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:13 | 200 | 178.458359ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:13 | 200 | 181.640853ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:13 | 200 | 164.563596ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:13 | 200 | 171.10879ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:13 | 200 | 159.209602ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:14 | 200 | 169.688319ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:14 | 200 | 176.610944ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:14 | 200 | 174.826762ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:14 | 200 | 178.417763ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:14 | 200 | 184.670333ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:15 | 200 | 150.776122ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:15 | 200 | 152.055415ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:15 | 200 | 180.823365ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:15 | 200 | 189.314273ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:15 | 200 | 159.755907ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:26:15 | 200 | 154.002658ms | 172.17.0.1 | POST /v1/vision/detection

Under the same conditions, i increased the number of simultaneous requests to 3. The speed was about ~300 ms

[GIN] 2019/04/29 - 08:28:09 | 200 | 302.084042ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:28:09 | 200 | 294.223913ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:28:10 | 200 | 302.120827ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:28:10 | 200 | 301.956944ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:28:10 | 200 | 304.817337ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:28:10 | 200 | 307.062709ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:28:10 | 200 | 307.180546ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:28:10 | 200 | 316.886207ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:28:11 | 200 | 290.602022ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:28:11 | 200 | 290.628968ms | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2019/04/29 - 08:28:11 | 200 | 294.63903ms | 172.17.0.1 | POST /v1/vision/detection

Also, at this rate, the GPU usage was between 51% and 60%.

I would like to know, what is the time delay between your requests?
And what version of CUDA do you have installed on the host?

hi @john

NVIDIA-SMI 418.39 Driver Version: 418.39 CUDA Version: 10.1

All test while sending a request every 2 seconds for recognize:

1 at the time:
[GIN] 2019/05/02 - 19:19:29 | 200 | 264.782115ms | 192.168.4.1 | POST /v1/vision/detection

2 in parallel:
[GIN] 2019/05/02 - 19:20:56 | 200 | 382.174353ms | 192.168.4.1 | POST /v1/vision/detection
[GIN] 2019/05/02 - 19:20:56 | 200 | 382.250314ms | 192.168.4.1 | POST /v1/vision/detection

7 in parallel:
[GIN] 2019/05/02 - 19:14:57 | 200 | 1.399554246s | 192.168.4.1 | POST /v1/vision/detection
[GIN] 2019/05/02 - 19:14:57 | 200 | 1.404882862s | 192.168.4.1 | POST /v1/vision/detection
[GIN] 2019/05/02 - 19:14:57 | 200 | 1.346302933s | 192.168.4.1 | POST /v1/vision/detection
[GIN] 2019/05/02 - 19:14:57 | 200 | 1.479314892s | 192.168.4.1 | POST /v1/vision/detection
[GIN] 2019/05/02 - 19:14:57 | 200 | 1.412461456s | 192.168.4.1 | POST /v1/vision/detection
[GIN] 2019/05/02 - 19:14:57 | 200 | 1.404826529s | 192.168.4.1 | POST /v1/vision/detection
[GIN] 2019/05/02 - 19:14:57 | 200 | 1.403879531s | 192.168.4.1 | POST /v1/vision/detection

For recognize the logs looks like (running 10 requests in the same time requests being sent every 2 seconds from another source):
[GIN] 2019/05/02 - 19:23:17 | 200 | 1.399573167s | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:17 | 200 | 2.407685505s | 192.168.4.1 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:16 | 200 | 2.143035182s | 192.168.4.1 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:16 | 200 | 1.877006071s | 192.168.4.1 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:16 | 200 | 1.620258997s | 192.168.4.1 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:16 | 200 | 1.36995364s | 192.168.4.1 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:15 | 200 | 1.095080824s | 192.168.4.1 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:15 | 200 | 819.837821ms | 192.168.4.1 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:15 | 200 | 620.671827ms | 192.168.4.1 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:15 | 200 | 381.816795ms | 192.168.4.1 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:14 | 200 | 332.230191ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:13 | 200 | 293.987688ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:11 | 200 | 288.374522ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:10 | 200 | 297.54166ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:08 | 200 | 299.465329ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:06 | 200 | 297.88231ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:05 | 200 | 296.534314ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:02 | 200 | 290.802074ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:23:00 | 200 | 288.408959ms | 192.168.2.10 | POST /v1/vision/face/recognize
[GIN] 2019/05/02 - 19:22:58 | 200 | 298.330993ms | 192.168.2.10 | POST /v1/vision/face/recognize

I just noticed that I’m running cuda version 10.1 while in docker it’s 9.0.176… could it be related?

Hello, it is possible that your cuda version might be responsible for the latencies.
DeepStack does not require cuda to be installed on the host system for it to take advantage of the GPU, however, due to the way Nvidia-Docker functions, when CUDA is installed on the system, sometimes, compatibility issues does arise. I would run some tests using CUDA 10.1 on my system and see if it leads to such issues.

Hi @john, any updates on that?

Thanks

Hi, this topic is opened for 1.5 month, and 9 days since the last time I asked for an update, should I expect for issue to be solved?