Intel GPU support?

These are my first experiences with Deepstack and I`m quite impressed, how easily I got in touch with Deep Learning. Until today I have used the docker image deepquestai/deepstack:noavx. My images (2560x1440) always were analyzed in 5-6 seconds (Medium setting) with a skylake NUC i3.

Today I was curious if it supports any hardware acceleration of the internal graphics card. I followed this guide to activate hardware acceleration in docker and started the container with deepquestai/deepstack:latest.

The objects are now detected in below 2 seconds. This is quite impressive and I`m asking me: is it really hardware acceleration or is the docker image with the latest-tag much better optimized? Can anybody help me to answer this question?

These are my results:

[GIN] 2020/06/08 - 21:26:29 | 200 | 4.714147089s | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2020/06/08 - 21:26:38 | 200 | 1.548099324s | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2020/06/08 - 21:27:03 | 200 | 1.598288724s | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2020/06/08 - 21:27:11 | 200 | 1.572769416s | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2020/06/08 - 21:27:22 | 200 | 1.836138123s | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2020/06/08 - 21:27:35 | 200 | 1.57784536s | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2020/06/08 - 21:27:49 | 200 | 1.545631456s | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2020/06/08 - 21:28:03 | 200 | 1.610441244s | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2020/06/08 - 21:28:10 | 200 | 1.419948261s | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2020/06/08 - 21:29:20 | 200 | 1.705897358s | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2020/06/08 - 21:29:28 | 200 | 1.671761332s | 172.17.0.1 | POST /v1/vision/detection
[GIN] 2020/06/08 - 21:39:21 | 200 | 1.707628028s | 172.17.0.1 | POST /v1/vision/detection

Some would say 1.5 seconds is still too slow with GPU support. But you know, this is free unlimited analysis so I wouldn’t complain too much.

The test is simple. Setup another docker:latest with NO GPU drivers on a modern CPU (as it uses CPU extensions instead) and compare

I am using docker:no-avx using 640x480 images from BlueIris CCTV

  • Server Specs - 2.8Ghz X5660 (Very Old Xeon but cheap and power efficient) with 2 CPU’s 24 Cores total resource
  • 6 seconds with 1 vCore 1GB RAM (default first install)
  • 2 seconds with 3 vcores 8GB RAM (optimimal)
  • 2 seconds with 16 vcores 16Gb (no speed gain, wasted resource)

I played with the setting until it couldn’t go any faster

I want to find a newer generation 8th+ gen Intel CPU and compare it with less RAM using the latest version and then I also have a 1070GPU to test it with (Linux only) But not had time yet (for me 2 seconds, unlimited is OK for now - I will see how it handles when I start to ramp up the requests from more cameras)

I think only NVIDIA GPU are supported by the underlying libraries