Didn't detect the cat

:smile:

I’ve finally gotten my BI/Deepstack/AI Tool combo running decent, and not alerting too often, and mostly alerting on stuff that matters. It’s been fairly stable / reliable the last week, so yesterday I decided (in AI Tool) to try detecting Dogs and Cats.

This morning, I saw a cat…but, it wasn’t detected or alerted on.

I don’t know if this is a BI, an AI Tool, or a Deepstack issue, but I do know there were pics sent to Deepstack for processing, and Mr. Cat wasn’t detected.

So, @john, here’s a link to a OneDrive folder with all the detected photos and even a video clip showing Mr. Cat wandering around (I have my SD cam recording 24x7). https://1drv.ms/u/s!AgRu8KsUhmARkg3FO8wd9dGgsqrx?e=qznN67

Can you postulate on a)why Mr. Cat wasn’t detected, and b)what I could do to enhance that detection?

This is mostly for “fun”, as we don’t really have an “animal problem” in our yard. I just thought I’d experiment with some other detections, and am now curious if I’m doing something wrong.

Edit: Oh, DeepStack is running on a VM in a docker container; I use this to start it:

docker run --restart always -d -e VISION-DETECTION=True -e MODE=Low -v localstorage:/datastore -p 80:5000 --name deepstack deepquestai/deepstack:cpu-x6-beta

Thanks!

1 Like

@wilddoktor there are many factors which may affect why an object is not detected, or detected with a low confidence. I notice you set MODE=Low and suggest you try MODE=High
With High setting I quickly put one of your images through the Deepstack-ui and think we have identified an issue - cats and dogs are easily confused by the AI…

[
0:{
"confidence":0.60416687
"label":"dog"
"y_min":740
"x_min":1108
"y_max":900
"x_max":1292
}
]

However in another image it is detecting cat:

[
0:{
"confidence":0.533543
"label":"cat"
"y_min":457
"x_min":1464
"y_max":559
"x_max":1597
}
]
1 Like

I would suggest cropping the frame to reasonable area - say 2 or 3 slabs and the grit on the right as
you don’t really need to run the detection over the street, bushes or walls. You could actually cut the area in 3-4 squares and feed them to deepstack one after another in a loop.

Cat
In the above crop, deepstack discovers cat with 95% confidence.

Hope that helps.

1 Like

@sickidolderivative this is a good suggestion, and this ‘tiling’ (or ‘slabs’) approach is commonly taken by people with large images (e.g. satellite imagery) so as to take advantage of the full resolution of the image. The only edge case to consider is what happens if the cat happens to be on 2 tiles at the same time, in which case it is cut in half.

1 Like

Thanks to both of you for your responses!

@robmarkcole, I run in mode=low because in that mode the average detection rate is between 350 and 600ms. In high, I average well over a second. (Since I’m mainly interested in knowing ASAP when a person approaches the front of our house (or when a hoodlum walks in from the side yard to burn our American Flag at 11pm when we’re all asleep), I thought having a shorter detection time might be better. But…I have no clue if that logic is sound and am willing to be taught! :smiley: )

@sickidolderivative, that’s a cool concept, but I’ve no idea how to implement it? Do you have a link to a post somewhere that explains it? I’d love to give it a try! Again, i’m willing to be taught!

@wilddoktor there is a tradeoff between speed and accuracy. In your case it sounds like you need both so I suggest you also need to instance of Deepstack running - one low and one high accuracy

@wilddoktor, this is very simple. Conceptually, you’d be doing something along the lines:

#!/usr/bin/python3

import itertools, enum
import numpy as np
import cv2

class Areas(enum.Enum):
    Area_1 = 1
    Area_2 = 2
    Area_3 = 3

# iterate infinitly over areas
for area in itertools.cycle(Areas):

    # read new frame from cv2.VideoCapture
    ret, new_frame = cap.read()

    # create distinct slice for each area
    if area == Areas.Area_1:
        slice = new_frame[y_min_1: y_max_1, x_min_1: x_max_1]
    elif area == Areas.Area_2:
        slice = new_frame[y_min_2: y_max_2, x_min_2: x_max_2]
    else:
        slice = new_frame[y_min_3: y_max_3, x_min_3: x_max_3]

    # pass slice to deepstack for processing
    deepstack.process(slice)
1 Like

You could perhaps overlap the slices to some extend to account for the edge case.

My cat often gets detected as a bird for some odd reason!

2 Likes

Hi, just adding to the cat chat fun…

I’m trying to detect when my cat jumps on the kitchen counter so home assistant shouts at it through a speaker to “get down”. Running a similar BI/Deepstack/AI Tool setup as discussed above.

I ran it today and got no detections at all on the cat. Deepstack is running on the windows CPU version (deepstack --VISION-DETECTION True --PORT 80). I did however get 2 false positives on me from at least 40-50 images of a human in the kitchen. On the plus side the automation aspect worked really well and the booming speaker made me jump!

Reading this forum thread I have now just tried running Deepstack with --MODE High and put the same images through manually. Got a hit rate on detecting the cat for 4 out of 10 images with reasonable certainty.

Can anyone recommend how I can improve the hit rate of detection and possibly the certainty?
I’m using the SD stream but maybe could switch to HD for better resolution on the images?

BI images and AI Tool outputs here: https://1drv.ms/u/s!AjcTSh3_t_lRhS77_r-FoiC-qre-?e=IZbbg7

Many thanks

clean your kitchen counter space? :laughing:

does cam have wide dynamic range? windows are overexposed, but cat is under exposed. getting good lighting or exposure on the counters might help.
can you reposition camera lower, and closer to counters. you probably dont need all that ceiling /cupboard in your image.

otherwise try to train a custom model for your specific ‘cat on kitchen counter’ needs. i bet your cat shown will get id’ pretty easy with it’s markings, and it wouldn’t need to try to match the generic cat model.

1 Like

Hahaha I considered spending time clearing up the counter tops at the weekend but then decided the automation would be way more fun :smiley:

Thanks, I’ve moved the camera to a lower view point and will see the results tomorrow along with some muddy paw prints I’m sure.

It is a fairly wide angle lens so if this all fails I could try getting a camera with a zoom feature.

I’ll try the training at the weekend too as you suggest. Will update soon!

1 Like

Moving the camera has helped with increasing the hit rate thanks. Now detecting the cat successfully in 66% of the images.

I have trained a custom model and created a best.pt file but I am having issues using it with AI tool. I get this error when processing an image relating to DetectObjects. I annotated the cat as ‘cat’.

Got http status code 'Forbidden' (403) in 27ms: Forbidden
Empty string returned from HTTP post

It looks like the image is being posted successfully to Deepstack but the returned data is not what AItool is expecting.

I’ve trawled the forums with no luck. Any ideas?

I don’t know if the AItool supports custom models or not. But as you say, it probably isn’t receiving its expected data&format. double check your logs that it has the same output. something like-

[
0:{
"confidence":0.533543
"label":"cat"
"y_min":457
"x_min":1464
"y_max":559
"x_max":1597
}
]

when i created and ran my own custom model I instead used a custom python script to test and perform the desired actions. here’s an example below, i think there are others in the docs.

###python file - Custom_LPR_detection_v20201229###


import requests

import os

import PIL


# Importing Image class from PIL module

from PIL import Image


input_directory = 'J:\\BlueIris\\LPR_Alerts'                          #where BlueIris saves jpg from motion trigger

output_directory = 'J:\\BlueIris\\Deepstack_LPR_dataset\\results'    #where python will save the cropped plates


#cleanup variables before run, just in case

left = 0

top = 0

right = 0

bottom = 0

label = 0

confidence = 0.0


#goes through entire directory and processes each file

for filename in os.listdir('J:\\BlueIris\\LPR_Alerts'):

 

    filepath = os.path.join(input_directory, filename)

 

    image_data = open(filepath,"rb").read()


    #posts to deepstack custom server and logs result

    response = requests.post("[URL unfurl="true"]http://localhost:80/v1/vision/custom/best[/URL]",files={"image":image_data}).json()     #change port 80 to whatever your deepstack custom server is on


    #log result for debugging

    for detection in response["predictions"]:

        label = detection["label"]

        confidence = detection["confidence"]

      

        print(label)

 

    print(response)

 

    #if object["confidence"] is not zero :

    #when a plate is found, perform an action (crop and save image to output_directory)

    if confidence > 0.2:

        #log details of the match

        print("yes, it is a positive match.  A plate was found in " + filename + "  with " + str(confidence) + " confidence.")


        # Opens a image in RGB mode

        im = Image.open(filepath)


        # Setting the points for cropped image

        # include extra margin of ~20pixels

        left = detection["x_min"]   - 20

        top = detection["y_min"]    - 20

        right = detection["x_max"]  + 20

        bottom = detection["y_max"] + 20


        #ensure margin does not extend beyond image limits  - turn this on if it helps

        #if left < 0:        left    = 0

        #if top < 0:         top     = 0

        #if right > 1080:    right   = 1080

        #if bottom > 1080:   bottom  = 1080


 

        ###avoid false positives due to timestamp.  filter out image corner  - this was commented out since it was usually actual plates under the timestamp

        ##if object["x_min"] > 100 or object["y_min"] > 40 :

          

            # Cropped image of above dimension

            # (It will not change orginal image)

        im1 = im.crop((left, top, right, bottom))


            # Shows the image in image viewer

            ##im1.show()

                      

            # save a image using extension

        im1 = im1.save(output_directory + "\\cropped_" + filename)

          

            #log action for debugging

        print("image saved as " + output_directory + "\\cropped_" + filename)


        ##else

            ##print("No image saved: likely a false positive due to timestamp")


        #cleanup variables for next image processed

        detection["confidence"] = 0.0

        confidence = 0.0

        left = 0

        top = 0

        right = 0

        bottom = 0


    else:

        print("No Plate was found in image " + filename)
2 Likes

@shallowstack thanks very much for this! Works perfectly. 100% hit rate. Now this is sorted I can use my spare time to clean the kitchen, cheers :grin:

2 Likes