Is it better to separate day and night custom models, or combine into one model?

I’m training a custom model and looking for advice.

I’m wondering if it’s better to combine my daytime source images (that are in color) and my night source images (in black and white, illuminated by IR) to create one model, or if I should train them separately and to generate a “day” model and “night” model, then switch between them based on time of day?

It would best to try both and see which works better, but I’m hoping to be efficient and choose the option that will likely have better results.

Thanks!

You could try both approaches and compare them, but I suspect the tricky thing is how to combine the input from two models that have different bias - e.g. around dusk which would you trust more

I’ll 2nd Rob as well,

If both models have suitably sized training datasets, then there is little doubt that the highly specific, separated models will outperform a single combination model - that is, if you can manage to always run the IR model on IR images only, and daytime model on non-IR images only, then of coarse do that. As I doubt that IR image training would help a non-IR image detection (if your daytime dataset is already sufficient). If your dataset was small, they could maybe help one another to combine, but i think this is for a different reason. How much better would they be? I don’t know, but probably not that much. The new deepstack ‘Dark mode’ custom model for person detection seems to be really really good (in the dark), not sure if this is one of the reasons they didn’t just combine it with the main general model (to keep it specific)?

Scheduling the models to run based on time might be fine, but could have it’s own challenges too. For instance, sunrise/sunset changes throughout the year, or camera time keeping drift (if they don’t regularly sync to NTP), etc.

I thought about this as well recently for a model, but ended up combining them purely because i did not think the potential benefit was worth the effort to try to control my variable scheduling for IR. At least you don’t have to re-label your images, just move to separate folders and retrain, so pretty easy if you want to give it a go and tell us.

Thanks @robmarkcole and @shallowstack for your thoughts! Good points and I’ll test both to see how they perform.

When your ipcam goes to IR mode it outputs a single channel image/stream. Normally your images have three channels: red, blue, green.

You can get significantly better performance training two models and then selecting which model you use. I filter my input and then train either a single channel or three channel model. For inferencing, you will need to check the image first to decide which model to use. You could put an http server in front of two deep stack models which would inspect the image and route it to the right model.

# sudo apt install imagemagick
if [[ $channels -eq 1 && $dataset == 'ipcams' ]]
then
  echo "Removing color images"
  ls {train,valid,test}/*.jpg | while read f
  do echo -n "$f      "; 
    if identify -colorspace HSL -verbose "$f" | egrep -q "(Channel 0: 1-bit|red: 1-bit)" 
    then 
      echo "BW"
    else
      echo "Color"
      rm "$f"
      rm "${f%.jpg}.txt"
    fi
  done
elif [[] $channels -eq 3  && $dataset == 'ipcams' ]]
then
  echo "Removing B&W images"
  ls {train,valid,test}/*.jpg | while read f
  do echo -n "$f      "; 
    if identify -colorspace HSL -verbose "$f" | egrep -q "(Channel 0: 1-bit|red: 1-bit)" 
    then 
      echo "BW"
      rm "$f"
      rm "${f%.jpg}.txt"
    else
      echo "Color"
    fi
  done
else
  echo "No color filtering required"
fi
3 Likes