Custom Model training error - Colab

When I run the below in the Colab Custom model training, it seems to be missing the classes.txt file. when/how should this classes file be created?
My unzipped file folder is there with the images and text, but no clases file.

!python3 train.py --dataset-path “/content/lpr_dataset”

Traceback (most recent call last):
File “train.py”, line 466, in
with open(classes_file,“r”) as f:
FileNotFoundError: [Errno 2] No such file or directory: ‘/content/lpr_dataset/train/classes.txt’

Also, in the instructions, I believe the unzip code is missing a ‘!’ at the start.
And the epochs variable should be --epochs

just realized I forgot to ‘yolo’ my annotation formats. probably has something to do with it.

is there a way to re-save existing labelled images with yolo format (without having to redraw all the boxes?)

*edit - best i could find was: next image, yolo, tick difficult box, untick difficult box, ctr+S ; and repeat

1 Like

Yes, classes.txt is now present after saving in the yolo format.
It is currently training, but I couldn’t get a GPU runtime… So it seems very slow ~16min per epoch on the default settings (3 days to train!)

Are there any tips for accessing the GPU runtime in colabs? will it automatically switch when available? or do i need to keep trying a new session?

1 Like

@shallowstack you can change the runtime for your colab training to GPU by

  • go to Runtime in the Notebook menu
  • Select Change Runtime

  • Select GPU as the runtime accelerator
    image

If GPU isn’t available at the time you tried, you can try again after a few hours.

must be extra busy… I’ve been trying for 3 days straight and no luck. unfortunately my non-GPU training seems to get disconnected after about 12 hours.

what sort of values would be considered ‘pretty good’? I’m guessing the last 4 numbers should be >0.90?

  Epoch   gpu_mem       box       obj       cls     total   targets  img_size
   126/149        0G   0.02498  0.004313   0.00103   0.03032         1       640: 100% 8/8 [05:23<00:00, 40.48s/it]
               Class      Images     Targets           P           R      mAP@.5  mAP@.5:.95: 100% 1/1 [00:08<00:00,  8.67s/it]
                 all          26          21       0.958           1       0.995       0.739

Yes, colab has a max time of 12 hours, after that it disconnects. I ran into the same issue and I’m wondering what I’m going to do with even larger datasets. I guess you can play with the traning parameters to make it faster. We can also use colab pro which allows for 24 hours of connection.

1 Like

thanks @mrpie, i would not have figured that out as quickly otherwise. i at least have a best.pt file to try now!

what language are people using for the inference? (python, javascript, or c#?)
is there a simple way I can test without installing lots of software?

i tried to install various things to be able to send test images to the deepstack server running, but hit a few roadblocks,

  • my visual studio installation is maybe missing a debugger?, so not sure how easy the C# would be,
  • too many python versions are possibly confused, installed requests library, but i’m having trouble getting the requests library to import for the example code to work,
  • i installed nodejs to run the javascript code, but again it doesn’t seem to like the ‘require’ library is importing. running from firefox browser also is missing it.

is there a very simple way to test without setting up a full development environment with special libraries? maybe a vbscript? or directly from powershell?
thanks

I’m using python to test but I’m developing on C#.
Yes, you can just install Postman, it’s very easy to use. That way you can test without coding or setting up a development enviroment:

1 Like

excellent, that works very well! thanks @mrpie

Are you are using C# for windows forms capability?
I’ll likely try python and javascript installs again at some point, as I’d like to process all images in a directory (every month or so, using a task scheduler or similar), and save / delete depending on the detection

*edit- i reinstalled python39, (then ‘pip install requests’), and this time used visual studio ‘code’ with python extension and had much better luck. all working now, time to test the model on new data!! :slight_smile:

1 Like

No problem.
I’m using C# because is the language that I know the most. I’m currently learning python on python institute

results are pretty darn good (even with --model "yolov5s" and only 80 epochs)!
~200,000 images processed, and a portion are cropped and saved.

Deepstack is running twice as fast as well using this model/method.

@mypie Let us know if you try out the Colab Pro!

1 Like

I must learn to configure it better then because it took 12 hours for me to train with only 1,100 images.

How you were able to annotate 200,000 images? :open_mouth:
I have some data waiting to be used on the next update that will allows us to train without needing to annotate each image. I just came across a database of annotated images and that allowed me to test a custom model.

ahha no, whoops didn’t mean to imply that. my training dateset was a measly 200 images, (although it is for a simple object i suppose).

i’ve just run the custom model (best.pt) on my full set of stored data (200,000 images, so no annotating required)

Next I need a solution for optical character recognition in my images, i wonder how a custom deepstack model would work from generic character training datasets… probably better methods out there, but interesting to try!

that sounds amazing with the ‘auto / skipped annotating’, please share after next update.

1 Like