Licence plate reader

Hello! I’m thinking about building a licence plate reader and I’m trying to decide what it would be the best approach:

1 - Use deepstack to detect licence plate and then pass that image to some text recognition like Tesseract.

2 - Train a deepstack model to detect licence plates and also to detect letters inside of it.

Option 2 would be a lot slower to develop, but I’m looking for the best accuracy possible.
What would be the best approach for you guys and why?

I hope I’m not asking in the wrong place because this isn’t about deepstack itself. If anyone knows a more appropriate forum to ask this kind of questions, please let me know.

Thanks!

1 Like

I looked into this a little bit, but in the end i found that my image setup was not good enough to be reliable (the plates were pretty easily read by a human, but the character recognition methods i tried would often mix up letters. i think i need to zoom in a bit and improve contrast slightly to get it where it would be ‘good’)

but i am interested in what you do, because i did end up creating a deepstack custom object detection model to identify plates, and then crop and save to long term storage (so i can use when I find a good enough character recognition method).

if you share a few or your more challenging plate examples, i could probably run it through the pytesseract OCR i was using, and give you an idea if that would be accurate enough. otherwise try your plates on the various methods first, and see if you need to go down option 2.

1 Like

Yes I saw your model and I’m thinking about using it, haven’t tested it yet. I did a little program in C# that can do OCR but it doesn’t read licence plates at all, even with very clear images.

I’m thinking about creating a camera (Raspberry Pi + cam) that will read licence plates and verify if they are registered as stolen.
There are some crimes that are always done with stolen cars.
Then I plan to train a model to detect cars without licence plates, and if it’s possible to also recognize abnormalities like partially covered licence plates.
They would be placed at low altitude favoring the licence plate reading. I think that if I use a normal image from a CCTV camera it would be harder to read, as licence plates are far away.

Licence plates can have one of two fonts, Helvética or F Schrift, I didn’t found a model trained for them but I saw that tesseract can train for custom fonts.
I wouldn’t expect good results without training for the specific font. But please try these licence plates.

I’ve read some articles from people making licence plate readers, and it seems doable. I’ve seen that is very important to improve tha image before processing. But I’m still not sure if it would be better to train for each letter or doing a custom model for Tesseract.

Thank you for your help.
patente2 patente

There is a thread on https://github.com/odd86/deepstack_licenceplate_model/issues/1

Yes I read it. @odd86 seems that went with option number 1. But before I decide which way to go I would love to know about the accurracy of this setup. Hopefully he can comment.

WordCapturex has a model for the font he needed, but I had to contact them to see if it’s possible to train for my fonts. So I’m patiently waiting for an answer. So far my OCR program can’t detect a single letter from a good quality licence plate.

What I didn’t understand of the steps he followed is that first he says that we need to download the fonts to train from WordCapturex. But then he says he needed 800 images of licence plates to train a custom model. So the WordCapturex model wasn’t good enough?

From what I learned so far, tesstrain is the tool to train with custom fonts:
tesseract-ocr/tesstrain: Train Tesseract LSTM with make (github.com)

if you can get plates that clear you wont have any major issues. maybe the font style as you say. do you have to factor in motion blur / nighttime? or are those the worst quality you will get?

good idea to include ‘no plate’ detection. does the local laws require plates both front and back?

i’ll try those plates and get back to you tomorrow

Good point. Maybe I should test the raspi cam before buying it, maybe I need a better camera.

But these two images are considered bad quality, they are from an IP camera that should be installed in lower altitude for this purpose. Would they be recognized correctly?
Screenshot_104 Screenshot_103

plate size (# of pixels) looks a little small, but assuming those images are at least 720 or 1080p, you will likely have decent success recognizing characters from them still, but you will probably get fairly regular mixups on a few characters (U vs V, 8 vs B, O vs 0, 7 vs 1, etc.). more zoom usually helps, as well as fine tuning the contrast.

Thanks for your input.
And forgot to say that yes local law requires licence plate both front and back.

I think I could solve the mixup issue by coding a method that will take care about that. Maybe if it reads an U then check with both U and V and so on.
I wonder if it could be solved by training a model for those “hard to read” letters. So if the licence plate has a 1, then use the specialized model to verify if it’s actually a 1 or a 7.

Also the images would be taken from a video so I should use the result with highest confidence.

I will go with the Tesseract idea then. Will build a custom model using
tesseract-ocr/tesstrain: Train Tesseract LSTM with make (github.com)

Hello, sorry for my late reply! You know, work, kids, corona stuff and everything else :sweat_smile:
I found that the model for detecting license plates is very accurate even tho it’s trained on only < 300 images. I`m working on a new one with over 1600 images.

When it comes to OCR I used Tesseract 4.0 and trained different models to test and one of my models almost beat me when it came to reading Norwegian license plates. It reads 23/26

Total tests: 26
LICENCE8: 23
JD73269BP35217AJ58866

These images are actual resolution witch in my eyes makes it even more impressive. My cam at home is 4k so I can feed the system with 2- 4x resolution photos.

https://github.com/odd86/deepstack_licenceplate_model/blob/main/tesseract%20model/LICENCE8.traineddata

2 Likes

Thanks, it’s really good to know that is possible to achieve a decent accurracy. Got a couple of questions:

How are you training a custom deepstack model with over 1600 images? I’m asking because so far we have to manually label each image, doing it for over a thousand pictures must be very time consuming. I got some interesing datasets for other applications but I’m waiting for the update that will allow us to train without manually labeling.

On the Tesseract model that you made that got the best results, what did you do different from the other models that you trained?
I’m finishing a AI specialization course on coursera this week, and then will get hands on my model for Helvetica and F Schrift

It’s a manual job! But first, you make a small set, then run that on the rest of the data and let deepstack label what it can, after you manually label what deepstack did not find (and these images are the important once, cos the model did not find them so it needs training on them).

I tested with various amounts of images and color settings etc. The best one was basically with no inverting and just plain images.

Good luck with your course :smiley:

1 Like

Took me a while to be able to train a new font on Tesseract, finally made it but results are very bad.
Maybe it’s because I’m using only 100 images, I wasn’t expecting very good results but at least to read a couple of characters but it doesn’t read any. Maybe I’m missing something else.

I’m sure that I’m training correctly the Tesseract model, I practiced training with new fonts where it’s a lot easier to generate data by typing on a text file and taking screenshots and those models work very well.

I can’t scrap pages of cars for sale because by law they are required to hide it. So I’m building my own dataset manually. But I wanted to try with what I have now.

So just to test I downloaded your model LICENCE8.traineddata and renamed to license.traineddata but when I use it it doesn’t read them. I’m trying to read an image taken from your dataset:
12
When I read that image this is what I get:
Screenshot_129
It seems that “failed to load any lstm…” is just informational message.
Please let me know if I’m missing something to use your model.

And did you get better results with the 1600 images model?

Thanks!

You cannot use my model to train on (dont know why, but i build all my models from eng model.)
Its been a while since i had fun with this, but im going back in a months time, if you dont have it by then i can do a discord with you and we can find out :slight_smile:

1 Like

100 images sounds far to few. recommend min 200 train/ 100 test/valid

2 Likes

Oh yes I know but I tried to use your tesseract model to make a prediction, not to train. And I’m trying to read an image taken from the train data, so I don’t know why it didn’t work.

Yes it’s a low number, is just I wasn’t expecting to not read a single character.
Unfortunately I can’t scrap car sales websites because they are required to hide the license plate, so it seems like I’m going to need to gather the data by myself.

One option is to get some videos captured as your drive around, then use the licence plate object detection model to find frames with plates in them

you could try posting in other lpr forums (for instance, ipcamtalk has an active forum) to see if anyone in your country has a dataset they could share with you. I have some, but they likely wouldn’t be the right format/font, and your model wouldn’t work well (for you :slight_smile: , but might work for me!)

I feel that this is going to be more accurate.