European Dataset

[biodatasciencearg]

Hello again! Amazing work with the new updates! I found the model trained on European license plates incredibly interesting and versatile! I even tested it on license plates from Mercosur and Mexico, and it performed very well! However, I’d like to gradually integrate more data to make it even more robust. I noticed you shared the validation dataset—do you plan to release the training dataset as well? Similar to what you did with Argentina, my idea is to generate synthetic data for Uruguay, for example.

On another note, I saw that you implemented object detection in https://github.com/ankandrew/open-image-models. In the past, we discussed how tilted license plates might pose a challenge for detection. With this new MobileViT v2 architecture you're suggesting, do you think it's necessary to correct that distortion before applying OCR? I ask because I’ve noticed that your model performs very well, even with distorted plates. On my end, I addressed that issue by determining the "pose" of the four corners of the plates and then applying an affine transformation to correct the distortion, which I implemented in YOLOv8.

[ankandrew]

Hi! Great to receive that feedback about the new model! I might release the full data in the future, it was trained with 41k plates.

Similar to what you did with Argentina, my idea is to generate synthetic data for Uruguay, for example.

When training the arg model it helped me to have a more balanced dataset, since what I collected didn't have much 2-row bikes. Also helped for the ones that my dataset didn't contain, i.e. the new A[G-Z]\d{3}\w{2} plates.

With this new MobileViT v2 architecture you're suggesting, do you think it's necessary to correct that distortion before applying OCR?

That's a good question. I tried this architecture (which has attention blocks) to see if we could obtain good results without the rectifier network, i.e. STN. The dataset I used for the European plates angles were much more difficult than the Argentinian dataset I had, and the model I used for the Argentinian ones didn't satisfy me in terms of accuracy. It would be nice to try the STN module before the MobileViTv2 and see the results if it provides noticeable improvement.

Having that said, I didn't have time to rework STN layer with Keras 3, but if you want to contribute that to the repo it would be great!

On my end, I addressed that issue by determining the "pose" of the four corners of the plates and then applying an affine transformation to correct the distortion, which I implemented in YOLOv8.

That seems like a nice idea. Did you apply the affine transformation before feeding the crop to an OCR?

[ankandrew]

I'm not tied down to the MobileViTv2 architecture I used for the European plates dataset, I mainly used it because I wanted to try with attention blocks, to have more global context at an early stage of the network. Also chose it because it is available in kimm library, here https://github.com/james77777778/keras-image-models/blob/bb231614858244615aec6ebb4caacd9208e42acf/kimm/_src/models/mobilevit.py#L888-L901.

I think for speed (which is a key criterion for me), it might be better to use a STN (rectifier) + ConvNet kind of network, like LPRNet paper does.

I'm planning to release a datase for experiments, which should be representative with worldwide plates (let me know if it sounds interesting to you). This way we can test different networks with speed/accuracy in mind.

Btw, feel free to contribute other architectures if you find them suitable for the task.