Plate recognition - Opencv - Tesseract

Question

I am doing a TCC with the theme of recognition of automotive plates. I found a problem when using the function pytesseract.image_to_string. This function should convert image to string. I tried to create a traineddata using JTESSBOXER and the SERAK TESSERACT TRAINER but did not succeed, as the right information is not coming from the plates...

I wonder if anyone can help me with this problem. If the problem really was the trining or if there could be other problems.

EDITION:

The problem was not 100% in training but in image quality. The problem has now become to improve the treatment of this image. This is the one enough to extract the text in the imagetostring. However, due to the "noises" she is not getting the information. If anyone knows how to help. I was indicated to use histogram projection to remove unwanted pixels...

Answer 1

The library pytesseract is used for OCR.

The .traineddata was created from the "Mandatory" source, which is the used in automotive plates and saved with the name Mandatory.traineddata

Code

try:
    from PIL import Image
except ImportError:
    import Image
import cv2
import pytesseract
import numpy as np
import urllib.request


def show_image(name, img):
    cv2.imshow(name, img)
    cv2.waitKey(0)


resp = urllib.request.urlopen("https://i.stack.imgur.com/davGw.jpg")
img = np.asarray(bytearray(resp.read()), dtype="uint8")
img = cv2.imdecode(img, cv2.IMREAD_COLOR)

copia = img.copy()
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(img_gray,127,255,cv2.THRESH_BINARY)

(_, conts, hierarquia) = cv2.findContours(thresh,cv2.RETR_CCOMP,2)

i=0
placa = ""
for cnt in conts:
    area = cv2.contourArea(cnt)
    x,y,w,h = cv2.boundingRect(cnt)
    if 2000 <= area <= 10000:
        cv2.rectangle(copia,(x,y),(x+w,y+h),(0,0,255),3)
        roi = thresh[y-2:y+h+2, x-2:x+w+2]
        show_image('ROI', roi)
        if i<4:
            txt = pytesseract.image_to_string(Image.fromarray(roi), lang='Mandatory', boxes=False, \
            config='--psm 10 --oem 3 -c tessedit_char_whitelist=0123456789')
            print(txt)
            placa += txt
        else:
            txt = pytesseract.image_to_string(Image.fromarray(roi), lang='Mandatory', boxes=False, \
            config='--psm 10 --oem 3 -c tessedit_char_whitelist=ABCDEFGHIJKLMNOPQRSTUVXWYZ')
            print(txt)
            placa += txt
        i+=1

print(placa[::-1])
show_image('Contornos', copia)

Explanation

• Load the Imgur image and convert to the standard format of the Opencv library

resp = urllib.request.urlopen("https://i.stack.imgur.com/davGw.jpg")
img = np.asarray(bytearray(resp.read()), dtype="uint8")
img = cv2.imdecode(img, cv2.IMREAD_COLOR)

• Copy the original image, convert to Grayscale and then binarize.

copia = img.copy()
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(img_gray,127,255,cv2.THRESH_BINARY)

• Find the Image borders with the hierarchy flag RETR_CCOMP

(_, conts, hierarquia) = cv2.findContours(thresh,cv2.RETR_CCOMP,2)

• The iteration for in each contour found with cv2.findContours()

for cnt in conts:

• Find the area of each contour and the rectangular borders of the contour, where the initial point is found (x,y), the height h and the width w

area = cv2.contourArea(cnt)
x,y,w,h = cv2.boundingRect(cnt)

• If the area is between 2000 and 10000 px

if 2000 <= area <= 10000:

• The Region of Interest (ROI) is created in each contour with a Slicer and is shown:

roi = thresh[y-2:y+h+2, x-2:x+w+2]

• Since the first four outlines are the numbers, because the outlines are found from right to left, it uses the Tesseract for digits in the first four outlines:

    if i<4:
        txt = pytesseract.image_to_string(Image.fromarray(roi), lang='Mandatory', boxes=False, \
        config='--psm 10 --oem 3 -c tessedit_char_whitelist=0123456789')
        print(txt)
        placa += txt

• In others, for capital letters:

    else:
        txt = pytesseract.image_to_string(Image.fromarray(roi), lang='Mandatory', boxes=False, \
        config='--psm 10 --oem 3 -c tessedit_char_whitelist=ABCDEFGHIJKLMNOPQRSTUVXWYZ')
        print(txt)
        placa += txt

• Settings of pytesseract.image_to_string():

Load the character image of the region of interest with Image.fromarray(roi)

The trained source is loaded lang='Mandatory'

onfig='--psm 10 --oem 3 -c tessedit_char_whitelist=0123456789'

Psm is 10 for one character as the Psm list is as follows:

Page segmentation modes:
  0    Orientation and script detection (OSD) only.
  1    Automatic page segmentation with OSD.
  2    Automatic page segmentation, but no OSD, or OCR.
  3    Fully automatic page segmentation, but no OSD. (Default)
  4    Assume a single column of text of variable sizes.
  5    Assume a single uniform block of vertically aligned text.
  6    Assume a single uniform block of text.
  7    Treat the image as a single text line.
  8    Treat the image as a single word.
  9    Treat the image as a single word in a circle.
 10    Treat the image as a single character.
 11    Sparse text. Find as much text as possible in no particular order.
 12    Sparse text with OSD.
 13    Raw line. Treat the image as a single text line,
                        bypassing hacks that are Tesseract-specific.

The oem is 3, ie the default. The Enum list of OCR Engine mode is:

  TesseractOnly           0   Run Tesseract only - fastest                                                                                                                                                                                                             
  CubeOnly                1   Run Cube only - better accuracy, but slower                                                                                                                                                                                              
  TesseractCubeCombined   2   Run both and combine results - best accuracy                                                                                                                                                                                             
  Default                 3   Specify this mode to indicate that any of the above modes should be automatically inferred from the variables in the language-specific config, or if not specified in any of the above should be set to the default OEM_TESSERACT_ONLY.

And Whitelist for numbers or uppercase letters

• Inverts the string placa

print(placa[::-1])

Upshot

Exit: EZG3164

Note: Opencv version 3.x was used, in case it is 2.x there are some incompatibilities. One of them can be solved reading this answer.