Python and Opencv add points and Houghline

Question

I’m doing my TCC in electrical engineering, where I have to automate a quadricoptero to take pictures of plantations, so far so good. The problem is that my teacher asked me to make a code in python where, with an image of a plantation, I applied a treshold, leaving the white soil and the black plants (even this part I managed to do without problems) then add adjustable size dots only in the black color area(plants) and then draw a line on top of those plants. I didn’t find anything on how to add dots on top, just the black area, and the part of the lines depends on it.

Treshold code

import cv2
import numpy as np
img = cv2.imread('C:\Users\Desktop\TCC\img1.jpg')
grayscaled = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
retval, threshold = cv2.threshold(grayscaled, 130, 255, cv2.THRESH_BINARY)
cv2.imwrite('C:\Users\Desktop\TCC\img5.jpg', threshold)
cv2.imshow('threshold', threshold)
cv2.waitKey(0)
cv2.destroyAllWindows()
cv2.imwrite('C:\Users\Desktop\TCC\img5.jpg', threshold)

To trace the lines to trying to adapt the code of this tutorial I found: https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_imgproc/py_houghlines/py_houghlines.html

Initial image:

Image after the Treshold application:

Image Illustrative of what I want the code to do :

In the last image, zoomada, has the points that would be distributed only in the black areas, and the lines on these points, I’m trying to put using the transformation of Hough, which is what I crave as a final product. They aim to present flaws in the harvesting lines.

Thanks in advance for any help or recommendation of reading and/ or tutorials.

Answer 1

It’s a cool TCC project, it doesn’t seem easy lol, I’ll try to put my ideas and perceptions.

I kept looking at your processed image and trying to find patterns with the naked eye, the crops cause a lot of noise, maybe with adaptive treshold it is possible to use only the transform of Hough, but this will depend on tests and various parameters that only you will be able to define, I have seen several algorithms that use function Hough of Opencv in the stackoverflow gringo, look only how many parameters you have to "Tunar" to maybe get to the level you want by following the answer gringa, I also share some ideas of the answer given by the author:

Yes putting everything in grayscale is highly recommended, the smaller the amount of colors, the easier it will be to find patterns, in your case you may need to test if the end result will work better in black and white or grayscale

gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

Another important point is to create a filter in your image to try to mitigate this amount of existing noise, as a student of Electrical Engineering I hope you eat this with flour for breakfast rsrs, low pass filters/high pass, etc, of course a low pass filter can soften your image, OpenCv, has functions ready for this, take a look which one will work best in your image; Blur, Gaussianblur, medianBlur, bilateral, as an example see the difference when using the Blur function

blur = cv2.blur(img,(5,5))

The image gave a blur, but look how the noises on the edges of the words were softened, noticed the value (5,5) within the function? it is more a parameter that should be set by you, what is the value of the Kernel Is it better to remove noise in your image? Only you go testing and filling the results, you will have to make an eye analysis to define which value used removes more noise in your visual perception...

The next step is to detect all the edges of the image, of course this step also has several values that will be found by you once again OpenCV has several native functions, already used Cany and Sobel to do this, see the effect that this algorithm causes on the image:

low_threshold = 50
high_threshold = 150
cv2.Canny(blur, low_threshold, high_threshold)

The example above applies Canny in the image, notice the thresholds, once again you will have to eye set what values work best for your image, I believe that when applying Canny, everything will "clear", the straight lines will be very visible and very easy for the next step(Hough Transform)

And now apply the Hough transform, according to gringo link code:

rho = 1  # distance resolution in pixels of the Hough grid
theta = np.pi / 180  # angular resolution in radians of the Hough grid
threshold = 15  # minimum number of votes (intersections in Hough grid cell)
min_line_length = 50  # minimum number of pixels making up a line
max_line_gap = 20  # maximum gap in pixels between connectable line segments
line_image = np.copy(img) * 0  # creating a blank to draw lines on

# Run Hough on edge detected image
# Output "lines" is an array containing endpoints of detected line segments
lines = cv2.HoughLinesP(edges, rho, theta, threshold, np.array([]),
                    min_line_length, max_line_gap)

Look at the amount of parameters you will have to find for your image, it is a manual work to find which values bring the best results, notice that the function cv2.HoughLinesP is returning all coordinates x,y where in your image there are possible lines...

Let’s say that this would be an "easy" way to do, just treating the image and seeing if Filtros + Canny + hough would be able to find the lines you need, test the code of the link I gave you, adapt all the parameters to the reality of your image, see if the result you get is as expected...

Now you have the Hard Way, which is the option your guidance counselor passed... I believe he asked you to draw circles in the black areas (plants) to then use only the marks(positions) of the circles, it makes sense because if you remove everything in the image and leave only the circles there will be no noise, would be a white background image with the positions of the circles and only, easy to hough find straight line patterns, but how to do it ? Hummm

It’s nothing simple lol, the initial steps I mentioned above remain a prerequisite(leave in gray or black and white scale and apply low pass filter), OK done these steps, now you will have to walk in the image with for, row by row, column by column, find the pixel of the current position, type I am in the row x spine y image and pixel here is black or white? if it’s black picked up the position I’m in the image (x,y) and draw a circle (small dot) on the image using the cvCircle

Example of use of this function:

cvCircle(img2, cvPoint(posicaoX, posicaoY), 2,  cvScalar(0, 0, 255, 0),    -1, 8, 0);

Of course I spoke in a simplistic way of more, an effective way to find the correct position and draw the circles would be, I’m in position x,y from the image, I look around this position and I see the proportion of black pixels, if it is a large proportion of black pixels I have great chances to draw a circle cvCircle in the right place, when I say look around is from the current position look a ray Z(above, below, left and right), I do not know if there is anything ready that does this in OpenCV, maybe you have to do this algorithm on the arm, or adapt and use the function connectedComponents this function can be useful for you to define a region of interest analysis (around your current position) and check the amount of points connected (black pixels), gives a read in all the documentation of the opencv, read carefully all existing functions and look for something that might make your life easier.

Walked all over the image and scored with dots, apply hough

I hope the ideas and insights I’ve given you will help clear things up.

Answer 2

The idea of solution I found was the following:

1. Upload image
2. Pull out the green
3. Threshold to binarize
4. Find lines with Probabilistic Hough Transform
5. Skeletonize or Thin

Upload image

img = cv2.imread('C:\\Users\\Desktop\\teste\\1.jpg')

Pull out the green

Converts to HSV and makes the color Slice green.

hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, (36, 0, 0), (70, 255,255))
## slice no verde
imask = mask>0
verde = np.zeros_like(img, np.uint8)
verde[imask] = img[imask]
cv2.imwrite('C:\\Users\\Desktop\\teste\\2.jpg', verde)

Threshold

Uses Threshold to transform HSV image channel V into binary

(canal_h, canal_s, canal_v) = cv2.split(verde)
retval, threshold = cv2.threshold(canal_v, 130, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)
cv2.imwrite('C:\\Users\\Desktop\\teste\\3.jpg', canal_v)

Probabilistic Transformation of Hough

Perform the Rough transform with the parameters th, minLineLength and maxLineGap variables, in which you can change them and test.

And change the width of the line created in: cv2.line(img,(x1,y1),(x2,y2),(0,0,0),15) where number 15 is the line width.

img = np.ones((3000,4000,3), np.uint8) generates a white image 4000 px x 3000 px, but the Opencv library uses binarization differently than numpy. Where numpy uses 0.1 and Opencv 0.255, therefore, to solve this problem: img[img==1]=255

th=255
minLineLength = 100
maxLineGap = 10
lines = cv2.HoughLinesP(threshold,1,np.pi/180,th,minLineLength,maxLineGap)
img = np.ones((3000,4000,3), np.uint8)
img[img==1]=255
for x in range(0, len(lines)):
    for x1,y1,x2,y2 in lines[x]:
        cv2.line(img,(x1,y1),(x2,y2),(0,0,0),15)
cv2.imwrite('C:\\Users\\Desktop\\teste\\4.jpg', img)

The transform was not very well formed by the image characteristics, in which the contrast and dimensions of the photo object are not perfect, probably a segmentation with CNN is better.

Skeletonize

Using the following function:

def find_skeleton3(img):
    # https://stackoverflow.com/a/42846932/7690982
    skeleton = np.zeros(img.shape,np.uint8)
    eroded = np.zeros(img.shape,np.uint8)
    temp = np.zeros(img.shape,np.uint8)

    retval,thresh = cv2.threshold(img,127,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)

    kernel = cv2.getStructuringElement(cv2.MORPH_CROSS,(3,3))

    iters = 0
    while(True):
    cv2.erode(thresh, kernel, eroded)
    cv2.dilate(eroded, kernel, temp)
    cv2.subtract(thresh, temp, temp)
    cv2.bitwise_or(skeleton, temp, skeleton)
    thresh, eroded = eroded, thresh # Swap instead of copy

    iters += 1
    if cv2.countNonZero(thresh) == 0:
        return (skeleton,iters)

to obtain the skeleton of the result of the Hough Transform, and then dilating the result of the Skeletonize:

img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
esqueleto, iters = find_skeleton3(img)
esqueleto =  cv2.dilate(esqueleto,kernel,iterations = 4)
cv2.imwrite('C:\\Users\\Desktop\\teste\\5.jpg', esqueleto)

Complete code

import cv2
import numpy as np

kernel = cv2.getStructuringElement(cv2.MORPH_CROSS, (3, 3))

def find_skeleton3(img):
    # https://stackoverflow.com/a/42846932/7690982
    skeleton = np.zeros(img.shape,np.uint8)
    eroded = np.zeros(img.shape,np.uint8)
    temp = np.zeros(img.shape,np.uint8)

    retval,thresh = cv2.threshold(img,127,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)

    kernel = cv2.getStructuringElement(cv2.MORPH_CROSS,(3,3))

    iters = 0
    while(True):
        cv2.erode(thresh, kernel, eroded)
        cv2.dilate(eroded, kernel, temp)
        cv2.subtract(thresh, temp, temp)
        cv2.bitwise_or(skeleton, temp, skeleton)
        thresh, eroded = eroded, thresh # Swap instead of copy

        iters += 1
        if cv2.countNonZero(thresh) == 0:
            return (skeleton,iters)

img = cv2.imread('C:\\Users\\Desktop\\teste\\1.jpg')
#HSV
#https://stackoverflow.com/a/47483966/7690982
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, (36, 0, 0), (70, 255,255))
## slice no verde
imask = mask>0
verde = np.zeros_like(img, np.uint8)
verde[imask] = img[imask]
cv2.imwrite('C:\\Users\\Desktop\\teste\\2.jpg', verde)

#Threshold
(canal_h, canal_s, canal_v) = cv2.split(verde)
retval, threshold = cv2.threshold(canal_v, 130, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)
cv2.imwrite('C:\\Users\\Desktop\\teste\\3.jpg', canal_v)

#Hough
minLineLength = 100
maxLineGap = 10
lines = cv2.HoughLinesP(threshold,1,np.pi/180,255,minLineLength,maxLineGap)
img = np.ones((3000,4000,3), np.uint8)
img[img==1]=255
for x in range(0, len(lines)):
    for x1,y1,x2,y2 in lines[x]:
        cv2.line(img,(x1,y1),(x2,y2),(0,0,0),15)
cv2.imwrite('C:\\Users\\Desktop\\teste\\4.jpg', img)

#Skeletonize
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
esqueleto, iters = find_skeleton3(img)
esqueleto =  cv2.dilate(esqueleto,kernel,iterations = 4)
cv2.imwrite('C:\\Users\\Desktop\\teste\\5.jpg', esqueleto)