Image transformation according to parameters of a photo

Question

Image transformation according to parameters of a photo

Asked 3 years, 8 months ago

Viewed 90 times

3

I have an aerial photo of a lake and I have a file . geojson with a polygon that represents the outline of that lake on a real map.

I am converting latitude, longitude and altitude to X, Y, Z using the following code:

lonDiff = lonMax - lonMin
latDiff = latMax - latMin
altDiff = altMax - altMin

if (lonDiff > latDiff and lonDiff > altDiff):
    distMax = lonDiff
if (latDiff > lonDiff and latDiff > altDiff):
    distMax = latDiff
if (altDiff > lonDiff and altDiff > latDiff):
    distMax = altDiff

adjustment = imgSize/distMax

vectors = []
vetores = np.zeros((len(coordList),3))

vectorsX = []
vectorsY = []
vectorsZ = []

for i in range(0,len(coordinates)):
    vectors[i][0] = (coordinates[i][0] - lonMin) * adjustment
    vectors[i][1] = (coordinates[i][1] - latMin) * adjustment
    vectors[i][2] = (coordinates[i][2] - altMin) * adjustment
    
    vectorsX.append((coordinates[i][0] - lonMin) * adjustment)
    vectorsY.append((coordinates[i][1] - latMin) * adjustment)
    vectorsZ.append((coordinates[i][2] - altMin) * adjustment)

What I need is to plot these values (X,Y,Z) on the screen, but with a distortion based on the angulation parameters of the moment the photo was taken so that the contour aligns perfectly to the lake. I have a txt file with the parameters (such as X_pos, Y_pos, Altitude, Yaw, Pitch, Roll, etc) of 16 different photographs of this lake.

Here’s a simple example of what I need (done in Paint):

I have tried using the related transformations of the Opencv package, which seems to be the best option here, but I couldn’t implement because I need the input of specific points to be used as reference and I can’t do that because the coordinates will change for each geojson file that is inserted in the program.

I don’t know what else I can try. It doesn’t seem to me to be something very complex in terms of code, just apply the parameters in the outline seen from above, but the math/image transformation is not clear in my head, and I’m not sure if I really need Opencv for this.

Any response or help will be very well received, and if something has not become sufficiently clear, please tell me so that I explain better and we can find some solution.

1 answer

Browser other questions tagged python opencv

You are not signed in. Login or sign up in order to post.

by Jeferson • 21 points · Answer 1 · 2021-05-21T04:24:09+00:00

As reported, you already have the points of the polygon in relation to the reference system $X,Y,Z$ (let’s call the points of $({}^{p}\mathbf{P})$ ), and the observation point parameters $({}^{o}\mathbf{P})$ . A new variable to be added is the center of the Polygon observation point $({}^{c}\mathbf{P})$ .

The difference of $({}^{c}\mathbf{P})$ and $({}^{o}\mathbf{P})$ inform it of the distance between the centre point of the map and the observation point of the photograph. This distance in linear algebra, with respect to Transformation Matrices, is called Translation $({}^d\mathbf{T})$ . The relationship between the points $({}^{c}\mathbf{P})$ and $({}^{o}\mathbf{P})$ also provides you the angles from which the photo was taken, let’s call this angle here the rotation matrix called by $({}^d\mathbf{R})$ . The set of rotation and translation operations represents a transformation matrix $(\mathbf{H})$ , as defined below:

$\underbrace { \left[ \begin{matrix} & {}^d\mathbf{R} & \\ \hline 0 & 0 & 0 \\ \end{matrix} \right. \left. \vline \begin{matrix} {}^d\mathbf{T} \\ \hline 1 \end{matrix} \right] }_{\mathbf{H}}$

The figure below graphically demonstrates the operations of a Matrix of Transformation for an X,Y reference system.

This way, you can apply a homogeneous transformation in the points of the polygon in order to "align" the same perspective of the photo by applying the following operation:

$\begin{bmatrix} {}^C_O\mathbf{P} \\ 1 \end{bmatrix} = \underbrace { \left[ \begin{matrix} & {}^C_O\mathbf{R} & \\ \hline 0 & 0 & 0 \\ \end{matrix} \right. \left. \vline \begin{matrix} {}^C_O\mathbf{T} \\ \hline 1 \end{matrix} \right] }_{{}^C_O\mathbf{H}} \begin{bmatrix} {}^P\mathbf{P} \\ 1 \end{bmatrix}$

The composition of the rotational matrices you can find here Matrix of Rotation.

Since I don’t have the original data, I’m going to demonstrate a set of dots that form the polygon of a lozenge. The rotation will be 45° only at one angle (gamma), however, rotation can be used at the other angles, as shown in Matrix of Transformation. The translation will occur in 0.4 for x and 0.6 for y, that is, the points will be moved in x and y, according to the values reported.

import numpy as np
import matplotlib.pyplot as plt


def H(R_gamma, T, angle = 'rad'):
    if angle == 'deg':
        R_gamma = np.deg2rad(R_gamma)
    th_matrix = np.array([[np.cos(R_gamma),   -np.sin(R_gamma), 0.,      T[0]],
                          [np.sin(R_gamma),   np.cos(R_gamma),  0.,      T[1]],
                          [0.,          0.,                     1.,      T[2]],
                          [0.,          0.,                     0.,      1]])
    return th_matrix


pP = np.array(
    [
     [-0.5, 0, 0, 0.5], # x
     [0, 1, -1, 0],     # y
     [1, 1, 1, 1],      # z
     [1, 1, 1, 1]       # escala
     ])

# translacao
T = np.array([0.6, 0.5, 0])
# rotacao
R_gamma = 45

coP =  H(R_gamma, T, angle = 'deg') @ pP

fig, (ax1, ax2) = plt.subplots(1, 2, figsize = (6*2,6))
ax1.plot(pP[0,:], pP[1,:], 'b .', label=r'$\mathbf{{}^oP}$')
ax1.plot(0, 0, 'x')
ax1.set_xlim(-2,2)
ax1.set_ylim(-2,2)
ax1.legend()
ax2.plot(coP[0,:], coP[1,:], 'r .', label=r'$\mathbf{{}^{co}P}$')
ax2.plot(0, 0, 'x')
ax2.set_xlim(-2,2)
ax2.set_ylim(-2,2)
ax2.legend()

The result can be seen by the chart below: