<jupyter_start><jupyter_text>Assignment 4 Instructions     Ensure that this assignment is your own work. PLagiarism will not be tolerated      Ensure that all cells of this notebook run without error when run in order      Follow the late day policy. Submit assignments on time      Provide clear explanation for the algorithms you implement Date released: 26/10/2020 Due date: 01/11/2020 Approximate time needed: 6-7 hours     DLT and Epipolar LinesThe goal of this assignment is to aid your understanding of the Direct Linear Transform as well as Epipolar geometry.For the first part of the assignment you will find correspondences between 2D-3D points and estimate the P Matrix. You will then be required to estimate Camera Parameters from this P matrix.The second part of this assignment will require you to construct epipolar lines on two corresponding images. --- 4.1 Generating CorrespondencesThe first step to perform DLT is to generate correspondences. The cell below opens a new window. Clicking anywhere on the image should give you the pixel location of the image. Once you're done clicking, close the image window. The cell after displays the points you have clicked.<jupyter_code>%matplotlib tk
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import numpy as np


image_points = []
fig = plt.figure(figsize=(20,30))

img=mpimg.imread('Images/dlt1.jpeg')

def onclick(event):
    ix, iy = event.xdata, event.ydata
    # print(ix, iy)
    image_points.append([ix, iy])

cid = fig.canvas.mpl_connect('button_press_event', onclick)

imgplot = plt.imshow(img)
plt.show()
%matplotlib inline
N = len(image_points)
image_points = np.array(image_points)
fig = plt.figure(figsize=(10,15))

img=mpimg.imread('Images/dlt1.jpeg')
imgplot = plt.imshow(img)

colors = np.random.rand(N)
area = (15 * np.ones(N))**2 

plt.scatter(image_points[:,0], image_points[:,1], c=colors, s=area)
plt.show()<jupyter_output><empty_output><jupyter_text>After generating pixel locations, you have to generate the corresponding world points. You have the freedom to chose which point you want as origin. You can consider each side as 4 units.Ensure that the ordering of your points matches the ordering of image points. Keep track of the sequence of corners clicked.<jupyter_code>#We'll rename image_points to x to match textbook convention
x = image_points
# X represents the 3D world points
X = []
#Add your points here
#Add them as 3D points only
# X.append([1,0,0])

X = np.array(X)<jupyter_output><empty_output><jupyter_text>Now that we have the correspondences, we can estimate the P matrix. Implement the DLT function that returns the P matrixQuick recap: x_i = P * X_i where (x_i, X_i) is the ith correspondence, and P is a 3x4 dimensional matrix. P is further decomposed as P = K * [ R | T].<jupyter_code>#Helper functions
#appends a ones column to input
def hom_coords(x,X):
    n = x.shape[0]
    size = (n,1)
    ones = np.ones(size)
    x = np.concatenate(
        (x,ones),
        axis = 1)
    X = np.concatenate(
        (X,ones),
        axis = 1)
    return x, X

x,X = hom_coords(x,X)<jupyter_output><empty_output><jupyter_text>ExplanationEdit this cell and explain the DLT algorithm:<jupyter_code>#DLT function takes as input 
#the image points and world points
#return a 3x4 matrix
def DLT(x,X):
    """
    your code here
    """
    P = np.random.randn(3,4)
    return P

P = DLT(x,X)<jupyter_output><empty_output><jupyter_text>Analysing PThe next part of this question is to analyse how good our estimate is. Report the reprojection error of your P matrix<jupyter_code>#Get reprojected points
def get_points(P,X):
    points = P@X.T
    points = points / points[2]
    points = points.T[:,0:2]
    return points

#Get reprojection error
def get_err(x, X, P):
    points = get_points(P, X)
    """
    find the MSE between reprojected points
    and x
    """
    #to err is to human
    return err<jupyter_output><empty_output><jupyter_text>Given the P matrix, we can estimate K, R and T. Write a function that returns K,R,T given P.Note: P = K * [R | T] where K is a 3x3 matrix and [R | T], P are 3x4 matrices<jupyter_code>def ret_RKT(P):
    """
    """
    return R, K , T<jupyter_output><empty_output><jupyter_text>Reproject the world points with P matrix you have obtained via DLT:<jupyter_code>re_proj = get_points(P, X)
#code to plot these on the image<jupyter_output><empty_output><jupyter_text>Zhangs methodFor this task, use the inbuilt Opencv function to estimate the K matrix of a camera. Use the checkerboard images 5456-5470 in images/zhangs folder for this task. Familiarize yourself with the different parameters for this method.<jupyter_code>#Code for zhangs here<jupyter_output><empty_output><jupyter_text>--- 4.2 Epipolar lines Task 1For this task, you have been given two images of the same scene taken from different view-points. The fundamental matrix that encodes their relative geometry as well as a subset of corresponding points have been provided to you. The images are in q2 folder.Recall that given a point in one image, it's corresponding location in the other image can be found to along a line viz. the epipolar line. The task given to you is to draw the epipolar lines in the second image for each given point in the first image. You have to repeat this for the other image as well. Draw epipolar lines on the first image for the corresponding points in the second image.The convention used for F is $x'^{T}Fx$ where $x'$ is the location of the point in the second image.<jupyter_code>data_file = "q2/data.txt"

with open(data_file, 'r') as f:
    A = f.readlines()
    
img_points1 = A[2:12]
img_points1 = [i.strip()[:-1] for i in img_points1]
img_points1[0] = img_points1[0][1:]
img_points2 = A[15:25]
img_points2 = [i.strip()[:-1] for i in img_points2]
img_points2[0] = img_points2[0][1:]

F_mat = A[28:31]
F_mat = [i.strip()[:-1] for i in F_mat]
F_mat[0] = F_mat[0][1:]

img_1 = []
img_2 = []
F_MAT = []

for i in img_points1:
    img_1.append(np.fromstring(i[1:-1], sep=' '))
    
for i in img_points2:
    img_2.append(np.fromstring(i[1:-1], sep=' '))
    
for i in F_mat:
    F_MAT.append(np.fromstring(i[1:-1], sep=' '))

img_1 = np.array(img_1)
img_2 = np.array(img_2)
F_mat = np.array(F_MAT)<jupyter_output><empty_output><jupyter_text>Provide a clear explanation on your approach for both this task and the next:<jupyter_code>#Write code to plot epipolar lines on the images<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09040333796940195, 'fraction_numerical': 0.019007881316643487, 'mean_word_length': 4.353184449958643, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '115538', 'license': 'MIT', 'n_tokens_mistral': 2168, 'n_tokens_neox': 2091, 'n_words': 807, 'path': 'Assignment4.ipynb', 'repo_name': 'AryanSakaria/Assignment-4', 'size': '6471'}