<jupyter_start><jupyter_text>Sphere center detection for extrinsic calibrationSuppose we capture a sphere (i.e., a ball on a *calibration wand*) by multiple cameras, and detect the sphere center in each image as corresponding points for extrinsic calibration.  Since perspective projection of a sphere is an ellipse on the image plane, we need to find the projection of the sphere center from an ellipse.We here assume that the ball detection has been done somehow, by color-based segmentation for example.  That is, our input is a binary image such that only the ball area (i.e., ellipse) is in white as follows.<jupyter_code>%matplotlib inline
import sys, os, cv2
import numpy as np
import matplotlib.pyplot as plt

module_path = os.path.abspath(os.path.join('..'))
if module_path not in sys.path:
    sys.path.append(module_path)

import pycalib

# dummy data
W = 640
H = 480
fx = 1000
fy = fx
u0 = W // 2
v0 = H // 2
K = np.array( [[ fx, 0, u0 ], [0, fy, v0], [0, 0, 1]])
RADIUS = 10

# GT sphere center in 3D
C_gt = np.array([-20, 10, 100])

# GT sphere center in 2D
c_gt = K @ C_gt.reshape((3,1))
c_gt = (c_gt / c_gt[2]).flatten()

# Input image
img = pycalib.render_sphere(C_gt, RADIUS, K, W, H)

plt.figure()
plt.imshow(img, cmap='gray')
plt.title('Input image')
plt.show()

print(f'Sphere center in 3D (GT): {C_gt}')
print(f'Sphere center in 2D (GT): {c_gt}')<jupyter_output><empty_output><jupyter_text>First, we need to find an ellipse from this image.<jupyter_code>contours, _ = cv2.findContours(img, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
ellipse = cv2.fitEllipse(contours[0])

buf = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
buf = cv2.ellipse(buf, ellipse, (255,0,0), 4)
plt.figure()
plt.imshow(buf)
plt.scatter(c_gt[0], c_gt[1], marker='x', label='Sphere center (GT)')
plt.scatter(ellipse[0][0], ellipse[0][1], marker='.', label='Ellipse center')
plt.legend()
plt.title('Ellipse fitting')
plt.show()

print(f'Ellipse center in 2D: {ellipse[0]}')<jupyter_output><empty_output><jupyter_text>We can compute the sphere center by `pycalib.calc_sphere_center_from_ellipse()`.<jupyter_code>c = pycalib.calc_sphere_center_from_ellipse(contours[0], K)

print(f'Sphere center in 3D (GT): {C_gt}')
print(f'Sphere center in 2D (GT): {c_gt}')
print(f'Ellipse center in 2D    : {np.array(ellipse[0])} (err={np.linalg.norm(c_gt[:2] - ellipse[0]):.02f}px)')
print(f'Sphere center in 2D     : {c} (err={np.linalg.norm(c_gt[:2] - c[:2]):.02f}px)')<jupyter_output>Sphere center in 3D (GT): [-20  10 100]
Sphere center in 2D (GT): [120. 340.   1.]
Ellipse center in 2D    : [117.99397278 341.00125122] (err=2.24px)
Sphere center in 2D     : [120.03111898 339.98269589] (err=0.04px)
{'fraction_non_alphanumeric': 0.14558383233532934, 'fraction_numerical': 0.053517964071856286, 'mean_word_length': 5.033860045146727, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '246868', 'license': 'Apache-2.0', 'n_tokens_mistral': 1173, 'n_tokens_neox': 1056, 'n_words': 302, 'path': 'ipynb/sphere.ipynb', 'repo_name': 'woodyZootopia/pycalib', 'size': '2672'}