More on Calibration and Reconstruction

Prévia do material em texto

More on Calibration and 
Reconstruction
Raul Queiroz Feitosa
6/5/2012 More on Calibration and Reconstruction 2
Objective
This chapter contains additional information on 
Camera Callibration and Reconstruction.
6/5/2012 More on Calibration and Reconstruction 3
Contents:
�Recalling Projective Geometry
�Vanishing Points
�Camera Calibration from VPs
�Camera from Fundamental Matrix
6/5/2012 More on Calibration and Reconstruction 4
Recall Projective Geometry 
2 Dimensional
P2 R2
x =(kx1,kx2,k)T =(x1,x2,1)T for k ≠ 0 x =(x1,x2)T
x
∞
=(kx1,kx2,0)T=(x1,x2,0)T for k ≠ 0 -
l =(kl1,kl2,k)T =(l1,l2,1)T for k ≠ 0 l =(a,b,c)T
l 
∞
=(0,0,k)T for k ≠ 0 -
p
o
i
n
t
s
l
i
n
e
s
points at 
infinity
lines at 
infinity
6/5/2012 More on Calibration and Reconstruction 5
Recall Projective Geometry 
2 Dimensional
� If a point x lies on line l
xT l = 0
� The intersection x of two lines l1 and l2
x = l1× l2
� The line joining points x1 and x2
l = x1× x2
� Note that in P2 parallel lines intersect at infinity
(a,b,c)×(a,b,c') = ( b(c'-c), a(c-c'), 0)
6/5/2012 More on Calibration and Reconstruction 6
Recall Projective Geometry 
3 Dimensional
P3 R3
x =(kx1,kx2,kx3,k)T =(x1,x2,x3,1)T for k ≠ 0 x =(x1,x2,x3)T
x
∞
=(kx1,kx2,kx3,0)T=(x1,x2,x3,0)T for k ≠ 0 -
pi=(kpi1,kpi2,kpi3,k)T= (pi1,pi2,pi3,1)T for k ≠ 0 pi = (pi1,pi2,pi3)T
pi
∞
=(0,0,0,k)T for k ≠ 0 -
p
o
i
n
t
s
p
l
a
n
e
s
points at 
infinity
planes at 
infinity
6/5/2012 More on Calibration and Reconstruction 7
Recall Projective Geometry 
3 Dimensional
� If a point x lies on plane pi xT pi = 0
� Three (non collinear) points define a plane 
� Three planes define a point 
� Note that in P3 points at infinity lie on a plane at infinity
(x1,x2,x3,0)T (0,0,0,k) =0
0pi =










 
T
T
T
1
3
2
x
x
x
0
pi
pi
pi
=










x 
T
T
T
1
3
2
6/5/2012 More on Calibration and Reconstruction 8
Contents:
�Recalling Projective Geometry
�Vanishing Points
�Camera Calibration from VPs
�Camera from Fundamental Matrix
6/5/2012 More on Calibration and Reconstruction 9
Vanishing Points
Examples
VPl VPr
VP
6/5/2012 More on Calibration and Reconstruction 10
Vanishing Points
A point in 3D space through point A=(aT,1)T with direction D=(dT,0)T
P(λ)=A+λD 
appears on a projective camera M =K [ I | 0] in�
p(λ) =M P (λ) = MA+λMD = a + λKd
where a is the image of A
A
P(λ1)
P(λ2)
d
a
C
v
d
6/5/2012 More on Calibration and Reconstruction 11
Vanishing Points
The vanishing point is given by
v = lim p(λ) =lim (a + λKd )=Kd
Note that v depends only on line direction d, not on A.
λ→∞ λ→∞
C
P1 P2 P3 P4
v D
x
v
P
The vanishing point is the intersection of a 
line parallel to the scene line through the 
camera centre and the image plane!image 
plane
6/5/2012 More on Calibration and Reconstruction 12
On computing VPs
v23 
v13
v14
v24
v
centroid
v
all intersect 
minimum average 
orthogonal distance to 
the lines → non linear
optimization
centroid estimate maximum likelihood estimate
vij=li× lj
l1 l2 l3 l4
6/5/2012 More on Calibration and Reconstruction 13
Contents:
�Recalling Projective Geometry
�Vanishing Points
�Camera Calibration from VPs
�Camera from Fundamental Matrix
6/5/2012 More on Calibration and Reconstruction 14
Camera Orientation from VPs
Result 1: Consider 2 images of a scene where 
� cameras differs only in orientation (R) and position (t) 
→ (both have the same K).
� vi and v'i are corresponding VPs
The directions di and d'i
and 
are related by the camera rotation R
'-1'-1
iii vvd KK=′
 ii dd R=′
iii vvd -1-1 KK=
6/5/2012 More on Calibration and Reconstruction 15
Angle between Lines from VPs 
Result 2: if v1 and v2 are the vanishing points of 
lines l1 and l2 with directions d1 and d2 , the angle 
between both lines can be computed from
where ω = (KKT)-1= K-T K-1
image of the absolute conic
2211
21
21
21θcos
ωvvωvv
ωvv
TT
TT
==
dd
dd
6/5/2012 More on Calibration and Reconstruction 16
K from VPs in a single view
Result 3 : From previous slide, if v1 and v2
are the vanishing points of 2 orthogonal 
lines, then
Clearly, if ω meets the constraint above, kω
also does, for all k≠0. Thus ω has 8 dof.
021 =ωvv
T
6/5/2012 More on Calibration and Reconstruction 17
K from VPs in a single view
Result 4: If it is known that
� skew angle is zero → K21=K12=0.
� pixels are square → K11=K22
ω takes the form
By setting w4 =1, ω can be estimated from three
orthogonal VPs.










=
432
31
21
0
0
www
ww
ww
ω
6/5/2012 More on Calibration and Reconstruction 18
K from VPs in a single view
Result 5: If it is known that
� skew angle is zero → K21=K12=0.
� pixels are square → K11=K22
� image center→ K13=K23=0 (by offsetting vp’s coordinates)
ω takes the form
A single pair of orthogonal VPs is enough to compute ω.










=










=










=
100
010
001
100
00
00
100
00
00
2
2
2
2
1
1
f
f
w
w
α
α
ω
focal length 
in pixels
6/5/2012 More on Calibration and Reconstruction 19
From ω to K and K'
By definition
ω=(KKT)-1=K-TK-1
Applying Cholesky factorization to ω followed by 
inversion yields K.
Cholesky factorization
A n× n symmetric positive-definite matrix A can 
be decomposed into a product of 
� a lower triangular matrix L and its transpose, i.e.,
A = L LT or
� an upper triangular matrix U and its transpose, i.e.
A = UT U
6/5/2012 More on Calibration and Reconstruction 21
Contents:
�Recalling Projective Geometry
�Vanishing Points
�Camera Calibration from VPs
�Camera from Fundamental Matrix
6/5/2012 More on Calibration and Reconstruction 22
p=M P = MHH-1P
p'=M' P = M'HH-1P
Ambiguity given F
Mi= MHi
M'i=M' Hi
Mj= MHi
M'j=M'Hj
M= K [R | t ] 
M'= K' [R' | t' ] 
F = K-T E K'-1t×R
where R=RR'-1 and t = t-Rt'
and H is some 4×4 projective transformation matrix
6/5/2012 More on Calibration and Reconstruction 23
Ambiguity given F
Two cameras M and M' determine uniquely a 
fundamental matrix F.
The converse is not true. Look
p = M P = (MH) (H-1 P)
p' = M' P = (M' H) (H-1 P)
where H is a 4×4 projective transformation matrix.
Thus, F determines the pair of camera matrices M and 
M' up to a 3D projective transformationH.
6/5/2012 More on Calibration and Reconstruction 24
Canonical camera given F
Let 
� F be a fundamental matrix and
� S be any skew symmetric matrix, 
� M =[ I | 0 ] 
� M' =[ S F | e' ]
with M' having rank 3, and e' being the epipole such 
that e' F =0.
Then F is the fundamental matrix of the pair (M ,M' ).
Possible S=[e'×].
6/5/2012 More on Calibration and Reconstruction 25
Camera from E
Recalling properties of the essential matrix :
� In the calibrated case (K and K' known) E can 
be computed from F
E= K' TF K
� E=[t×] R , where
� t vector defined by the origins of 1st to the 2nd camera 
frames 
� R rotation matrix of the 2nd to 1st camera frames 
� E is rank 2 and has 2 equal non zero singular 
values.
6/5/2012 More on Calibration and Reconstruction 26
Camera from E
Useful matrices:
Note that W Z = diag(1,1,0) andZ WT = -diag(1,1,0) 










−=









 −
=
000
001
010
 and 
100
001
010
ZW
orthogonal skew-symmetric
6/5/2012 More on Calibration and Reconstruction 27
Camera from E
Result 6: If the svd of E is U diag(1,1,0)V T, then
a) [t×]=UZUT and R = UWVT
b) [t×]=-UZUT and R =UWVT
c) [t×]=UZUT and R = UWTVT
d) [t×]=-UZUT and R =-UWTVT
are possible factorizations of E
6/5/2012 More on Calibration and Reconstruction 28
Camera from E
Geometric Interpretation of the 4 solutions:
a)A B
b)
AB
d) AB
c)
A B
base line 
reversal
B
 
r
o
t
a
t
e
d
 
1
8
0
º
6/5/2012 More on Calibration and Reconstruction 29
Reconstruction from Ground Truth
The true projective transformation H (or H-1) is such that
� H has 15 dof.
� Each correspondence provides 3 independent equations.
� Thus, 5 correspondences are enough.
� Same methods for computing homographies work here.
Pi =H Pi 
∼∼∼∼
measured using one 
pair of cameras 
(M,M')
Ground 
truth
6/5/2012 More on Calibration and Reconstruction 30
Reconstruction from Ground Truth
Algorithm
camera
estimation
(K,K’,F)
Recons-
truction
estimate
projective
transformation
×
{pi, pi}
pairs of
corresponding
points with known
ground truth
´
M
M
true camera
matrices
H
3D ground
truth
M
M
preliminar 
camera
matrices
´
´
~
~
´
~
{ Pi }
{ Pi }
preliminar 
reconstruction
~
projective
transformation
input - intermediary result - output
6/5/2012 More on Calibration and Reconstruction 31
Next Topic
Structure
from 
Motion