Vanishing Point (Line)

Presentation on theme: "Vanishing Point (Line)"— Presentation transcript:

1 Vanishing Point (Line)
9.2 基于灭点几何的深度图重建 Vanishing Point (Line)

2 从2D图像进入3D世界

3 基于灭点几何的深度图重建方法适用于结构化环境下.
将场景模化为一系列平面的集合

4 基础知识: 射影几何 Ames Room

5 观察这样一个现象 平行线交汇于一点

6 投影平面 齐次坐标系 用于表示无穷远处的点，即消失点 消失点具有单应性 图像中的一个点对应于投影空间的一条射线
image plane (x,y,1) y (sx,sy,s) (0,0,0) x z 图像平面上每个点 (x,y) 对应于一条射线 (sx,sy,s) 射线上所有点在齐次坐标系下是等价的: (x, y, 1)  (sx, sy, s)

7 直线的投影 图像中的一条直线对应于投影空间中的什么呢？ l p 直线对应于一个通过初始直线的平面（由无数条射线构成）
每条射线 (x,y,z)满足: ax + by + cz = 0 一条直线也可以表示为一个三维向量 l l p

8 点与直线的二元性质 可以得出结论： 假设有两条直线 l1 and l2 ，其交点对应于平面内一点P 所以，点和直线在投影空间内具有二元性
它 于直线上的每个点（射线）p : l p=0 l l1 l2 p p2 p1 可以得出结论： l  于 p1 和 p2  l = p1  p2 l 是平面的法线 假设有两条直线 l1 and l2 ，其交点对应于平面内一点P p  于 l1 和 l2  p = l1  l2 所以，点和直线在投影空间内具有二元性

9 理想的点和直线 理想点 (“无穷远处的点”) 理想直线 p  (x, y, 0) – 平行于图像平面 坐标无穷大
l  (a, b, 0) – 平行于图像平面 (a,b,0) y x z image plane (sx,sy,0) y x z image plane 理想点 (“无穷远处的点”) p  (x, y, 0) – 平行于图像平面 坐标无穷大 对应于图像中的一条直线 (坐标空间有限)

10 消失点 消失点 无穷远点在图像平面上的投影 由理想直线引起 image plane vanishing point camera
center ground plane 消失点 无穷远点在图像平面上的投影 由理想直线引起

11 消失点（2D） image plane vanishing point camera center line on ground plane

12 消失点 性质 两条平行线具有同一个消失点 由 C点到v点的射线平行于地平面上的直线 一幅图像可能含有多个消失点 image plane
vanishing point V line on ground plane camera center C line on ground plane 性质 两条平行线具有同一个消失点 由 C点到v点的射线平行于地平面上的直线 一幅图像可能含有多个消失点

13 消失线 v1 v2 多个消失点 平面上每组平行线定义一个消失点 所有消失点的集合构成地平线，也就是消失线 不同的平面定义了不同的消失线

14 消失线 多个消失点 平面上每组平行线定义一个消失点 所有消失点的集合构成地平线，也就是消失线 不同的平面定义了不同的消失线

15 Computing vanishing points
D Properties P is a point at infinity, v is its projection They depend only on line direction Parallel lines P0 + tD, P1 + tD intersect at P

16 Computing vanishing lines
ground plane Properties l is intersection of horizontal plane through C with image plane Compute l from two sets of parallel lines on ground plane All points at same height as C project to l points higher than C project above l Provides way of comparing height of objects in the scene

17 Is this parachuter higher or lower than the person taking this picture?
Lower—he is below the horizon

18 Fun with vanishing points

19 “Tour into the Picture” (SIGGRAPH ’97)
Create a 3D “theatre stage” of five billboards Specify foreground objects through bounding polygons Use camera transformations to navigate through the scene

20 The idea Many scenes (especially paintings), can be represented as an axis-aligned box volume (i.e. a stage) Key assumptions: All walls of volume are orthogonal Camera view plane is parallel to back of volume Camera up is normal to volume bottom How many vanishing points does the box have? Three, but two at infinity Single-point perspective Can use the vanishing point to fit the box to the particular Scene!

21 Fitting the box volume User controls the inner box and the vanishing point placement (# of DOF???) Q: What’s the significance of the vanishing point location? A: It’s at eye level: ray from COP to VP is perpendicular to image plane. Why?

22 Example of user input: vanishing point and back face of view volume are defined
High Camera

23 Example of user input: vanishing point and back face of view volume are defined
High Camera

24 Example of user input: vanishing point and back face of view volume are defined
Low Camera

25 Example of user input: vanishing point and back face of view volume are defined
Low Camera

26 Comparison of how image is subdivided based on two different camera positions. You should see how moving the vanishing point corresponds to moving the eyepoint in the 3D world. High Camera Low Camera

27 Another example of user input: vanishing point and back face of view volume are defined
Left Camera

28 Another example of user input: vanishing point and back face of view volume are defined
Left Camera

29 Another example of user input: vanishing point and back face of view volume are defined
Right Camera

30 Another example of user input: vanishing point and back face of view volume are defined
Right Camera

31 Comparison of two camera placements – left and right
Comparison of two camera placements – left and right. Corresponding subdivisions match view you would see if you looked down a hallway. Left Camera Right Camera

32 2D to 3D conversion First, we can get ratios left right top
vanishing point bottom back plane

33 2D to 3D conversion Size of user-defined back plane must equal size of camera plane (orthogonal sides) Use top versus side ratio to determine relative height and width dimensions of box Left/right and top/bot ratios determine part of 3D camera placement left right top bottom camera pos

34 Depth of the box Can compute by similar triangles (CVA vs. CV’A’)
Need to know focal length f (or FOV) Note: can compute position on any object on the ground Simple unprojection What about things off the ground?

35 DEMO Now, we know the 3D geometry of the box
We can texture-map the box walls with texture from the image

36 Foreground Objects Use separate billboard for each
For this to work, three separate images used: Original image. Mask to isolate desired foreground images. Background with objects removed

37 Foreground Objects Add vertical rectangles for each foreground object
Can compute 3D coordinates P0, P1 since they are on known plane. P2, P3 can be computed as before (similar triangles)

38 Shape From Defocus（Focus）
9.3 基于焦点变换的深度图重建 Shape From Defocus（Focus）

39 相机镜头的结构 波长越长,折射率越低 convex concave 物理学中的镜头 解决像差 (aberration )的相机镜头
两种像差 : Chromatic (色像) , Spherical aberration. 波长越长,折射率越低 convex concave

40 相机镜头的结构 Common solutions for aberrations: 几何参数
“Aspherical” 镜头 多个补偿性的光学镜头. 几何参数 F : 焦距 u: 物距. v: 像距. D: 镜头直径 R: 模糊半径 s: 镜头与 (CCD)距离 Complementary convex and concave lenses

41 相机镜头的结构 实际相机使用的是一组镜头. 有效焦距长度 : F L1 L2

42 傅立叶光学 . 孔径衍射效应. 衍射. strict on field distance, 当光波经过孔径入射时，其观测场是两种波集合：
满足几何光学的入射波. 由孔径边缘产生的衍射波. 衍射. Fresnel principle ( near-field diffraction ) Fraunhofer principle ( far-field diffraction ) . strict on field distance,

43 傅立叶光学 The Huygens-Fresnel 转换. 方波实验 :

44 傅立叶光学 通过圆形孔径（比如相机孔径）的光场强度的衍射效应近似满足近场效应，因此 其强度变化模式可用Gaussian function来描述 y1 y0 x1 x0 r01 P0(x0,y0) P1(x1,y1)

45 模糊函数 Blurring radius: R<0 Blurring radius: R>0 screen F D/2 s u
v Biconvex 2R : R<0 Blurring radius: R<0 Blurring radius: R>0 F D/2 u s 2R : R>0 screen v

46 模糊函数 根据成像光学，点光源成像后的光场强度应该是一个常数，但因为受到像差和衍射等因素的影响出现扩散模糊，扩散模糊函数可以描述为:
: 扩散系数 扩散系数与模糊半径有关: 可以根据三角测量法由成像几何推导出 为简化计算，通常假设前景以及背景都具有相同的扩散效果 这种假设可能会带来问题 K: 由相机特性决定

47 Depth from focus 主动在不同的距离拍摄多幅图像 估计焦距与图像清晰间关系.
由于散焦模型是低通滤波器，所以估计器设计为拉普拉斯算子 对所有象素点做拉普拉斯变换并求和:

48 高斯插值 使用高斯插值获得模糊函数 dp 是照相机获取最佳效果的距离 , Focus measure Measured curve
dk [SML] displacement NP Focus measure Nk Nk-1 dk-1 dp Measured curve Ideal condition dk+1 Nk+1

49 高斯插值 根据上述高斯函数可以得到dp ： 对焦深度法是一种建立在搜寻过程上的对焦方式。它通过一系列对焦逐渐准确的图像来确定物点至成像系统的距离。这个搜寻过程需要不同成像参数下的很多幅图像(一般为10幅以上，所用图像越多则对焦精度越高)。

50 Depth from Defocus 散焦深度法是一种从散焦的图像中获得物体深度信息的方法。在实际应用中，因为散焦深度法只需单眼拍摄2-3幅散焦程度不同的图像即可估计出景物到像机的距离，可实现实时的测量，因此实用性较高。 在DFD方法下，利用不同成像参数下的图像之间存在的相对模糊量，通过对图像的局部区域进行处理和分析，确定其模糊程度以及深度信息。 散焦深度法最早是由Pentland于1987年提出，他的散焦图像取样方法是将摄像机镜头光圈调到最小取得第一张图像，接着将光圈放大摄取另一张不同模糊程度的图像，并且以逆滤波(inverse filtering)的方法来计算近似为高斯函数的点扩散函数PSF (Point Spread Function)，并由此估测出目标物体到镜头的距离。

51 改变扩散系数，获得两个方程 通过改变相机参数，拍摄2-3幅图像 在空域或者频域内求解下面的方程 由方程组可以解出 u
i : 等焦距的各个子区域

52 Light Field Camera A light-field camera, also called a plenoptic camera, is a camera that uses a microlens array to capture 4D light field information about a scene. Such light field information can be used to improve the solution of computer graphics and computer vision-related problems.

53 Stanford multi-camera array
640 × 480 pixels × 30 fps × 128 cameras synchronized timing continuous streaming flexible arrangement

54 Light field photography using a handheld plenoptic camera
Ren Ng, Marc Levoy, Mathieu Brédif, Gene Duval, Mark Horowitz and Pat Hanrahan

55 Conventional versus light field camera

56 Conventional versus light field camera
uv-plane st-plane

57 Prototype camera Contax medium format camera Kodak 16-megapixel sensor Adaptive Optics microlens array 125μ square-sided microlenses 4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens

58

59 Digital refocusing Σ Σ refocusing = summing windows extracted from several microlenses

60 Example of digital refocusing

61 Digitally moving the observer
Σ Σ moving the observer = moving the window we extract from the microlenses

62 Example of moving the observer

63 Moving backward and forward