Abstract
For 3-D video applications, dense depth maps are required. We present a segment-based structure-from-motion technique. After image segmentation, we estimate the motion of each segment. With knowledge of the camera motion, this can be translated into depth. The optimal depth is found by minimizing a suitable error norm, which can handle occlusions as well. This method combines the advantages of motion estimation on the one hand, and structure-from-motion algorithms on the other hand. The resulting depth maps are pixel-accurate due to the segmentation, and have a high accuracy: depth differences corresponding to motion differences of 1/8th of a pixel can be recovered.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
M. Accame, F.G.B. De Natale, and D. Giusto. Hierarchical block matching for disparity estimation in stereo sequences. In ICIP95, pages 374–377, 1995.
G. Adiv. Determining three-dimensional motion and structure from optical flow generated by several moving objects. IEEE Trans. PAMI, 7:384–401, 1985.
S.T. Barnard and W.B. Thompson. Disparity analysis of images. IEEE Trans. PAMI, 2:333–340, 1980.
H.A. Beyer. Some aspects of the geometric calibration of CCD cameras. In ISPRS Intercomm. Conf. on Fast Processing of Photogrammetic Data, Interlaken, 1987.
G. de Haan and P. Biezen. Sub-pixel motion estimation with 3D recursive search block matching. Signal Processing: Image Communication, 6:229–239, 1994.
R. Hartley and A. Zisserman. Multiple view geometry in computer vision. Cambridge University Press, 2000.
MPEG-4 Video group ISO WG11. MPEG-4 overview (Maui Version). Technical Report ISO/IEC/JTC1/SC29/WG11 N3156, ISO, 1999.
J.R. Jain and A.K. Jain. Displacement measurement and its application in interframe image coding. IEEE Trans. Comm., 29:1799–1808, 1981.
Tony Jebara, Ali Azarbayejani, and Alex Pentland. 3D structure from 2D motion. IEEE Signal Processing Magazine, pages 66–84, May 1999.
J.L. Mallet. Discrete smooth interpolation in geometric modelling. Computer Aided Design, 24:178–191, 1992.
M. Pollefeys, R. Koch, M. Vergauwen, and L. Van Gool. Metric 3D surface reconstruction from uncalibrated image sequences. In Proc. SMILE Workshop (post-ECCV’98), LNCS 1506, pages 138–153. Springer-Verlag, 1998.
P.A. Redert, E.A. Hendriks, and J. Biemond. Correspondence estimation in image pairs. IEEE Signal Processing Magazine, 16:29–46, 1999.
R. Rodrigues, K. van Overveld, and P. Wilinski. Depth reconstruction based on irregular patches. In Proc. EPCG no 9, Marinha Grande, Portugal, 1999.
P. Salembier and F. Marques. Region-based representations of image and video: segmentation for multimedia services. IEEE Trans. CSVT, 9:1147–1169, 1999.
S. Soatto and P. Perona. Reducing “Structure from Motion”: A general framework for dynamic vision. part 1: Modeling. IEEE Trans. PAMI, 20:933–942, 1998.
H. Tao, H.S. Sawhney, and R. Kumar. A global matching framework for stereo computation. In Proc. ICCV, pages 532–539, Vancouver, Canada, 2001.
D. Terzopoulos. The computation of visible-surface representations. IEEE Trans. PAMI, 10:417–438, 1988.
R.Y. Tsai. A versatile camera calibration technique for high accuracy 3D machine vision metrology using off-the-shelf TV camera lenses. IEEE Journal on Robotics and Automation, RA-3:323–344, 1987.
C.W.A.M. van Overveld. The application of relaxation and optimisation methods in computer aided geometric design. In B. Özgüc and V. Akman, editors, Proc. of First Bilent Comp. Graphics Conf., pages 161–180, Ankara, Turkey, 1993.
L. Vincent and P. Soille. Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Trans. PAMI, 13:583–598, 1991.
D. Wang. Unsupervised video segmentation based on watersheds and temporal tracking. IEEE Trans. CSVT, 8:539–546, 1998.
A. Yezzi and S. Soatto. Stereoscopic segmentation. In Proc. ICCV, pages 59–66, Vancouver, Canada, 2001.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ernst, F., Wilinski, P., van Overveld, K. (2002). Dense Structure-from-Motion: An Approach Based on Segment Matching. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds) Computer Vision — ECCV 2002. ECCV 2002. Lecture Notes in Computer Science, vol 2351. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-47967-8_15
Download citation
DOI: https://doi.org/10.1007/3-540-47967-8_15
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43744-4
Online ISBN: 978-3-540-47967-3
eBook Packages: Springer Book Archive