Abstract
We propose in this work to synthesize online the visual content of a complex video into multiple sprites, while detecting on-the-fly each sprite’s limits. For each received frame, physically meaningful camera rotation angles and focal lengths are firstly estimated from frame-to-sprite homography. These physical parameters are then used by a thresholding-based assignment stage to confirm whether the frame can be added without distortions on one of the sprites that have been built up to that instant. In this case, the selected sprite will be updated online accordingly by applying direct frame-to-sprite warping and advanced median blending of only the newly discovered background content. Else, a new sprite must be defined while selecting the current frame as a reference frame. Extensive experiments accomplished on challenging videos demonstrate that the proposed approach can process online a long video while preventing accumulation of alignment errors.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agapito L., Hayman E., Reid I.D.: Self-calibration of rotating and zooming cameras. Int. J. Comput. Vis. 45(2), 107–127 (2002)
Amri, S., Barhoumi, W., Zagrouba, E.: Unsupervised background reconstruction based on iterative median blending and spatial segmentation. In: IEEE International Conference on Imaging Systems and Techniques, Thessaloniki, Greece, July, pp. 411–416 (2010)
Amri S., Barhoumi W., Zagrouba E.: A robust framework for joint background/foreground segmentation in complex video scenes filmed with freely moving camera. Multimed. Tools Appl. 46(2), 175–205 (2010)
Chien, S.Y., Chen, C.Y., Huang, Y.W., Chen, L.G.: Multiple sprites and frame skipping techniques for sprite generation with high subjective quality and fast speed. In: IEEE International Conference on Multimedia and Expo, Lausanne, Switzerland, August, pp. 785–788 (2002)
Farin, D.: Automatic video segmentation employing object/camera modeling techniques, Ph.D. thesis, Technische Universiteit Eindhoven (2005)
Farin, D., Haller, M., Krutz, A., Sikora, T.: Recent developments in panoramic image generation and sprite coding. In: IEEE International Workshop on Multimedia Signal Processing, Cairns, Australia, October, pp. 64–69 (2008)
Farin, D., de With, P.H.N.: Automatic video-object segmentation employing multi-sprites with constrained delay. In: IEEE International Conference on Consumer Electronics, Las Vegas, USA, January, pp. 479–480 (2006)
Farin D., de With P.H.N., Effelsberg W.: Minimizing MPEG-4 sprite coding-cost using multiple sprites. SPIE Vis. Commun. Image Process. 5308(1), 234–245 (2004)
Glantz A., Krutz A., Sikora T., Nunes P., Pereira F.: Automatic MPEG-4 sprite coding: comparison of integrated object segmentation algorithms. Multimed. Tools Appl. 49(3), 483–512 (2010)
Guerreiro, R.F.C., Aguiar, P.M.Q.: Global motion estimation: feature-based, featureless, or both ?!, In: International Conference on Image Analysis and Recognition, Póvoa de Varzim, Portugal, September, pp. 721–730 (2006)
Hartley R.I.: Self-calibration of stationary cameras. Int. J. Comput. Vis. 22(1), 5–23 (1997)
Jay W.M.: Visual field defects. Am. Fam. Physician 24(2), 138–142 (1981)
Jinzenji, K., Watanabe, H., Okada, S., Kobayashi, N.: MPEG-4 very low bit-rate video compression using sprite coding. In: IEEE International Coference on Multimedia and Expo, Tokyo, Japan, August, pp. 4–7 (2001)
Karayiannis N.B., Li Y.: A replenishment technique for low bit-rate video compression based on wavelets and vector quantization. IEEE Trans. Circuits Syst. Video Technol. 11(5), 658–663 (2001)
Kim H., Hong K.S.: Practical self-calibration of pan-tilt cameras. IEEE Vis. Image Signal Process. 148(5), 349–355 (2001)
Kunter, M., Krutz, A., Drose, M., Frater, M., Sikora, T.: Object-based multiple sprite coding of unsegmented videos using H.264/AVC’. In: IEEE International Conference on Image Processing, San Antonio, USA, October, pp. 65–68 (2007)
Kunter, M., Krutz, A., Mandal, M., Sikora T.: Optimal multiple sprite generation based on physical camera parameter estimation. In: Proceedings of SPIE Visual Communications and Image Processing, San Jose, USA, January, pp. 1–10 (2007)
Kunter, M.: Advances in sprite-based video coding towards universal usability. Ph.D. thesis, Technischen Universitat Berlin (2008)
Kuo I.S., Chen L.H.: A fast multisprite generator with near-optimum coding bit-rate. Int. J. Pattern Recognit. Artif. Intell. 23(2), 331–353 (2009)
Lai, J.H., Kao, C.C., Chien, S.Y.: Super-resolution sprite with foreground removal. In: IEEE International Conference on Multimedia and Expo, New York, USA, July, pp. 1306–1309 (2009)
Lowe D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)
Majumdar A., Sachs D.G., Kozintsev I.V., Ramchandran K., Yeung M.M.: Multicast and unicast real-time video streaming over wireless LANs. IEEE Trans. Circuits Syst. Video Technol. 12(6), 524–534 (2002)
Mei T., Yang B., Yang S.Q., Hua X.S.: Video collage: presenting a video sequence using a single image. Vis. Comput. 25(1), 39–51 (2009)
Min, C., Yu, Q., Medioni, G.: Multi-layer mosaics in the presence of motion and depth effects. In: International Conference on Pattern Recognition, Hong Kong, August, pp. 992–995 (2006)
Seo Y., Heyden A.: Auto-calibration by linear iteration using the DAC equation. Image Vis. Comput. 22(11), 919–926 (2004)
Szeliski, R., Shum, H.Y.: ‘Creating full view panoramic image mosaics and environment maps’, Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, USA, August, pp. 251–258 (1997)
Vivet, M., Martnez, B., Binefa, X.: Real-time motion detection for a mobile observer using multiple kernel tracking and belief propagation. In: Iberian Conference on Pattern Recognition and Image Analysis, Póvoa de Varzim, Portugal, June, pp.144–151 (2009)
Winkelman, F., Patras, I.: Online globally consistent mosaicing using an efficient representation. In: IEEE International Conference on Systems, Man and Cybernetics, Den Haag, The Netherlands, October, pp. 3116–3121 (2004)
Ye, G.: High-resolution multi-sprite generation for background sprite coding. In: International Conference on Advanced Concepts for Intelligent Vision Systems, Delft, The Netherlands, August, pp. 756–767 (2007)
Zagrouba E., Barhoumi W., Amri S.: An efficient image-mosaicing method based on multifeature matching. Mach. Vis. Appl. 20(3), 139–162 (2009)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Barhoumi, W., Bakkay, M.C. & Zagrouba, E. An online approach for multi-sprite generation based on camera parameters estimation. SIViP 7, 843–853 (2013). https://doi.org/10.1007/s11760-011-0273-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-011-0273-1