View upsampling optimization for mixed resolution 3D video coding | Multidimensional Systems and Signal Processing Skip to main content
Springer Nature Link
Log in

View upsampling optimization for mixed resolution 3D video coding

  • Published:
Multidimensional Systems and Signal Processing Aims and scope Submit manuscript

Abstract

3D video is composed out of two or more, temporally synchronized, 2D video streams acquired at different camera poses and accompanied by geometrical information. In a mixed resolution 3D video stream, a subset of views is coded at reduced resolution. It has been shown in the literature that subjective quality of mixed resolution 3D video is close to that of full resolution 3D video. In order to improve the coding gain in mixed resolution coding scenario we present a new depth encoding method called view upsampling optimization. A novel depth distortion metric based on the performance of the depth-based super resolution is also presented. Finally, to improve the quality of the decoded video an improved depth-based super resolution method that uses view synthesis quality mapping is used for upsampling of low resolution views. The simulations, performed with the recently standardized MVC+D encoder, show that the proposed solution combined with the state of the art view synthesis distortion outperforms the anchor MVC+D coding scheme by 14.5 % of dBR on average for the total coded bitrate and by 17 % of dBR on average for the synthesized views.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Aflaki, P., Hannuksela, M. M., Hakkinen, J., Lindroos, P., & Gabbouj, M. (2010). Subjective study on compressed asymmetric stereoscopic video. In 17th IEEE international conference on image processing (pp. 4021–4024).

  • Aflaki, P., Su, W., Joachimiak, M., Rusanovskyy, D., Hannuksela, M. M., Li, H., & Gabbouj, M. (2013). Coding of mixed-resolution multiview video in 3D video application. In International conference on image processing (pp. 1704–1708).

  • Aksay, A., Bilen, C., Kurutepe, E., Ozcelebi, T., Akar, G. B., Civanlar, M. R., & Tekalp, A. M. (2006). Temporal and spatial scaling for stereoscopic video compression. In Proceedings of EUSIPCO (Vol. 6, No. 8).

  • Bjontegaard, G. (2001). Calculation of average PSNR differences between RD-curves. ITU-T SG16 Q.6 document VCEG-M33.

  • Brust, H., Smolic, A., Mueller, K., Tech, G., & Wiegand, T. (2009). Mixed resolution coding of stereoscopic video for mobile devices. In 3DTV conference: The true vision-capture, transmission and display of 3D Video.

  • Chen, Y., Hannuksela, M. M., Suzuki, T., & Hattori, S. (2013). Overview of the MVC+D 3D video coding standard. Journal of Visual Communication and Image Representation, 25(4), 679–688.

    Article  Google Scholar 

  • Chen, Y., Liu, S., Wang, Y.-K., Hannuksela, M. M., Li, H., & Gabbouj, M. (2008). Low-complexity asymmetric multiview video coding. In IEEE international conference on multimedia and expo (pp. 773–776).

  • Chen, Y., Wang, Y.-K., Gabbouj, M., & Hannuksela, M. M. (2009). Regionally adaptive filtering for asymmetric stereoscopic video coding. In IEEE International Symposium on Circuits and Systems, 2009 (ISCAS 2009) (pp. 2585–2588). IEEE.

  • Chen, Y., Wang, Y.-K., Hannuksela, M. M., & Gabbouj, M. (2008). Picture-level adaptive filter for asymmetric stereoscopic video. In 15th IEEE international conference on image processing (pp. 1944–1947).

  • Chen, Y., Wang, Y.-K., Ugur, K., Hannuksela, M. M., Lainema, J., & Gabbouj, M. (2009). The emerging MVC standard for 3D video services. EURASIP Journal on Advances in Signal Processing.

  • Dinstein, I., Kim, M. G., Tselgov, J., & Henik, A. (1989). Compression of stereo images and the evaluation of its effects on 3-D perception. In Proceedings of SPIE, applications of digital image processing XII (Vol. 1153, p. 522).

  • Dong, J., He, Y., & Ye, Y. (2012). Downsampling filters for anchor generation for scalable extensions of HEVC. ISO/IEC JTC1/SC29/WG11 MPEG2012/M23485.

  • Fehn, C. (2004). Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV. In SPIE conference on stereoscopic displays and virtual reality systems XI (Vol. 5291, pp. 93–104).

  • Fehn, C., Kauff, P., Cho, S., Kwon, H., Hur, N., & Kim, J. (2007). Asymmetric coding of stereoscopic video for transmission over T-DMB. In 3DTV conference.

  • Garcia, D. C., Dorea, C., & de Queiroz, R. L. (2012). Super resolution for multiview images using depth information. IEEE Transactions on Circuits and Systems for Video Technology, 22(9), 1249–1256.

    Article  Google Scholar 

  • Hannuksela, M. M., Rusanovskyy, D., Su, W., Chen, L., Li, R., Aflaki, P., et al. (2013). Multiview-video-plus-depth coding based on the advanced video coding standard. IEEE Transactions on Image Processing, 22(9), 3449–3458.

    Article  Google Scholar 

  • ISO/IEC JTC1/SC29/WG11. (2013). Common test conditions of 3DV core experiments. JCT3V-E1100.

  • ITU-T and ISO/IEC JTC 1. (2014). Advanced video coding for generic audiovisual services. ITU-T Recommendation H.264 and ISO/IEC 14496–10 (MPEG-4 AVC).

  • Joachimiak, M., Aflaki, P., Hannuksela M. M., & Gabbouj, M. (2014). Evaluation of depth-based super resolution on compressed mixed resolution 3D video. In Asian conference on computer vision.

  • Joachimiak, M., Hannuksela, M. M., & Gabbouj, M. (2014). View synthesis quality mapping for depth-based super resolution on mixed resolution 3D video. In 3DTV-conference: The true vision-capture, transmission and display of 3D video (3DTV-CON).

  • Julesz, B. (1971). Foundations of cyclopean perception (xiv 406 pp.). University of Chicago Press.

  • Kauff, P., Atzpadin, N., Fehn, C., Muller, M., Schreer, O., Smolic, A., et al. (2007). Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability. EURASIP International Journal on Signal Processing, 22(2), 217–234.

    Google Scholar 

  • Kurutepe, E., Civanlar, M. R., & Tekalp, A. M. (2007). Client-driven selective streaming of multiview video for interactive 3DTV. IEEE Transactions on Circuits and Systems for Video Technology, 17(11), 1558–1565.

    Article  Google Scholar 

  • Meegan, D. V., Stelmach, L. B., & Tam, W. J. (2001). Unequal weighting of monocular inputs in binocular combination: implications for the compression of stereoscopic imagery. Journal of Experimental Psychology: Applied, 7(2), 143.

    Google Scholar 

  • Milanfar, P. (Ed.). (2010). Super-resolution imaging. Boca Raton: CRC Press.

    Google Scholar 

  • Oh, B. T., Lee, J., & Park, D. S. (2011). Depth map coding based on synthesized view distortion function. IEEE Journal of Selected Topics in Signal Processing, 5(7), 1344–1352.

    Article  Google Scholar 

  • Ortega, A., & Ramchandran, K. (1998). Rate-distortion methods for image and video compression. IEEE Signal Processing Magazine, 15(6), 23–50.

    Article  Google Scholar 

  • Perkins, M. G. (1992). Data compression of stereopairs. IEEE Transactions on Communications, 40(4), 684–696.

    Article  Google Scholar 

  • Quan, J., Hannuksela, M. M., & Li, H. (2011). Asymmetric spatial scalability in stereoscopic video coding. In 3DTV conference: The true vision-capture, transmission and display of 3D video (3DTV-CON).

  • Rusanovskyy, D., Hannuksela, M. M., & Su, W. (2013). Depth-based coding of MVD data for 3D video extension of H. 264/AVC. 3D Research, 4(2), 1–10.

    Article  Google Scholar 

  • Saygili, G., Gurler, C. G., & Tekalp, A. M. (2010). Quality assessment of asymmetric stereo video coding. In 17th IEEE international conference on image processing (ICIP) (pp. 4009–4012).

  • Schwarz, H., et al. (2011). Description of 3D Video Technology Proposal by Fraunhofer HHI (MVC compatible). ISO/IEC JTC1/SC29/WG11 MPEG2011/M22569.

  • Seuntiens, P., Meesters, L., & Ijsselsteijn, W. (2006). Perceived quality of compressed stereoscopic images: Effects of symmetric and asymmetric JPEG coding and camera separation. ACM Transactions on Applied Perception (TAP), 3(2), 95–109.

    Article  Google Scholar 

  • Stelmach, L., Tam, W. J., Meegan, D., & Vincent, A. (2000). Stereo image quality: Effects of mixed spatio-temporal resolution. IEEE Transactions on Circuits and Systems for Video Technology, 10(2), 188–193.

    Article  Google Scholar 

  • Tam, W. J. (2007). Image and depth quality of asymmetrically coded stereoscopic video for 3D-TV. Joint Video Team document JVT-W094.

  • Tian, J., Chen, L., & Liu, Z. (2012). Dual regularization-based image resolution enhancement for asymmetric stereoscopic images. Signal Processing, 92(2), 490–497.

    Article  Google Scholar 

  • Wenge, H., & Feng, L. (2009). Asymmetric stereoscopic video encoding algorithm based on joint compensation prediction. In WRI international conference on communications and mobile computing (Vol. 2, pp. 191–194).

Download references

Author information

Authors and Affiliations

  1. Tampere University of Technology, Tampere, Finland

    Michal Joachimiak & Moncef Gabbouj

  2. Nokia Research Center, Tampere, Finland

    Miska M. Hannuksela

Authors
  1. Michal Joachimiak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Miska M. Hannuksela
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Moncef Gabbouj
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michal Joachimiak.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Joachimiak, M., Hannuksela, M.M. & Gabbouj, M. View upsampling optimization for mixed resolution 3D video coding. Multidim Syst Sign Process 27, 763–783 (2016). https://doi.org/10.1007/s11045-015-0332-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11045-015-0332-9

Keywords

Search

Navigation