Compressed-sensing recovery of multiview image and video sequences using signal prediction | Multimedia Tools and Applications Skip to main content
Springer Nature Link
Log in

Compressed-sensing recovery of multiview image and video sequences using signal prediction

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In the compressed sensing of multiview images and video sequences, signal prediction is incorporated into the reconstruction process in order to exploit the high degree of interview and temporal correlation common to multiview scenarios. Instead of recovering each individual frame independently, neighboring frames in both the view and temporal directions are used to calculate a prediction of a target frame, and the difference is used to drive a residual-based compressed-sensing reconstruction. The proposed approach demonstrates a significant gain in reconstruction quality relative to the straightforward compressed-sensing recovery of each frame independently of the others in the multiview set, as well as a significant performance advantage as compared to a pair of benchmark multiple-frame compressed-sensing reconstructions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://www.ece.msstate.edu/~fowler/BCSSPL

  2. http://www.caam.rice.edu/~optimization/L1/TVAL3/

  3. http://people.csail.mit.edu/celiu/OpticalFlow/

  4. http://cat.middlebury.edu/stereo/data.html

  5. Five 555 × 626 × 3 multiview image sets: Aloe, Baby3, Bowling1, Plastic, and Monopoly

  6. http://bisp.kaist.ac.kr/research_02.htm

  7. http://home.engineering.iastate.edu/~luwei/modcs/

  8. Provided courtesy of Fraunhoffer HHI.

  9. The “Ballet” and “Break Dancer” multiview video sequences are available, courtesy of Microsoft Research, from http://research.microsoft.com/en-us/um/people/sbkang/3dvideodownload/.

References

  1. Bioucas-Dias JM, Figueiredo MAT (2007) A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans Image Process 16(12):2992–3004

    Article  MathSciNet  Google Scholar 

  2. Blumensath T, Davies ME (2009) Iterative hard thresholding for compressed sensing. Appl Comput Harmon Anal 27(3):265–274

    Article  MATH  MathSciNet  Google Scholar 

  3. Candès E, Tao T (2006) Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans Inf Theory 52(12):5406–5425

    Article  Google Scholar 

  4. Candès EJ (2006) Compressive sampling. In: Proceedings of the International Congress of Mathematicians, vol 3, pp 1433–1452. Madrid, Spain

  5. Candès EJ, Wakin MB (2008) An introduction to compressive sampling. IEEE Signal Process Mag 25(2):21–30

    Article  Google Scholar 

  6. Chambolle A, Lions PL (1997) Image recovery via total variation minimization and related problems. Numer Math 76(2):168–188

    Article  MathSciNet  Google Scholar 

  7. Chan TF, Esedoglu S, Park F, Yip A (2006) Total variation image reconstruction: overview and recent developments. In: Paragios N, Chen Y, Faugeras OD (eds) Handbook of mathematical models in computer vision, chap 2. Springer, New York

    Google Scholar 

  8. Chen SS, Donoho DL, Saunders MA (1998) Atomic decomposition by basis pursuit. SIAM J Sci Comput 20(1):33–61

    Article  MathSciNet  Google Scholar 

  9. Chen X, Frossard P (2009) Joint reconstruction of compressed multi-view images. In: Proceedings of the international conference on acoustics, speech, and signal processing, pp 1005–1008. Taipei, Taiwan

  10. Duarte MF, Davenport MA, Takhar D, Laska JN, Sun T, Kelly KF, Baraniuk RG (2008) Single-pixel imaging via compressive sampling. IEEE Signal Process Mag 25(2):83–91

    Article  Google Scholar 

  11. Figueiredo MAT, Nowak RD, Wright SJ (2007) Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J Sel Areas Commun 1(4):586–597

    Google Scholar 

  12. Fowler JE, Mun S, Tramel EW (2011) Multiscale block compressed sensing with smoother projected Landweber reconstruction. In: Proceedings of the European signal processing conference, pp 564–568. Barcelona, Spain

  13. Fowler JE, Mun S, Tramel EW (2012) Block-based compressed sensing of images and video. Foundations and Trends in Signal Processing 4(4):297–416

    Article  Google Scholar 

  14. Gamper U, Boesiger P, Kozerke S (2008) Compressed sensing in dynamic MRI. Magn Reson Med 59(2):365–373

    Article  Google Scholar 

  15. Gan L (2007) Block compressed sensing of natural images. In: Proceedings of the international conference on digital signal processing, pp 403–406. Cardiff, UK

  16. Gan L, Do TT, Tran TD (2008) Fast compressive imaging using scrambled block Hadamard ensemble. In: Proceedings of the European signal processing conference. Lausanne, Switzerland

  17. Guillemot C, Pereira F, Torres L, Ebrahimi T, Leonardi R, Ostermann J (2007) Distributed monoview and multiview video coding. IEEE Signal Process Mag 24(5):67–76

    Article  Google Scholar 

  18. Jung H, Sung K, Nayak KS, Kim EY, Ye JC (2009) k-t FOCUSS: a general compressed sensing framework for high resolution dynamic MRI. Magn Reson Med 61(1):103–116

    Article  Google Scholar 

  19. Jung H, Ye JC (2010) Motion estimated and compensated compressed sensing dynamic magnetic resonance imaging: what we can learn from video compression techniques. Int J Imaging Syst Technol 20(2):81–98

    Article  Google Scholar 

  20. Kingsbury NG (2001) Complex wavelets for shift invariant analysis and filtering of signals. Appl Comput Harmon Anal 10:234–253

    Article  MATH  MathSciNet  Google Scholar 

  21. Li C (2009) An efficient algorithm for total variation regularization with applications to the single pixel camera and compressive sensing. Master’s thesis, Rice University

  22. Li X, Wei Z, Xiao L (2010) Compressed sensing joint reconstruction for multi-view images. Electron Lett 46(23):1548–1550

    Article  Google Scholar 

  23. Liu C (2009) Beyond pixels: exploring new representations and applications for motion analysis. Ph.D. thesis, Massachusetts Institute of Technology

  24. Lu W, Vaswani N (2009) Modified compressive sensing for real-time dynamic MR imaging. In: Proceedings of the international conference on image processing, pp 3045–3048. Cairo, Egypt

  25. Mun S, Fowler JE (2009) Block compressed sensing of images using directional transforms. In: Proceedings of the international conference on image processing, pp 3021–3024. Cairo, Egypt

  26. Park JY, Wakin MB (2012) A geometric approach to multi-view compressive imaging. EURASIP J Appl Signal Process 2012:37. doi:10.1186/1687-6180-2012-37

    Article  Google Scholar 

  27. Puri R, Majumdar A, Ishwar P, Ramchandran K (2006) Distributed video coding in wireless sensor networks. IEEE Signal Process Mag 23(4):94–106

    Article  Google Scholar 

  28. Qiu C, Lu W, Vaswani N (2009) Real-time dynamic MR image reconstruction using Kalman filtered compressed sensing. In: Proceedings of the international conference on acoustics, speech, and signal processing, pp 393–396. Taipei, Taiwan

  29. Rauhut H (2010) Compressive sensing and structured random matrices. In: Fornasier M (ed) Theoretical foundations and numerical methods for sparse recovery. Walter de Gruyter, Inc., Berlin

    Google Scholar 

  30. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica D 60(1–4):259–268

    Article  MATH  Google Scholar 

  31. Trocan M, Maugey T, Fowler JE, Pesquet-Popescu B (2010) Disparity-compensated compressed-sensing reconstruction for multiview images. In: Proceedings of the IEEE international conference on multimedia and expo, pp 1225–1229. Singapore

  32. Trocan M, Maugey T, Tramel EW, Fowler JE, Pesquet-Popescu B (2010) Compressed sensing of multiview images using disparity compensation. In: Proceedings of the international conference on image processing, pp 3345–3348. Hong Kong

  33. Trocan M, Maugey T, Tramel EW, Fowler JE, Pesquet-Popescu B (2010) Multistage compressed-sensing reconstruction of multiview images. In: Proceedings of the IEEE workshop on multimedia signal processing, pp 111–115. Saint-Malo, France

  34. Tropp J, Gilbert A (2007) Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans Inf Theory 53(12):4655–4666

    Article  MATH  MathSciNet  Google Scholar 

  35. Vaswani N (2008) Kalman filtered compressed sensing. In: Proceedings of the international conference on image processing, pp 893–896. San Diego, CA

  36. Vaswani N (2010) LS-CS-Residual (LS-CS): compressive sensing on least squares residual. IEEE Trans Signal Process 57(8):4108–4120

    Article  MathSciNet  Google Scholar 

  37. Vaswani N, Lu W (2010) Modified-CS: modifiying compressive sensing for problems with partially known support. IEEE Trans Signal Process 58(9):4595–4607

    Article  MathSciNet  Google Scholar 

  38. Wakin MB (2009) A manifold lifting algorithm for multi-view compressive imaging. In: Proceedings of the picture coding symposium. Chicago, IL

  39. Wakin MB, Laska JN, Duarte MF, Baron D, Sarvotham S, Takhar D, Kelly KF, Baraniuk RG (2006) An architecture for compressive imaging. In: Proceedings of the international conference on image processing, pp 1273–1276. Atlanta, GA

  40. Wakin MB, Laska JN, Duarte MF, Baron D, Sarvotham S, Takhar D, Kelly KF, Baraniuk RG (2006) Compressive imaging for video representation and coding. In: Proceedings of the picture coding symposium. Beijing, China

  41. Wang Y, Yang J, Yin W, Zhang Y (2008) A new alternating minimization algorithm for total variation image reconstruction. SIAM J Imaging Sci 1(3):248–272

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut Supérieur d’Électronique de Paris, 28 rue Notre Dame des Champs, 75006, Paris, France

    Maria Trocan

  2. Geosystems Research Institute, Mississippi State University, Box 9571, Mississippi State, MS, 39762, USA

    Eric W. Tramel & James E. Fowler

  3. Télécom ParisTech, 46 rue Barrault, 75634, Paris, Cédex 13, France

    Beatrice Pesquet

Authors
  1. Maria Trocan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric W. Tramel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James E. Fowler
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Beatrice Pesquet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maria Trocan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Trocan, M., Tramel, E.W., Fowler, J.E. et al. Compressed-sensing recovery of multiview image and video sequences using signal prediction. Multimed Tools Appl 72, 95–121 (2014). https://doi.org/10.1007/s11042-012-1330-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-012-1330-7

Keywords

Search

Navigation