Abstract
Well-designed indoor scenes incorporate interior design knowledge, which has been an essential prior for most indoor scene modeling methods. However, the layout qualities of indoor scene datasets are often uneven, and most existing data-driven methods do not differentiate indoor scene examples in terms of quality. In this work, we aim to explore an approach that leverages datasets with differentiated indoor scene examples for indoor scene modeling. Our solution conducts subjective evaluations on lightweight datasets having various room configurations and furniture layouts, via pairwise comparisons based on fuzzy set theory. We also develop a system to use such examples to guide indoor scene modeling using user-specified objects. Specifically, we focus on object groups associated with certain human activities, and define room features to encode the relations between the position and direction of an object group and the room configuration. To perform indoor scene modeling, given an empty room, our system first assesses it in terms of the user-specified object groups, and then places associated objects in the room guided by the assessment results. A series of experimental results and comparisons to state-of-the-art indoor scene synthesis methods are presented to validate the usefulness and effectiveness of our approach.
Article PDF
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Yu, L. F.; Yeung, S. K.; Tang, C. K.; Terzopoulos, D.; Chan, T. F.; Osher, S. J. Make it home: Automatic optimization of furniture arrangement. ACM Transactions on Graphics Vol. 30, No. 4, Article No. 86, 2011.
Merrell, P.; Schkufza, E.; Li, Z. Y.; Agrawala, M.; Koltun, V. Interactive furniture layout using interior design guidelines. In: Proceedings of the ACM SIGGRAPH 2011 Papers, Article No. 87, 2011.
Chen, X. W.; Li, J. W.; Li, Q.; Gao, B.; Zou, D. Q.; Zhao, Q. P. Image2Scene: Transforming style of 3D room. In: Proceedings of the 23rd ACM International Conference on Multimedia, 321–330, 2015.
Fisher, M.; Ritchie, D.; Savva, M.; Funkhouser, T.; Hanrahan, P. Example-based synthesis of 3D object arrangements. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 135, 2012.
Jiang, Y.; Lim, M.; Saxena, A. Learning object arrangements in 3D scenes using human context. In: Proceedings of the 29th International Conference on International Conference on Machine Learning, 907–914, 2012.
Fisher, M.; Savva, M.; Li, Y. Y.; Hanrahan, P.; Nießner, M. Activity-centric scene synthesis for functional 3D scene modeling. ACM Transactions on Graphics Vol. 34, No. 6, Article No. 179, 2015.
Savva, M.; Chang, A. X.; Hanrahan, P.; Fisher, M.; Nießner, M. PiGraphs: Learning interaction snapshots from observations. ACM Transactions on Graphics Vol. 35, No. 4, Article No. 139, 2016.
Fu, Q. A.; Chen, X. W.; Wang, X. T.; Wen, S. J.; Zhou, B.; Fu, H. B. Adaptive synthesis of indoor scenes via activity-associated object relation graphs. ACM Transactions on Graphics Vol. 36, No. 6, Article No. 201, 2017.
Qi, S. Y.; Zhu, Y. X.; Huang, S. Y.; Jiang, C.; Zhu, S. C. Human-centric indoor scene synthesis using stochastic grammar. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5899–5908, 2018.
Wang, K.; Savva, M.; Chang, A. X.; Ritchie, D. Deep convolutional priors for indoor scene synthesis. ACM Transactions on Graphics Vol. 37, No. 4, Article No. 70, 2018.
Wang, K.; Lin, Y.; Weissmann, B.; Savva, M.; Chang, A. X.; Ritchie, D. PlanIT: Planning and instantiating indoor scenes with relation graph and spatial prior networks. ACM Transactions on Graphics Vol. 38, No. 4, Article No. 132, 2019.
Saaty, T. L. The Analytic Hierarchy Process. New York: McGraw-Hill, 1980.
Klir, G. J.; Yuan, B. Fuzzy Sets and Fuzzy Logic: Theory and Applications. Prentice Hall, 1995.
Li, M. Y.; Patil, A.; Xu, K.; Chaudhuri, S.; Khan, O.; Shamir, A.; Tu, C. H.; Chen, B. Q.; Cohen-Or, D.; Zhang, H. GRAINS: Generative recursive autoencoders for INdoor scenes. ACM Transactions on Graphics Vol. 38, No. 2, Article No. 12, 2019.
Fisher, M.; Hanrahan, P. Context-based search for 3D models. ACM Transactions on Graphics Vol. 29, No. 6, Article No. 182, 2010.
Fisher, M.; Savva, M.; Hanrahan, P. Characterizing structural relationships in scenes using graph kernels. ACM Transactions on Graphics Vol. 30, No. 4, Article No. 34, 2011.
Sharf, A.; Huang, H.; Liang, C.; Zhang, J. P.; Chen, B. Q.; Gong, M. L. Mobility-trees for indoor scenes manipulation. Computer Graphics Forum Vol. 33, No. 1, 2–14, 2014.
Xu, K.; Ma, R.; Zhang, H.; Zhu, C. Y.; Shamir, A.; Cohen-Or, D.; Huang, H. Organizing heterogeneous scene collections through contextual focal points. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 35, 2014.
Liu, T. Q.; Chaudhuri, S.; Kim, V. G.; Huang, Q. X.; Mitra, N. J.; Funkhouser, T. Creating consistent scene graphs using a probabilistic grammar. ACM Transactions on Graphics Vol. 33, No. 6, Article No. 211, 2014.
Zhang, S. H.; Zhang, S. K.; Xie, W. Y.; Luo, C. Y.; Yang, Y. L.; Fu, H. B. Fast 3D indoor scene synthesis by learning spatial relation priors of objects. IEEE Transactions on Visualization and Computer Graphics Vol. 28, No. 9, 3082–3092, 2022.
Ma, R.; Li, H. H.; Zou, C. Q.; Liao, Z. C.; Tong, X.; Zhang, H. Action-driven 3D indoor scene evolution. ACM Transactions on Graphics Vol. 35, No. 6, Article No. 173, 2016.
Ma, R.; Patil, A. G.; Fisher, M.; Li, M. Y.; Pirk, S.; Hua, B. S.; Yeung, S. K.; Tong, X.; Guibas, L.; Zhang, H. Language-driven synthesis of 3D scenes from scene databases. ACM Transactions on Graphics Vol. 37, No. 6, Article No. 212, 2018.
Xu, K.; Chen, K.; Fu, H.; Sun, W. L.; Hu, S. M. Sketch2Scene: Sketch-based co-retrieval and co-placement of 3D models. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 123, 2013.
Chen, K.; Lai, Y. K.; Wu, Y. X.; Martin, R.; Hu, S. M. Automatic semantic modeling of indoor scenes from low-quality RGB-D data using contextual information. ACM Transactions on Graphics Vol. 33, No. 6, Article No. 208, 2014.
Shao, T.; Monszpart, A.; Zheng, Y.; Koo, B.; Xu, W.; Zhou, K.; Mitra, N. J. Imagining the unseen: Stability-based cuboid arrangements for scene understanding. ACM Transactions on Graphics Vol. 33, No. 6, Article No. 209, 2014.
Zhang, S. K.; Li, Y. X.; He, Y.; Yang, Y. L.; Zhang, S. H. MageAdd: Real-time interaction simulation for scene synthesis. In: Proceedings of the 29th ACM International Conference on Multimedia, 965–973, 2021.
Frontczak, M.; Wargocki, P. Literature survey on how different factors influence human comfort in indoor environments. Building and Environment Vol. 46, No. 4, 922–937, 2011.
Huang, L.; Zhu, Y. X.; Ouyang, Q.; Cao, B. A study on the effects of thermal, luminous, and acoustic environments on indoor environmental comfort in offices. Building and Environment Vol. 49, 304–309, 2012.
Konis, K. Predicting visual comfort in side-lit open-plan core zones: Results of a field study pairing high dynamic range images with subjective responses. Energy and Buildings Vol. 77, 67–79, 2014.
Ochoa, C. E.; Capeluto, I. G. Evaluating visual comfort and performance of three natural lighting systems for deep office buildings in highly luminous climates. Building and Environment Vol. 41, No. 8, 1128–1135, 2006.
Zhang, Z. W.; Yang, Z. P.; Ma, C. Y.; Luo, L. J.; Huth, A.; Vouga, E.; Huang, Q. X. Deep generative modeling for scene synthesis via hybrid representations. ACM Transactions on Graphics Vol. 39, No. 2, Article No. 17, 2020.
3D Warehouse. Available at https://3dwarehouse.sketchup.com/.
Song, S. R.; Yu, F.; Zeng, A.; Chang, A. X.; Savva, M.; Funkhouser, T. Semantic scene completion from a single depth image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 190–198, 2017.
Acknowledgements
We thank the anonymous reviewers for their constructive comments. This work was partially supported by grants from the National Natural Science Foundation of China (61902032), Research Grants Council of the Hong Kong Special Administrative Region, China (CityU 11237116), and City University of Hong Kong (7004915).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
The authors have no competing interests to declare that are relevant to the content of this article.
Additional information
Qiang Fu is an associate professor in the School of Digital Media and Design Arts, Beijing University of Posts and Telecommunications. He received his Ph.D. degree in computer science from Beihang University, China, in 2018 and his B.E. degree in automation from Beihang University in 2011. His research interests include computer graphics and virtual reality.
Shuhan He received her B.E. degreee in digital media technology from Beijing Forestry University. She is a postgraduate student in the School of Digital Media and Design Arts, Beijing University of Posts and Telecommunications. Her research interests include computer graphics and machine learning.
Hongbo Fu is a professor in the School of Creative Media, City University of Hong Kong. He received his Ph.D. degree in computer science from Hong Kong University of Science and Technology in 2007 and his B.S. degree in information sciences from Peking University, China, in 2002. His primary research interests fall in the fields of computer graphics and human computer interaction. He has served as an associate editor of The Visual Computer, Computers & Graphics, and Computer Graphics Forum.
Xueming Li received his B.E. degree in electronics engineering from the University of Science and Technology of China, in 1992, and his Ph.D. degree in electronics engineering from Beijing University of Posts and Telecommunications in 1997. He is a professor in the School of Digital Media and Design Arts, Beijing University of Posts and Telecommunications. His research interests include digital image processing, video coding, and multimedia telecommunication.
Zhigang Deng received his B.S. degree in mathematics from Xiamen University, China, his M.S. degree in computer science from Peking University, and his Ph.D. degree in computer science from the Department of Computer Science, University of Southern California, in 2006. He is Moores Professor of computer science with the University of Houston. His research interests include computer graphics, computer animation, and HCI.
Rights and permissions
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Fu, Q., He, S., Fu, H. et al. Fuzzy-based indoor scene modeling with differentiated examples. Comp. Visual Media 9, 717–732 (2023). https://doi.org/10.1007/s41095-022-0299-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-022-0299-z