{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T17:00:58Z","timestamp":1726851658342},"reference-count":82,"publisher":"Wiley","issue":"2","license":[{"start":{"date-parts":[[2023,5,23]],"date-time":"2023-05-23T00:00:00Z","timestamp":1684800000000},"content-version":"vor","delay-in-days":22,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Computer Graphics Forum"],"published-print":{"date-parts":[[2023,5]]},"abstract":"Abstract<\/jats:title>We propose a method to estimate the mechanical parameters of fabrics using a casual capture setup with a depth camera. Our approach enables to create mechanically\u2010correct digital representations of real\u2010world textile materials, which is a fundamental step for many interactive design and engineering applications. As opposed to existing capture methods, which typically require expensive setups, video sequences, or manual intervention, our solution can capture at scale, is agnostic to the optical appearance of the textile, and facilitates fabric arrangement by non\u2010expert operators. To this end, we propose a sim\u2010to\u2010real strategy to train a learning\u2010based framework that can take as input one or multiple images and outputs a full set of mechanical parameters. Thanks to carefully designed data augmentation and transfer learning protocols, our solution generalizes to real images despite being trained only on synthetic data, hence successfully closing the sim\u2010to\u2010real loop. Key in our work is to demonstrate that evaluating the regression accuracy based on the similarity at parameter space leads to an inaccurate distances that do not match the human perception. To overcome this, we propose a novel metric for fabric drape similarity that operates on the image domain instead on the parameter space, allowing us to evaluate our estimation within the context of a similarity rank. We show that out metric correlates with human judgments about the perception of drape similarity, and that our model predictions produce perceptually accurate results compared to the ground truth parameters.<\/jats:p>","DOI":"10.1111\/cgf.14750","type":"journal-article","created":{"date-parts":[[2023,5,23]],"date-time":"2023-05-23T09:49:51Z","timestamp":1684835391000},"page":"149-160","update-policy":"http:\/\/dx.doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["How Will It Drape Like? Capturing Fabric Mechanics from Depth Images"],"prefix":"10.1111","volume":"42","author":[{"ORCID":"http:\/\/orcid.org\/0000-0001-6121-7738","authenticated-orcid":false,"given":"Carlos","family":"Rodriguez\u2010Pardo","sequence":"first","affiliation":[{"name":"SEDDI Madrid Spain"},{"name":"Universidad Rey Juan Carlos Madrid Spain"},{"name":"Universidad Carlos III de Madrid Spain"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-4053-3720","authenticated-orcid":false,"given":"Melania","family":"Prieto\u2010Martin","sequence":"additional","affiliation":[{"name":"Universidad Rey Juan Carlos Madrid Spain"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-3664-089X","authenticated-orcid":false,"given":"Dan","family":"Casas","sequence":"additional","affiliation":[{"name":"Universidad Rey Juan Carlos Madrid Spain"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-3509-8485","authenticated-orcid":false,"given":"Elena","family":"Garces","sequence":"additional","affiliation":[{"name":"SEDDI Madrid Spain"},{"name":"Universidad Rey Juan Carlos Madrid Spain"}]}],"member":"311","published-online":{"date-parts":[[2023,5,23]]},"reference":[{"key":"e_1_2_10_2_2","unstructured":"BommasaniR. HudsonD. A. AdeliE. AltmanR. AroraS. vonArxS. BernsteinM. S. BohgJ. BosselutA. BrunskillE. et al.:On the opportunities and risks of foundation models. 3"},{"key":"e_1_2_10_3_2","doi-asserted-by":"publisher","DOI":"10.1167\/18.5.12"},{"key":"e_1_2_10_4_2","volume-title":"Symposium on Computer Animation","author":"Bhat K. S.","year":"2003"},{"key":"e_1_2_10_5_2","unstructured":"BoumanK. L. XiaoB. BattagliaP. FreemanW. T.: Estimating the material properties of fabric from video. InProc. IEEE International Conference on Computer Vision(2013) pp.1984\u20131991. 2 3"},{"key":"e_1_2_10_6_2","first-page":"1597","volume-title":"International conference on machine learning","author":"Chen T.","year":"2020"},{"key":"e_1_2_10_7_2","doi-asserted-by":"publisher","DOI":"10.1177\/004051759006000906"},{"key":"e_1_2_10_8_2","doi-asserted-by":"crossref","unstructured":"ClydeD. TeranJ. TamstorfR.: Modeling and data-driven parameter estimation for woven fabrics. InProceedings of the ACM SIGGRAPH\/Eurographics Symposium on Computer Animation(2017) pp.1\u201311. 2","DOI":"10.1145\/3099564.3099577"},{"key":"e_1_2_10_9_2","doi-asserted-by":"crossref","unstructured":"ChenX. XieS. HeK.: An empirical study of training self-supervised vision transformers. InProceedings of the IEEE\/CVF International Conference on Computer Vision(2021) pp.9640\u20139649. 3","DOI":"10.1109\/ICCV48922.2021.00950"},{"key":"e_1_2_10_10_2","first-page":"83","volume-title":"The International Conference on Healthcare Science and Engineering","author":"Chen Y.","year":"2018"},{"key":"e_1_2_10_11_2","doi-asserted-by":"crossref","unstructured":"DavisA. BoumanK. L. ChenJ. G. RubinsteinM. DurandF. FreemanW. T.: Visual vibrometry: Estimating material properties from small motion in video. InProceedings of the ieee conference on computer vision and pattern recognition(2015) pp.5335\u20135343. 3","DOI":"10.1109\/CVPR.2015.7299171"},{"key":"e_1_2_10_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_2_10_13_2","doi-asserted-by":"crossref","unstructured":"DingK. MaK. WangS. SimoncelliE. P.: Image quality assessment: Unifying structure and texture similarity.IEEE transactions on pattern analysis and machine intelligence(2020). 3","DOI":"10.1109\/TPAMI.2020.3045810"},{"key":"e_1_2_10_14_2","volume-title":"31st British Machine Vision Conference 2020, BMVC 2020","author":"Fu R.","year":"2020"},{"key":"e_1_2_10_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3550454.3555464"},{"key":"e_1_2_10_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601131"},{"key":"e_1_2_10_17_2","doi-asserted-by":"crossref","unstructured":"GatysL. A. EckerA. S. BethgeM.: Image style transfer using convolutional neural networks. InProceedings of the IEEE conference on computer vision and pattern recognition(2016) pp.2414\u20132423. 3","DOI":"10.1109\/CVPR.2016.265"},{"key":"e_1_2_10_18_2","first-page":"62","volume-title":"Proceedings of the 2003 ACM SIGGRAPH\/Eurographics symposium on Computer animation","author":"Grinspun E.","year":"2003"},{"key":"e_1_2_10_19_2","doi-asserted-by":"crossref","unstructured":"GuoY. ShiH. KumarA. GraumanK. RosingT. FerisR.: Spottune: transfer learning through adaptive fine-tuning. InProceedings of the IEEE\/CVF conference on computer vision and pattern recognition(2019) pp.4805\u20134814. 3","DOI":"10.1109\/CVPR.2019.00494"},{"key":"e_1_2_10_20_2","unstructured":"HuY. AndersonL. LiT.-M. SunQ. CarrN. Ragan-KelleyJ. DurandF.: Difftaichi: Differentiable programming for physical simulation. InInternational Conference on Learning Representations(2019). 3"},{"key":"e_1_2_10_21_2","doi-asserted-by":"crossref","unstructured":"HeK. ChenX. XieS. LiY. Doll\u00e1rP. GirshickR.: Masked autoencoders are scalable vision learners. InProceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition(2022) pp.16000\u201316009. 3","DOI":"10.1109\/CVPR52688.2022.01553"},{"key":"e_1_2_10_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2017.4031068"},{"key":"e_1_2_10_23_2","doi-asserted-by":"crossref","unstructured":"HuJ. ShenL. SunG.: Squeeze-and-excitation networks. InProceedings of the IEEE conference on computer vision and pattern recognition(2018) pp.7132\u20137141. 5","DOI":"10.1109\/CVPR.2018.00745"},{"key":"e_1_2_10_24_2","unstructured":"HanK. WangY. ChenH. ChenX. GuoJ. LiuZ. TangY. XiaoA. XuC. XuY. et al.: A survey on vision transformer.IEEE transactions on pattern analysis and machine intelligence(2022). 5"},{"key":"e_1_2_10_25_2","doi-asserted-by":"crossref","unstructured":"HeK. ZhangX. RenS. SunJ.: Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition(2016) pp.770\u2013778. 5","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_10_26_2","unstructured":"JaquesM. BurkeM. HospedalesT.: Physics-as-inverse-graphics: Unsupervised physical parameter estimation from video. InInternational Conference on Learning Representations(2019). 3"},{"key":"e_1_2_10_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3033765"},{"key":"e_1_2_10_28_2","unstructured":"KawabataS.: The standardization and analysis of hand evaluation.The Textile Machinery Society of Japan(1980). 2"},{"key":"e_1_2_10_29_2","first-page":"491","volume-title":"European conference on computer vision","author":"Kolesnikov A.","year":"2020"},{"key":"e_1_2_10_30_2","unstructured":"KuijpersS. Luible-B\u00e4rC. GongR. H.: The measurement of fabric properties for virtual simulation\u2014a critical review.IEEE SA INDUSTRY CONNECTIONS(2020) 1\u201343. 2"},{"key":"e_1_2_10_31_2","unstructured":"KumarA. RaghunathanA. JonesR. MaT. LiangP.: Fine-tuning can distort pretrained features and underperform out-of-distribution.arXiv preprint arXiv:2202.10054(2022). 3"},{"key":"e_1_2_10_32_2","doi-asserted-by":"crossref","unstructured":"KornblithS. ShlensJ. LeQ. V.: Do better imagenet models transfer better? InProceedings of the IEEE\/CVF conference on computer vision and pattern recognition(2019) pp.2661\u20132671. 3","DOI":"10.1109\/CVPR.2019.00277"},{"key":"e_1_2_10_33_2","doi-asserted-by":"crossref","unstructured":"LiY. DuT. WuK. XuJ. MatusikW.: Diffcloth: Differentiable cloth simulation with dry frictional contact.ACM Transactions on Graphics (TOG)(2022). 3","DOI":"10.1145\/3527660"},{"key":"e_1_2_10_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-018-6628-7"},{"key":"e_1_2_10_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/2766929"},{"key":"e_1_2_10_36_2","doi-asserted-by":"crossref","unstructured":"LiW.-H. LiuX. BilenH.: Cross-domain few-shot learning with task-specific adapters. InProceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition(2022) pp.7161\u20137170. 3","DOI":"10.1109\/CVPR52688.2022.00702"},{"key":"e_1_2_10_37_2","article-title":"Differentiable cloth simulation for inverse problems","volume":"32","author":"Liang J.","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_10_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323036"},{"key":"e_1_2_10_39_2","unstructured":"LuibleC. Magnenat-ThalmannN.: The simulation of cloth using accurate physical parameters. InProceedings of the Tenth IASTED International Conference on Computer Graphics and Imaging (CGIM '08)(2008). 2"},{"key":"e_1_2_10_40_2","first-page":"519","volume-title":"Computer Graphics Forum","author":"Miguel E.","year":"2012"},{"key":"e_1_2_10_41_2","unstructured":"MinazioP. G.: Fast\u2013fabric assurance by simple testing.International Journal of Clothing Science and Technology(1995). 2"},{"key":"e_1_2_10_42_2","unstructured":"MurthyJ. K. MacklinM. GolemoF. VoletiV. PetriniL. WeissM. ConsidineB. Parent-L\u00e9vesqueJ. XieK. ErlebenK. et al.: gradsim: Differentiable simulation for system identification and visuomotor control. InInternational Conference on Learning Representations(2020). 3"},{"key":"e_1_2_10_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/CADCG.2007.4407832"},{"key":"e_1_2_10_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2016.2604042"},{"key":"e_1_2_10_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366171"},{"key":"e_1_2_10_46_2","doi-asserted-by":"crossref","unstructured":"PrashnaniE. CaiH. MostofiY. SenP.: Pieapp: Perceptual image-error assessment through pairwise preference. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition(2018) pp.1808\u20131817. 3","DOI":"10.1109\/CVPR.2018.00194"},{"key":"e_1_2_10_47_2","unstructured":"PhamM. Q. CregoJ.-M. YvonF. SenellartJ.: A study of residual adapters for multi-domain neural machine translation. InConference on Machine Translation(2020). 3"},{"key":"e_1_2_10_48_2","doi-asserted-by":"crossref","unstructured":"PowerJ.: Fabric objective measurements for commercial 3d virtual garment simulation.International Journal of Clothing Science and Technology(2013). 2","DOI":"10.1108\/IJCST-12-2012-0080"},{"key":"e_1_2_10_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.image.2017.11.001"},{"key":"e_1_2_10_50_2","article-title":"Learning multiple visual domains with residual adapters","volume":"30","author":"Rebuffi S.-A.","year":"2017","journal-title":"Advances in neural information processing systems"},{"key":"e_1_2_10_51_2","doi-asserted-by":"crossref","unstructured":"RebuffiS.-A. BilenH. VedaldiA.: Efficient parametrization of multi-domain deep neural networks. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition(2018) pp.8119\u20138127. 3","DOI":"10.1109\/CVPR.2018.00847"},{"key":"e_1_2_10_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/BTAS.2017.8272767"},{"key":"e_1_2_10_53_2","doi-asserted-by":"crossref","unstructured":"RuniaT. F. GavrilyukK. SnoekC. G. SmeuldersA. W.: Cloth in the wind: A case study of physical measurement through simulation. InProceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition(2020) pp.10498\u201310507. 3","DOI":"10.1109\/CVPR42600.2020.01051"},{"key":"e_1_2_10_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCSP.2019.8697909"},{"key":"e_1_2_10_55_2","first-page":"8748","volume-title":"International Conference on Machine Learning","author":"Radford A.","year":"2021"},{"key":"e_1_2_10_56_2","first-page":"508","volume-title":"Iberian Conference on Pattern Recognition and Image Analysis","author":"Rodr\u00edguez-Pardo C.","year":"2019"},{"key":"e_1_2_10_57_2","article-title":"Stand-alone self-attention in vision models","volume":"32","author":"Ramachandran P.","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_10_58_2","unstructured":"RasheedA. H. RomeroV. Bertails-DescoubesF. WuhrerS. FrancoJ.-S. LazarusA.: Learning to measure the static friction coefficient in cloth contact. InProceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition(2020) pp.9912\u20139921. 3"},{"key":"e_1_2_10_59_2","article-title":"Transfusion: Understanding transfer learning for medical imaging","volume":"32","author":"Raghu M.","year":"2019","journal-title":"Advances in neural information processing systems"},{"key":"e_1_2_10_60_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00371-008-0273-5"},{"key":"e_1_2_10_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530167"},{"key":"e_1_2_10_62_2","unstructured":"SimonyanK. ZissermanA.: Very deep convolutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556(2014). 5"},{"key":"e_1_2_10_63_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.185.4157.1124"},{"key":"e_1_2_10_64_2","unstructured":"TamuzO. LiuC. BelongieS. ShamirO. KalaiA. T.: Adaptively learning the crowd kernel. InProceedings of the 28th International Conference on International Conference on Machine Learning(2011) pp.673\u2013680. 7 8"},{"key":"e_1_2_10_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/1559755.1559762"},{"key":"e_1_2_10_66_2","article-title":"Attention is all you need","volume":"30","author":"Vaswani A.","year":"2017","journal-title":"Advances in neural information processing systems"},{"key":"e_1_2_10_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2003.819861"},{"key":"e_1_2_10_68_2","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-016-0043-6"},{"key":"e_1_2_10_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2018.2791721"},{"issue":"4","key":"e_1_2_10_70_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2010324.1964966","article-title":"Data-driven elastic models for cloth: modeling and measurement","volume":"30","author":"Wang H.","year":"2011","journal-title":"ACM transactions on graphics (TOG)"},{"key":"e_1_2_10_71_2","doi-asserted-by":"crossref","unstructured":"WooS. ParkJ. LeeJ.-Y. KweonI. S.: Cbam: Convolutional block attention module. InProceedings of the European conference on computer vision (ECCV)(2018) pp.3\u201319. 5","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"e_1_2_10_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2015.2505285"},{"key":"e_1_2_10_73_2","doi-asserted-by":"crossref","unstructured":"YangS. LiangJ. LinM. C.: Learning-based cloth material recovery from video. InProceedings of the IEEE International Conference on Computer Vision(2017) pp.4383\u20134393. 2 3","DOI":"10.1109\/ICCV.2017.470"},{"key":"e_1_2_10_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/3026479"},{"key":"e_1_2_10_75_2","first-page":"808","volume-title":"European Conference on Computer Vision","author":"Zhang H.","year":"2016"},{"key":"e_1_2_10_76_2","first-page":"7354","volume-title":"International conference on machine learning","author":"Zhang H.","year":"2019"},{"key":"e_1_2_10_77_2","unstructured":"ZhuW. GuoX. OwakiD. KutsuzawaK. HayashibeM.: A survey of sim-to-real transfer techniques applied to reinforcement learning for bioinspired robots.IEEE Transactions on Neural Networks and Learning Systems(2021). 2"},{"key":"e_1_2_10_78_2","unstructured":"ZhangR. IsolaP. EfrosA. A. ShechtmanE. WangO.: The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recognition(2018) pp.586\u2013595. 2 3 7"},{"key":"e_1_2_10_79_2","doi-asserted-by":"crossref","unstructured":"ZhaoH. JiaJ. KoltunV.: Exploring self-attention for image recognition. InProceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition(2020) pp.10076\u201310085. 5","DOI":"10.1109\/CVPR42600.2020.01009"},{"key":"e_1_2_10_80_2","doi-asserted-by":"crossref","unstructured":"ZhouB. KhoslaA. LapedrizaA. OlivaA. TorralbaA.: Learning deep features for discriminative localization. InProceedings of the IEEE conference on computer vision and pattern recognition(2016) pp.2921\u20132929. 5","DOI":"10.1109\/CVPR.2016.319"},{"key":"e_1_2_10_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2012.6467149"},{"key":"e_1_2_10_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2014.2346028"},{"key":"e_1_2_10_83_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2011.2109730"}],"container-title":["Computer Graphics Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/cgf.14750","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,19]],"date-time":"2023-08-19T20:40:10Z","timestamp":1692477610000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/cgf.14750"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5]]},"references-count":82,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,5]]}},"alternative-id":["10.1111\/cgf.14750"],"URL":"https:\/\/doi.org\/10.1111\/cgf.14750","archive":["Portico"],"relation":{},"ISSN":["0167-7055","1467-8659"],"issn-type":[{"value":"0167-7055","type":"print"},{"value":"1467-8659","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5]]},"assertion":[{"value":"2023-05-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}