{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,7,26]],"date-time":"2024-07-26T15:55:57Z","timestamp":1722009357809},"reference-count":56,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2018,7,31]],"date-time":"2018-07-31T00:00:00Z","timestamp":1532995200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Zhejiang Provincial Public Fund","award":["2016C33136"]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"Localization systems play an important role in assisted navigation. Precise localization renders visually impaired people aware of ambient environments and prevents them from coming across potential hazards. The majority of visual localization algorithms, which are applied to autonomous vehicles, are not adaptable completely to the scenarios of assisted navigation. Those vehicle-based approaches are vulnerable to viewpoint, appearance and route changes (between database and query images) caused by wearable cameras of assistive devices. Facing these practical challenges, we propose Visual Localizer, which is composed of ConvNet descriptor and global optimization, to achieve robust visual localization for assisted navigation. The performance of five prevailing ConvNets are comprehensively compared, and GoogLeNet is found to feature the best performance on environmental invariance. By concatenating two compressed convolutional layers of GoogLeNet, we use only thousands of bytes to represent image efficiently. To further improve the robustness of image matching, we utilize the network flow model as a global optimization of image matching. The extensive experiments using images captured by visually impaired volunteers illustrate that the system performs well in the context of assisted navigation.<\/jats:p>","DOI":"10.3390\/s18082476","type":"journal-article","created":{"date-parts":[[2018,8,1]],"date-time":"2018-08-01T07:10:01Z","timestamp":1533107401000},"page":"2476","source":"Crossref","is-referenced-by-count":34,"title":["Visual Localizer: Outdoor Localization Based on ConvNet Descriptor and Global Optimization for Visually Impaired Pedestrians"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"http:\/\/orcid.org\/0000-0003-4911-9443","authenticated-orcid":false,"given":"Shufei","family":"Lin","sequence":"first","affiliation":[{"name":"State Key Laboratory of Modern Optical Instrumentation, Zhejiang University, Hangzhou 310027, China"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-7951-196X","authenticated-orcid":false,"given":"Ruiqi","family":"Cheng","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Modern Optical Instrumentation, Zhejiang University, Hangzhou 310027, China"}]},{"given":"Kaiwei","family":"Wang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Modern Optical Instrumentation, Zhejiang University, Hangzhou 310027, China"}]},{"given":"Kailun","family":"Yang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Modern Optical Instrumentation, Zhejiang University, Hangzhou 310027, China"}]}],"member":"1968","published-online":{"date-parts":[[2018,7,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"e888","DOI":"10.1016\/S2214-109X(17)30293-0","article-title":"Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis","volume":"5","author":"Bourne","year":"2017","journal-title":"Lancet Glob. Health"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Brilhault, A., Kammoun, S., Gutierrez, O., Truillet, P., and Jouffrais, C. (2011, January 7\u201310). Fusion of Artificial Vision and GPS to Improve Blind Pedestrian Positioning. Proceedings of the 2011 4th IFIP International Conference on New Technologies, Mobility and Security, Paris, France.","DOI":"10.1109\/NTMS.2011.5721061"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TRO.2015.2496823","article-title":"Visual Place Recognition: A Survey","volume":"32","author":"Lowry","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Pepperell, E., Corke, P.I., and Milford, M.J. (2015, January 26\u201330). Automatic image scaling for place recognition in changing environments. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139316"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Kanji, T. (2, January 28). Cross-season place recognition using NBNN scene descriptor. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7353453"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Garg, S., Suenderhauf, N., and Milford, M. (arXiv, 2018). LoST? Appearance-Invariant Place Recognition for Opposite Viewpoints Using Visual Semantics, arXiv.","DOI":"10.15607\/RSS.2018.XIV.022"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"665","DOI":"10.1007\/s10514-017-9664-7","article-title":"Are you ABLE to perform a life-long visual topological localization?","volume":"42","author":"Arroyo","year":"2018","journal-title":"Auton. Robots"},{"key":"ref_8","unstructured":"Vysotska, O., and Stachniss, C. (2017, January 24). Relocalization under Substantial Appearance Changes Using Hashing. Proceedings of the IROS Workshop on Planning, Perception and Navigation for Intelligent Vehicles, Vancouver, BC, Canada."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Vysotska, O., Naseer, T., Spinello, L., Burgard, W., and Stachniss, C. (2015, January 25\u201330). Efficient and effective matching of image sequences under substantial appearance changes exploiting GPS priors. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139576"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Kim, J.E., Bessho, M., Kobayashi, S., Koshizuka, N., and Sakamura, K. (2016, January 4\u20138). Navigating visually impaired travelers in a large train station using smartphone and bluetooth low energy. Proceedings of the 31st Annual ACM Symposium on Applied Computing (SAC \u201916), Pisa, Italy.","DOI":"10.1145\/2851613.2851716"},{"key":"ref_11","unstructured":"Tang, X., Chen, Y., Zhu, Z., and Lu, X. (2011, January 26\u201328). A visual aid system for the blind based on RFID and fast symbol recognition. Proceedings of the 2011 6th International Conference on Pervasive Computing and Applications, Port Elizabeth, South Africa."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"449","DOI":"10.3233\/AIS-170441","article-title":"An approach for developing indoor navigation systems for visually impaired people using Building Information Modeling","volume":"9","author":"Ivanov","year":"2017","journal-title":"J. Ambient Intell. Smart Environ."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1016\/j.compeleceng.2016.07.015","article-title":"Ebsar: Indoor guidance for the visually impaired","volume":"54","year":"2016","journal-title":"Comput. Electr. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"448","DOI":"10.1007\/978-3-319-48881-3_31","article-title":"ISANA: Wearable Context-Aware Indoor Assistive Navigation with Obstacle Avoidance for the Blind","volume":"Volume 9914","author":"Hua","year":"2016","journal-title":"Lecture Notes in Computer Science, ECCV Workshop"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Murata, M., Ahmetovic, D., Sato, D., Takagi, H., Kitani, K.M., and Asakawa, C. (2018, January 19\u201323). Smartphone-based Indoor Localization for Blind Navigation across Building Complexes. Proceedings of the IEEE International Conference on Pervasive Computing and Communications (PerCom), Athens, Greece.","DOI":"10.1109\/PERCOM.2018.8444593"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1246","DOI":"10.1109\/TCSVT.2014.2372371","article-title":"A Compressive Sensing Approach to Describe Indoor Scenes for Blind People","volume":"25","author":"Mekhalfi","year":"2015","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Nguyen, Q.H., and Tran, T.H. (2013, January 16\u201318). Scene description for visually impaired in outdoor environment. Proceedings of the 2013 International Conference on Advanced Technologies for Communications (ATC 2013), Ho Chi Minh City, Vietnam.","DOI":"10.1109\/ATC.2013.6698144"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Fusco, G., Shen, H., and Coughlan, J.M. (2014, January 6\u20139). Self-Localization at Street Intersections. Proceedings of the 2014 Canadian Conference on Computer and Robot Vision, Montreal, QC, Canada.","DOI":"10.1109\/CRV.2014.14"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1309","DOI":"10.1109\/TRO.2016.2624754","article-title":"Simultaneous Localization and Mapping: Present, Future, and the Robust-Perception Age","volume":"32","author":"Cadena","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1109\/TRO.2017.2705103","article-title":"ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras","volume":"33","author":"Tardos","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1188","DOI":"10.1109\/TRO.2012.2197158","article-title":"Bags of Binary Words for Fast Place Recognition in Image Sequences","volume":"28","author":"Tardos","year":"2012","journal-title":"IEEE Trans. Robot."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Arroyo, R., Alcantarilla, P.F., Bergasa, L.M., and Romera, E. (2016, January 1\u20134). OpenABLE: An open-source toolbox for application in life-long visual localization of autonomous vehicles. Proceedings of the 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), Rio de Janeiro, Brazil.","DOI":"10.1109\/ITSC.2016.7795672"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Naseer, T., Spinello, L., Burgard, W., and Stachniss, C. (2014, January 27\u201331). Robust Visual Robot Localization Across Seasons Using Network Flows. Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI\u201914), Quebec City, QC, Canada.","DOI":"10.1609\/aaai.v28i1.9057"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Kameda, Y., and Ohta, Y. (2010, January 23\u201326). Image Retrieval of First-Person Vision for Pedestrian Navigation in Urban Area. Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey.","DOI":"10.1109\/ICPR.2010.1140"},{"key":"ref_25","unstructured":"Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems 25, Curran Associates, Inc."},{"key":"ref_26","unstructured":"Simonyan, K., and Zisserman, A. (arXiv, 2014). Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_29","unstructured":"Iandola, F.N., Moskewicz, M.W., Ashraf, K., Han, S., Dally, W.J., and Keutzer, K. (arXiv, 2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size, arXiv."},{"key":"ref_30","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (arXiv, 2017). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications, arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhang, X., Zhou, X., Lin, M., and Sun, J. (arXiv, 2017). ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices, arXiv.","DOI":"10.1109\/CVPR.2018.00716"},{"key":"ref_32","unstructured":"S\u00fcnderhauf, N., Shirazi, S., Dayoub, F., Upcroft, B., and Milford, M. (October, January 28). On the performance of ConvNet features for place recognition. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Hamburg, Germany."},{"key":"ref_33","unstructured":"Chen, Z., Lam, O., Jacobson, A., and Milford, M. (arXiv, 2014). Convolutional Neural Network-Based Place Recognition, arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Suenderhauf, N., Shirazi, S., Jacobson, A., Dayoub, F., Pepperell, E., Upcroft, B., and Milford, M. (2015, January 13\u201317). Place Recognition with ConvNet Landmarks: Viewpoint-Robust, Condition-Robust, Training-Free. Proceedings of the Robotics Science Systems XI, Rome, Italy.","DOI":"10.15607\/RSS.2015.XI.022"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Zitnick, C.L., and Doll, P. (2014, January 6\u201312). Edge Boxes:Locating Object Proposals from Edges. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1_26"},{"key":"ref_36","first-page":"8104386","article-title":"Efficient ConvNet Feature Extraction with Multiple RoI Pooling for Landmark-Based Visual Localization of Autonomous Vehicles","volume":"2017","author":"Hou","year":"2017","journal-title":"Mob. Inf. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Kendall, A., Grimes, M., and Cipolla, R. (2015, January 7\u201313). PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.336"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., and Sivic, J. (2016, January 27\u201330). NetVLAD: CNN Architecture for Weakly Supervised Place Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.572"},{"key":"ref_39","unstructured":"and Ohta, N. (2013, January 5\u20138). Performance Evaluation of Image Feature Detectors and Descriptors for Outdoor-Scene Visual Navigation. Proceedings of the 2013 2nd IAPR Asian Conference on Pattern Recognition, Naha, Japan."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Milford, M.J., and Wyeth, G.F. (2012, January 14\u201318). SeqSLAM: Visual route-based navigation for sunny summer days and stormy winter nights. Proceedings of the 2012 IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6224623"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1452","DOI":"10.1109\/TPAMI.2017.2723009","article-title":"Places: A 10 million Image Database for Scene Recognition","volume":"40","author":"Zhou","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (arXiv, 2018). Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation, arXiv.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"430","DOI":"10.1287\/moor.15.3.430","article-title":"Finding Minimum-Cost Circulations by Successive Approximation","volume":"15","author":"Goldberg","year":"1990","journal-title":"Math. Oper. Res."},{"key":"ref_45","unstructured":"(2018, May 14). Google Optimization Tools. Available online: https:\/\/developers.google.com\/optimization\/."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.robot.2014.08.005","article-title":"Superpixel-based appearance change prediction for long-term navigation across seasons","volume":"69","author":"Neubert","year":"2015","journal-title":"Robot. Auton. Syst."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Arroyo, R., Alcantarilla, P.F., Bergasa, L.M., and Romera, E. (2016, January 9\u201314). Fusion and binarization of CNN features for robust topological localization across seasons. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759685"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1186\/s13040-017-0155-3","article-title":"Ten quick tips for machine learning in computational biology","volume":"10","author":"Chicco","year":"2017","journal-title":"BioData Min."},{"key":"ref_49","unstructured":"KrVision (2018, July 28). Intoer: Auxiliary Glasses for People with Visual Impairments. (In Chinese)."},{"key":"ref_50","unstructured":"Intel (2018, July 28). RealSense Camera ZR300. Available online: https:\/\/www.intel.com."},{"key":"ref_51","unstructured":"AfterShokz (2018, July 28). Bone-Conduction Earphone. Available online: https:\/\/aftershokz.co.uk."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Yang, K., Wang, K., Hu, W., and Bai, J. (2016). Expanding the Detection of Traversable Area with RealSense for the Visually Impaired. Sensors, 16.","DOI":"10.3390\/s16111954"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Yang, K., Wang, K., Cheng, R., Hu, W., Huang, X., and Bai, J. (2017). Detecting Traversable Area and Water Hazards for the Visually Impaired with a pRGB-D Sensor. Sensors, 17.","DOI":"10.3390\/s17081890"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Yang, K., Wang, K., Bergasa, L.M., Romera, E., Hu, W., Sun, D., Sun, J., Cheng, R., Chen, T., and L\u00f3pez, E. (2018). Unifying Terrain Awareness for the Visually Impaired through Real-Time Semantic Segmentation. Sensors, 18.","DOI":"10.3390\/s18051506"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"053025","DOI":"10.1117\/1.JEI.26.5.053025","article-title":"Crosswalk navigation for people with visual impairments on a wearable device","volume":"26","author":"Cheng","year":"2017","journal-title":"J. Electron. Imaging"},{"key":"ref_56","first-page":"1","article-title":"Real-time pedestrian crossing lights detection algorithm for the visually impaired","volume":"1\u201321","author":"Cheng","year":"2017","journal-title":"Multimedia Tools Appl."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/8\/2476\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,11]],"date-time":"2024-06-11T22:58:24Z","timestamp":1718146704000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/8\/2476"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,31]]},"references-count":56,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2018,8]]}},"alternative-id":["s18082476"],"URL":"https:\/\/doi.org\/10.3390\/s18082476","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,31]]}}}