Logic could be learned from images | International Journal of Machine Learning and Cybernetics Skip to main content
Springer Nature Link
Log in

Logic could be learned from images

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Logic reasoning is a significant ability of human intelligence and also an important task in artificial intelligence. The existing logic reasoning methods, quite often, need to design some reasoning patterns beforehand. This has led to an interesting question: can logic reasoning patterns be directly learned from given data? The problem is termed as a data concept logic. In this study, a learning logic task from images, called a LiLi task, first is proposed. This task is to learn and reason the logic relation from images, without presetting any reasoning patterns. As a preliminary exploration, we design six LiLi data sets (Bitwise And, Bitwise Or, Bitwise Xor, Addition, Subtraction and Multiplication), in which each image is embedded with a n-digit number. It is worth noting that a learning model beforehand does not know the meaning of the n-digit numbers embedded in images and the relation between the input images and the output image. In order to tackle the task, in this work we use many typical neural network models and produce fruitful results. However, these models have the poor performances on the difficult logic task. For furthermore addressing this task, a novel network framework called a divide and conquer model by adding some label information is designed, achieving a high testing accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Antol S, Agrawal A, Lu J, Mitchell M, Parikh D (2015) VQA: visual question answering. Int J Comput Vis 123(1):4–31

    MathSciNet  Google Scholar 

  2. Chen L, Huang P, Li Y, Meng Z (2020) Edge-dependent efficient grasp rectangle search in robotic grasp detection. IEEE/ASME Trans Mechatron. https://doi.org/10.1109/TMECH.2020.3048441

    Article  Google Scholar 

  3. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  4. Chen SM, Cheng SH, Chiou CH (2016) Fuzzy multiattribute group decision making based on intuitionistic fuzzy sets and evidential reasoning methodology. Inf Fusion 27:215–227

    Article  Google Scholar 

  5. Colom R, Karama S, Jung RE, Haier RJ (2010) Human intelligence and brain networks. Dialogues Clin Neurosci 12(4):489

    Article  Google Scholar 

  6. Dai WZ, Xu Q, Yu Y, Zhou ZH (2019) Bridging machine learning and logical reasoning by abductive learning. In: Advances in neural information processing systems. Vancouver, Canada

  7. Golinskapilarek J, Orlowska E (2007) Relational reasoning in formal concept analysis. In: IEEE international fuzzy systems conference. London, UK

  8. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems. Montréal, Canada, pp 2672–2680

  9. Graves A (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  10. Guo Q, Qian Y, Liang X (2019) Mining logic patterns from visual data. In: International conference on data mining workshops. Beijing, China

  11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition. Las Vegas, USA, pp 770–778

  12. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: IEEE international conference on computer vision. Venice, Italy, pp 2961–2969

  13. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  Google Scholar 

  14. Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366

    Article  Google Scholar 

  15. Hoshen Y, Peleg S (2016) Visual learning of arithmetic operation. In: Association for the advancement of artificial intelligence. Phoenix, USA, pp 3733–3739

  16. Hu R, Andreas J, Rohrbach M, Darrell T, Saenko K (2017) Learning to reason: end-to-end module networks for visual question answering. In: IEEE international conference on computer vision. Venice, Italy, pp 804–813

  17. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: IEEE conference on computer vision and pattern recognition. Honolulu, USA, pp 4700–4708

  18. Johnson J, Karpathy A, Fei-Fei L (2016) Densecap: fully convolutional localization networks for dense captioning. In: IEEE conference on computer vision and pattern recognition. Las Vegas, USA, pp 4565–4574

  19. Johnson J, Hariharan B, Maaten LVD, Li FF, Zitnick CL, Girshick R (2017) CLEVR: a diagnostic dataset for compositional language and elementary visual reasoning. In: IEEE conference on computer vision and pattern recognition. Honolulu, USA, pp 1988–1997

  20. Ke L, Zhang Q, Battiti R (2014) Hybridization of decomposition and local search for multiobjective optimization. IEEE Trans Cybern 44(10):1808–1820

    Article  Google Scholar 

  21. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  22. Li M, Chen M, Xu W (2019) Double-quantitative multigranulation decision-theoretic rough fuzzy set model. Int J Mach Learn Cybern 10(5):3225–3244

    Article  Google Scholar 

  23. Li SY, Tam LM, Chen HK, Chen CS (2020) A novel-designed fuzzy logic control structure for control of distinct chaotic systems. Int J Mach Learn Cybern 11:2391–2406

    Article  Google Scholar 

  24. Liang J, Fadili J, Peyré G (2016) A multi-step inertial forward-backward splitting method for non-convex optimization. In: Advances in neural information processing systems. Barcelona, Spain, pp 4035–4043

  25. Liang X, Guo Q, Qian Y, Ding W, Zhang Q (2021) Evolutionary deep fusion method and its application in chemical structure recognition. IEEE Trans Evol Comput. https://doi.org/10.1109/TEVC.2021.3064943

    Article  Google Scholar 

  26. Lin Y, Li J, Tan A, Zhang J (2020) Granular matrix-based knowledge reductions of formal fuzzy contexts. Int J Mach Learn Cybern 11:643–656

    Article  Google Scholar 

  27. Mizumoto M (1982) Comparison of fuzzy reasoning methods. Fuzzy Sets Syst 8(3):253–283

    Article  MathSciNet  Google Scholar 

  28. Nilsson NJ (1986) Probabilistic logic. Artif Intell 28(1):71–87

    Article  MathSciNet  Google Scholar 

  29. Nilsson NJ (1993) Probabilistic logic revisited. Artif Intell 59(1–2):39–42

    Article  MathSciNet  Google Scholar 

  30. Pearl J (1987) Evidential reasoning using stochastic simulation of causal models. Artif Intell 32(2):245–257

    Article  MathSciNet  Google Scholar 

  31. Pei DW (2004) On the strict logic foundation of fuzzy reasoning. Soft Comput 8(8):539–545

    Article  Google Scholar 

  32. Qian Y, Liang J, Pedrycz W, Dang C (2010a) Positive approximation: an accelerator for attribute reduction in rough set theory. Artif Intell 174:597–618

    Article  MathSciNet  Google Scholar 

  33. Qian Y, Liang J, Yao Y, Dang C (2010b) Mgrs: a multi-granulation rough set. Inf Sci 180(6):949–970

    Article  MathSciNet  Google Scholar 

  34. Qian Y, Liang X, Qi W, Liang J, Bing L, Skowron A, Yao Y, Ma J, Dang C (2018) Local rough set: a solution to rough data analysis in big data. Int J Approx Reason 97:38–63

    Article  MathSciNet  Google Scholar 

  35. Reed S, Akata Z, Yan X, Logeswaran L, Schiele B, Lee H (2016) Generative adversarial text to image synthesis. In: International conference on machine learning. New York, USA, pp 1060–1069

  36. Ren S, He K, Girshick R, Sun J (2017) Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  37. Shao MW, Lv MM, Li KW, Wang CZ (2020) The construction of attribute (object)-oriented multi-granularity concept lattices. Int J Mach Learn Cybern 11(4):1017–1032

    Article  Google Scholar 

  38. She Y, He X, Shi H, Qian Y (2017) A multiple-valued logic approach for multigranulation rough set model. Int J Approx Reason 82:270–284

    Article  MathSciNet  Google Scholar 

  39. She Y, He X, Qian Y, Xu W, Li J (2018) A quantitative approach to reasoning about incomplete knowledge. Inf Sci 451–452:100–111

    Article  MathSciNet  Google Scholar 

  40. Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651

    Article  Google Scholar 

  41. Smith R (2007) An overview of the tesseract ocr engine. In: Ninth international conference on document analysis and recognition, vol 2. Curitiba, Brazil, pp 629–633

  42. Tadrat J, Boonjing V, Pattaraintakorn P (2012) A new similarity measure in formal concept analysis for case-based reasoning. Expert Syst Appl 39(1):967–972

    Article  Google Scholar 

  43. Tan A, Wu WZ, Shia S, Zhao S (2019) Granulation selection and decision making with multigranulation rough set over two universes. Int J Mach Learn Cybern 10(9):2501–2513

    Article  Google Scholar 

  44. Tenenbaum JB, Griffiths TL, Kemp C (2006) Theory-based Bayesian models of inductive learning and reasoning. Trends Cogn Sci 10(7):309–318

    Article  Google Scholar 

  45. Vinyals O, Toshev A, Bengio S, Erhan D (2016) Show and tell: lessons learned from the 2015 mscoco image captioning challenge. IEEE Trans Pattern Anal Mach Intell 39(4):652–663

    Article  Google Scholar 

  46. Wang G (1996) Fuzzy reasoning and fuzzy logic. In: Soft computing in intelligent systems and information processing. Proceedings of the 1996 asian fuzzy systems symposium. Kenting, China, pp 478–483

  47. Wille R (1982) Restructuring lattice theory: an approach based on hierarchies of concepts. In: Rival I (ed) Ordered sets. Springer, pp 445–470

  48. Wu Q, Shen C, Wang P, Dick A, van den Hengel A (2018) Image captioning and visual question answering based on attributes and external knowledge. IEEE Trans Pattern Anal Mach Intell 40(6):1367–1381

    Article  Google Scholar 

  49. Yang Z, Bonsall S, Wang J (2008) Fuzzy rule-based Bayesian reasoning approach for prioritization of failures in FMEA. IEEE Trans Reliab 57(3):517–528

    Article  Google Scholar 

  50. Yang Z, He X, Gao J, Deng L, Smola A (2016) Stacked attention networks for image question answering. In: IEEE conference on computer vision and pattern recognition. Las Vegas, USA, pp 21–29

  51. Yen J (1999) Fuzzy logic-a modern perspective. IEEE Trans Knowl Data Eng 11(1):153–165

    Article  Google Scholar 

  52. Zadeh AL (1973) Outline of a new approach to the analysis of complex systems and decision processes. IEEE Trans Syst Man Cybern SMC 3(1):28–44

    Article  MathSciNet  Google Scholar 

  53. Zhang P, Goyal Y, Summers-Stay D, Batra D, Parikh D (2016) Yin and yang: balancing and answering binary visual questions. In: IEEE conference on computer vision and pattern recognition. Las Vegas, USA, pp 5014–5022

Download references

Acknowledgements

This work was supported by National Key R&D Program of China (no. 2018YFB1004300), National Natural Science Fund of China (nos. 61672332, 61432011, 61502289), Key R&D program (International Science and Technology Cooperation Project) of Shanxi Province, China (no. 201903D421003), Program for the Young San Jin Scholars of Shanxi (no. 2016769), Young Scientists Fund of the National Natural Science Foundation of China (nos. 61802238, 61906115, 61603228, 62006146, 61906114), Shanxi Province Science Foundation for Youths (no. 201901D211169, 201901D211170, 201901D211171), Research Project Supported by Shanxi Scholarship Council of China (no. HGKY2019001), and Scientific and Technological Innovation Programs of Higher Education Institutions in Shanxi (no. 2020L0036).

Author information

Authors and Affiliations

  1. Institute of Big Data Science and Industry, Shanxi University, Taiyuan, 030006, Shanxi, China

    Qian Guo, Yuhua Qian & Xinyan Liang

  2. Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University, Taiyuan, 030006, Shanxi, China

    Yuhua Qian, Deyu Li & Jiye Liang

  3. School of Computer and Information Technology, Shanxi University, Taiyuan, 030006, Shanxi, China

    Qian Guo, Yuhua Qian, Xinyan Liang, Deyu Li & Jiye Liang

  4. College of Science, Xi’an Shiyou University, Xi’an, 710065, Shanxi, China

    Yanhong She

Authors
  1. Qian Guo
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yuhua Qian
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Xinyan Liang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Yanhong She
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Deyu Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Jiye Liang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yuhua Qian.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, Q., Qian, Y., Liang, X. et al. Logic could be learned from images. Int. J. Mach. Learn. & Cyber. 12, 3397–3414 (2021). https://doi.org/10.1007/s13042-021-01366-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01366-w

Keywords