Abstract
Contemporary industrial and production systems produce huge amounts of data in various models, used for process monitoring, predictive maintenance of the machines, historical analysis and statistics, and more. Apache Hadoop brings a cost-effective opportunity for Big Data analysis, including the data generated in various industries. Integrating Hadoop into industrial environments creates new possibilities, as well as many challenges. The authors of this paper are involved into commercial and scientific projects utilizing Hadoop for industry as predictive analytics platform. In such initiatives the lack of standardization of monitoring of the industrial process in terms of Hadoop cluster utilization is especially perplexing. In this paper, authors propose the methodology of monitoring Hadoop in industrial environments, based on dedicated ontology and widely adopted standards: OPC Unified Architecture and RESTful Web Services.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Gartner. Big Data definition. Gartner IT glossary. http://www.gartner.com/it-glossary/big-data/. Accessed 28 Jan 2015
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST). IEEE (2010)
Bakshi, K.: Considerations for big data: architecture and approach. In: Aerospace Conference, 2012. IEEE (2012)
White, T.: Hadoop: The Definitive Guide. O’Reilly Media Inc, Sebastopol (2012)
Bahga, A., Madisetti, V.K.: Analyzing massive machine maintenance data in a computing cloud. IEEE Trans. Parallel Distrib. Syst. 23(10), 1831–1843 (2012)
Kiss, I., Genge, B., Haller, P., Sebestyen, G.: Data clustering-based anomaly detection in industrial control systems. In: 2014 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP). IEEE (2014)
Scholten, B.: The Road to Integration: A Guide to Applying the ISA-95 Standard in Manufacturing. Isa, USA (2007)
EMC2-Factory. http://www.emc2-factory.eu. Accessed 28 Jan 2015
Factories of the Future. http://ec.europa.eu/research/industrial_technologies/factories-of-the-future_en.html. Accessed 28 Jan 2015
Cupek, R., Drewniak, M., Zonenberg, D.: Online energy efficiency assessment in serial production-statistical and data mining approaches. In: 2014 IEEE 23rd International Symposium on Industrial Electronics (ISIE). IEEE (2014)
Cupek, R., Fojcik, M., Sande, O.: Object oriented vertical communication in distributed industrial systems. In: Kwiecień, A., Gaj, P., Stera, P. (eds.) CN 2009. CCIS, vol. 39, pp. 72–78. Springer, Heidelberg (2009)
Haaland Thorsen, K.A., Rong, C.: Towards dataintegration from WITSML to ISO 15926. In: Sandnes, F.E., Zhang, Y., Rong, C., Yang, L.T., Ma, J. (eds.) UIC 2008. LNCS, vol. 5061, pp. 626–635. Springer, Heidelberg (2008)
King, R.L.: Information services for smart grids. In: Power and Energy Society General Meeting-Conversion and Delivery of Electrical Energy in the 21st Century, pp. 1–5. IEEE (2008)
Van Deursen, D., Poppe, C., Martens, G., Mannens, E., Walle, R.: XML to RDF conversion: a generic approach. In: 2008 International Conference on Automated Solutions for Cross Media Content and Multi-channel Distribution, AXMEDIS 2008. IEEE (2008)
Stopper, M., Katalinic, B.: Service-oriented architecture design aspects of OPC UA for industrial applications. In: Proceedings of the International Multi-Conference of Engineers and Computer Scientists (2009)
Rohjans, S., Fensel, D., Fensel, A.: OPC UA goes semantics: Integrated communications in smart grids. In: 2011 IEEE 16th Conference on Emerging Technologies and Factory Automation (ETFA), pp. 1–4 (2011)
Thorsen, K.A.H., Torbjørnse, O.F., Rong, C.: Automatic web service detection in oil and gas. In: Ślęzak, D., Kim, T., Chang, A.C.-C., Vasilakos, T., Li, M.C., Sakurai, K. (eds.) FGCN/ACN 2009. CCIS, vol. 56, pp. 193–200. Springer, Heidelberg (2009)
Hortonworks Data Platform. http://hortonworks.com/hdp. Accessed 29 Jan 2015
Richardson, L., Ruby, S.: RESTful Web Services. O’Reilly Media Inc, USA (2008)
Clavel, M., Durán, F., Eker, S., Lincoln, P., Martí-Oliet, N., Meseguer, J., Talcott, C.: Introduction to OPC UA performance. In: Clavel, M., Durán, F., Eker, S., Lincoln, P., Martí-Oliet, N., Meseguer, J., Talcott, C. (eds.) CN 2012, CCIS 291. LNCS, vol. 4350, pp. 1–28. Springer, Heidelberg (2007)
Folkert, K., Fojcik, M., Cupek, R.: Efficiency of OPC UA communication in java-based implementations. In: Kwiecień, A., Gaj, P., Stera, P. (eds.) CN 2011. CCIS, vol. 160, pp. 348–357. Springer, Heidelberg (2011)
Acknowledgement
This work was supported by the European Union from the European Social Fund (grant agreement number: UDA-POKL.04.01.01-00-106/09).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Folkert, K., Fojcik, M. (2015). Ontology-Based Integrated Monitoring of Hadoop Clusters in Industrial Environments with OPC UA and RESTful Web Services. In: Gaj, P., Kwiecień, A., Stera, P. (eds) Computer Networks. CN 2015. Communications in Computer and Information Science, vol 522. Springer, Cham. https://doi.org/10.1007/978-3-319-19419-6_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-19419-6_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19418-9
Online ISBN: 978-3-319-19419-6
eBook Packages: Computer ScienceComputer Science (R0)