Abstract
An increasing variety of products contributes to the challenge of efficient manufacturing on production lines, e.g. in the Fast Moving Consumer Goods (FMCG) sector. Due to the complexity and multitude of adjustment levers, the identification of economic actions for improvement is challenging. Reinforcement learning offers a way to deal with such complex problems with little problem-specific adaptation. This paper presents a method for decision support for economic productivity improvement of production lines. A combination of discrete event simulation and reinforcement learning is used to identify efficient, sequential trajectories of improvements. The approach is validated with a fill-and-pack line of the FMCG industry.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Nakajima, S.: Introduction to TPM. Productivity Press, Cambridge, Mass, Total productive maintenance (1988)
Bartkowiak, T., Ciszak, O., Jablonski, P., Myszkowski, A., Wisniewski, M.: A simulative study approach for improving the efficiency of production process of floorboard middle layer. In: Hamrol, A., Ciszak, O., Legutko, S., Jurczyk, M. (eds.) Advances in Manufacturing. LNME, pp. 13–22. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-68619-6_2
Bech, S., Brunoe, T.D., Nielsen, K., Andersen, A.-L.: Product and Process Variety Management: Case study in the Food Industry. Procedia CIRP (2019)
Butollo, F.: Digitalization and the geographies of production: Towards reshoring or global fragmentation? Competition & Change (2020)
Andersson, C., Bellgran, M.: On the complexity of using performance measures: Enhancing sustained production improvement capability by combining OEE and productivity. J. Manuf. Syst. 35, 144–154 (2015)
Del Carmen Ng Corrales, L., Lambán, M.P., Hernandez Korner, M.E., Royo, J.: Overall equipment effectiveness: systematic literature review and overview of different approaches. Applied Sciences (2020)
Koren, Y., Hu, S.J., Weber, T.W.: Impact of Manufacturing System Configuration on Performance. CIRP Annals (1998)
Yegul, M.F., Erenay, F.S., Striepe, S., Yavuz, M.: Improving configuration of complex production lines via simulation-based optimization. Computers & Industrial Engineering (2017)
Xi, S., Chen, Q., MacGregor Smith, J., Mao, N., Yu, A., Zhang, H.: A new method for solving buffer allocation problem in large unbalanced production lines. Int. J. Prod. Res. 58, 6846–6867 (2020)
Rabe, M., Deininger, M., Juan, A.A.: Speeding up computational times in simheuristics combining genetic algorithms with discrete-Event simulation. Simulation Modelling Practice and Theory (2020)
Karlsson, I.: An interactive decision support system using simulation-based optimization and knowledge extraction. Dissertation Series. Doctoral thesis, monograph, University of Skövde (2018). http://urn.kb.se/resolve?urn=urn:nbn:se:his:diva-16369
Trigueiro de Sousa Junior, W., Barra Montevechi, J.A., Carvalho Miranda, R. de, Teberga Campos, A.: Discrete simulation-based optimization methods for industrial engineering problems: a systematic literature review. Computers & Industrial Engineering (2019)
Ylipää, T., Skoogh, A., Bokrantz, J., Gopalakrishnan, M.: Identification of maintenance improvement potential using OEE assessment. Int. J. Productivity Perf. Mgmt. 1, 126–143 (2017). https://doi.org/10.1108/IJPPM-01-2016-0028
Wu, K., Zheng, M., Shen, Y.: A generalization of the Theory of Constraints: Choosing the optimal improvement option with consideration of variability and costs. IISE Trans. 3, 276–287 (2020). https://doi.org/10.1080/24725854.2019.1632503
Burggraf, P., Wagner, J., Koke, B.: Artificial Intelligence in Production Management: A Review of the Current State of Affairs and Research Trends in Academia. IEEE, Piscataway (2018)
Burggräf, P., Wagner, J., Koke, B., Bamberg, M.: Performance assessment methodology for AI-supported decision-making in production management. Procedia CIRP (2020)
Gosavi, A.: Solving markov decision processes via simulation. In: Handbook of Simulation Optimization, vol. 216 (2014)
Sutton, R.S., Barto, A.: Reinforcement Learning. An Introduction. Adaptive Computation and Machine Learning. The MIT Press, Cambridge, MA (2018)
Gosavi, A.: Simulation-Based Optimization. Parametric Optimization Techniques and Reinforcement Learning. Springer, New York 55 (2015)
Juan, A.A., Faulin, J., Grasman, S.E., Rabe, M., Figueira, G.: A review of simheuristics: Extending metaheuristics to deal with stochastic combinatorial optimization problems. Operations Research Perspectives (2015)
Hubscher-Younger, T., Mosterman, P.J., DeLand, S., Orqueda, O., Eastman, D.: Integrating discrete-event and time-based models with optimization for resource allocation. IEEE (2012)
Tempelmeier, H.: Practical considerations in the optimization of flow production systems. Int. J. Prod. Res. 1, 149–170 (2003). https://doi.org/10.1080/00207540210161641
Bergeron, D., Jamali, M.A., Yamamoto, H.: Modelling and analysis of manufacturing systems: a review of existing models. IJPD (2010)
Nourelfath, M., Nahas, N., Ait‐Kadi, D.: Optimal design of series production lines with unreliable machines and finite buffers. J. Qual. Maintenance Eng. 2, 121–138 (2005). https://doi.org/10.1108/13552510510601348
Liu, Y., Li, J.: Split and merge production systems: performance analysis and structural properties. IIE Trans. 6, 422–434 (2010). https://doi.org/10.1080/07408170903394348
Spinellis, D.D., Papadopoulos, C.T.: A simulated annealing approach for buffer allocation in reliable production lines. Annals of Operations Research (2000)
Zhang, M., Matta, A.: Models and algorithms for throughput improvement problem of serial production lines via downtime reduction. IISE Trans. 11, 1189–1203 (2020). https://doi.org/10.1080/24725854.2019.1700431
Filho, M.G., Utiyama, M.H.R.: Comparing the effect of different strategies of continuous improvement programmes on repair time to reduce lead time. Int. J. Adv. Manuf. Technol. 87(1–4), 315–327 (2016). https://doi.org/10.1007/s00170-016-8483-x
Godinho Filho, M., Utiyama, M.H.R.: Comparing different strategies for the allocation of improvement programmes in a flow shop environment. Int. J. Adv. Manuf. Technol. 77(5–8), 1365–1385 (2014). https://doi.org/10.1007/s00170-014-6553-5
Jasiulewicz-Kaczmarek, M., Bartkowiak, T.: Improving the performance of a filling line based on simulation. IOP Conf. Ser.: Mater. Sci. Eng. 145, 042024 (2016). https://doi.org/10.1088/1757-899X/145/4/042024
Harrington, B.:Breakdowns Happen. How to Factor Downtime into your Simulation. Whitepaper (2014)
Dagkakis, G., Heavey, C., Robin, S., Perrin, J.: ManPy: an open-source layer of DES manufacturing objects implemented in simPy. IEEE, Piscataway, NJ (2013)
Sutton, R.S., McAllester, D., Singh, S., Mansour, Y.: Policy Gradient Methods for Reinforcement Learning with Function Approximation. MIT Press, Cambridge (1999)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal Policy Optimization Algorithms (2017). https://arxiv.org/pdf/1707.06347
Acknowledgment
The authors would like to thank the German Research Foundation DFG for funding this work within the Cluster of Excellence “Internet of Production” (Project ID: 390621612).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 IFIP International Federation for Information Processing
About this paper
Cite this paper
Schuh, G., Gützlaff, A., Schmidhuber, M., Maetschke, J., Barkhausen, M., Sivanesan, N. (2021). Identification of Superior Improvement Trajectories for Production Lines via Simulation-Based Optimization with Reinforcement Learning. In: Dolgui, A., Bernard, A., Lemoine, D., von Cieminski, G., Romero, D. (eds) Advances in Production Management Systems. Artificial Intelligence for Sustainable and Resilient Production Systems. APMS 2021. IFIP Advances in Information and Communication Technology, vol 634. Springer, Cham. https://doi.org/10.1007/978-3-030-85914-5_43
Download citation
DOI: https://doi.org/10.1007/978-3-030-85914-5_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-85913-8
Online ISBN: 978-3-030-85914-5
eBook Packages: Computer ScienceComputer Science (R0)
