Abstract
Workflow management systems (WFMS) are a cornerstone of mission-criticial, possibly cross-organizational business processes. For large-scale applications both their performance and availability are crucial factors, and the system needs to be properly configured to meet the application demands. Despite ample work on scalable system architectures for workflow management, the literature has neglected the important issues of how to systematically measure the performance of a given system configuration and how to determine viable configurations without resorting to expensive trial-and-error or guesswork. This paper proposes a synthetic benchmark for workflow management systems; based on the TPC-C order-entry benchmark, a complete e-commerce workflow is specified in a system-independent form. This workflow benchmark, which stresses all major components of a workflow system and is parameterized in a flexible manner, has been applied to two operational systems, the commercial system Staffware97 and our own prototype system Mentor-lite. The paper reports performance results from our measurements and discusses lessons learned. In particular, the results demonstrate the scalability of the Mentor-lite architecture. The measurements also underline the need for configuring systems intelligently, and the paper outlines an auto-configuration tool that we have been building to this end.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Alonso, G., Agrawal, D., Abbadi, A., Mohan, C.: Functionality and Limitations of Current Workflow Management Systems. IEEE Expert 12(5) (1997)
Bauer, T., Dadam, P.: A Distributed Execution Environment for Large-Scale Workflow Management Systems with Subnets and Server Migration. In: IFCIS Intl Conf. on Cooperative Information Systems (CoopIS), Kiawah Island, South Carolina (1997)
Bauer, T., Dadam, P.: Distribution Models for Workflow Management Systems - Classification and Simulation (in German), Technical Report, University of Ulm, Germany (1999)
Bonner, A., Shrufi, A., Rozen, S.: LabFlow-1: a Database Benchmark for High-Throughput Workflow Management. In: Int’l Conf. on Extending Database Technology (EDBT), Avignon, France (1996)
Cichoki, A., Helal, A., Rusinkiewicz, M., Woelk, D.: Workflow and Process Automation, Concepts and Technology. Kluwer, Dordrecht (1998)
Dogac, A., Kalinichenko, L., Tamer Ozsu, M., Sheth, A. (eds.): Workflow Management Systems and Interoperability, NATO Advanced Study Institute. Springer, Heidelberg (1998)
Geppert, A., Berndtsson, M., Lieuwen, D., Roncancio, C.: Performance Evaluation of Object-Oriented Active Database Management Systems Using the BEAST Benchmark. Theory and Practice of Object Systems, TAPOS 4(4) (1998)
Georgakopoulos, D., Hornick, M., Sheth, A.: An Overview of Workflow Management: From Process Modeling to Workflow Automation Infrastructure. Distributed and Parallel Databases 3(2) (1995)
Gillmann, M., Weissenfels, J., Weikum, G., Kraiss, A.: Performance and Availability Assessment for the Configuration of Distributed Workflow Management Systems. In: Int’l Conf. on Extending Database Technology (EDBT), Konstanz, Germany (2000)
Gray, J. (ed.): The Benchmark Handbook, 2nd edn. Morgan Kaufmann, San Francisco (1993)
Hagen, C., Alonso, G.: Flexible Exception Handling in the OPERA Process Support System. In: Int’l Conf. on Distributed Computing Systems (ICDCS), Amsterdam, The Netherlands (1998)
Harel, D.: State Charts: A Visual Formalism for Complex Systems. Science of Computer Programming 8 (1987)
Harel, D., Gery, E.: Executable Object Modeling with Statecharts. IEEE Computer 30(7) (1997)
Jain, R.: The Art of Computer Systems Performance Analysis. John Wiley & Sons, Chichester (1991)
Jablonski, S., Bussler, C.: Workflow-Management, Modeling Concepts, Architecture and Implementation. International Thomson Computer Press, London (1996)
Klingemann, J., Waesch, J., Aberer, K.: Deriving Service Models in Cross-Organizational Workflows. In: 9th Int’l Workshop on Reasearch Issues in Data Engineering (RIDE), Sydney, Australia (1999)
Kamath, M., Alonso, G., Günthör, R., Mohan, C.: Providing High Availability in Very Large Workflow Management Systems. In: 5th Int’l Conf. on Extending Database Technology (EDBT), Avignon, France (1996)
Leymann, F., Roller, D.: Production Workflow: Concepts and Techniques. Prentice-Hall, Englewood Cliffs (1999)
Mohan, C.: Workflow Management in the Internet Age, Tutorial, http://www-rodin.inria.fr/~mohan
Moore, K., Peterson, M.: A Groupware Benchmark Based on Lotus Notes. In: Int’l Conf. on Data Engineering (ICDE), New Orleans, Louisiana (1996)
Muth, P., Weissenfels, J., Gillmann, M., Weikum, G.: Integrating Light-Weight Workflow Management Systems within Existing Business Environments. In: Int’l Conf. on Data Engineering (ICDE), Sydney, Australia (1999)
Muth, P., Wodtke, D., Weissenfels, J., Weikum, G., Kotz Dittrich, A.: Enterprise-wide Workflow Management based on State and Activity Charts. In: [6]
Muth, P., Wodtke, D., Weissenfels, J., Kotz Dittrich, A., Weikum, G.: From Centralized Workflow Specification to Distributed Workflow Execution. Intelligent Information Systems, Special Issue on Workflow Management 10(2) (1998)
Nüttgens, M., Feld, T., Zimmermann, V.: Business Process Modeling with EPC and UML: Transformation or Integration. In: Schader, M., Korthaus, A. (eds.) The Unified Modeling Language - Technical Aspects and Applications, Workshop des Arbeitskreises “Grundlagen objektorientierter Modellierung” (GROOM), Heidelberg, Germany (1998)
O’Neil, P.: Database Performance Measurement. In: Tucker, A.B. (ed.) The Computer Science and Engineering Handbook. CRC Press, Boca Raton (1997)
SAP AG: SAP E-Business Solutions, http://www.sap-ag.de/solutions/technology/index.htm
Scheer, A.W.: Benchmarking Business Processes. In: Okino, N., Tamura, H., Fujii, S. (eds.) Advances in Production Management Systems, IFIP TC5/WG5.7 Int’l Conf. on Production Management Systems (APMS), Kyoto, Japan (1996)
Schuster, H., Neeb, J., Schamburger, R.: A Configuration Management Approach for Large Workflow Management Systems. In: Int’l Joint Conf. on Work Activities Coordination and Collaboration (WACC), San Francisco, California (1999)
Staffware, http://www.staffware.com
Transaction Processing Performance Council, http://www.tpc.org
TPC-W Benchmark Specification, http://www.tpc.org/wspec.html
Workflow Management Coalition, http://www.wfmc.org
Wodtke, D., Weikum, G.: A Formal Foundation For Distributed Workflow Execution Based on State Charts. In: Int’l Conf. on Database Theory (ICDT), Delphi, Greece (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gillmann, M., Mindermann, R., Weikum, G. (2000). Benchmarking and Configuration of Workflow Management Systems. In: Scheuermann, P., Etzion, O. (eds) Cooperative Information Systems. CoopIS 2000. Lecture Notes in Computer Science, vol 1901. Springer, Berlin, Heidelberg. https://doi.org/10.1007/10722620_19
Download citation
DOI: https://doi.org/10.1007/10722620_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41021-8
Online ISBN: 978-3-540-45266-9
eBook Packages: Springer Book Archive