A UML Profile for the Design, Quality Assessment and Deployment of Data-intensive Applications

Abstract

Big Data or Data-Intensive applications (DIAs) seek to mine, manipulate, extract or otherwise exploit the potential intelligence hidden behind Big Data. However, several practitioner surveys remark that DIAs potential is still untapped because of very difficult and costly design, quality assessment and continuous refinement. To address the above shortcoming, we propose the use of a UML domain-specific modeling language or profile specifically tailored to support the design, assessment and continuous deployment of DIAs. This article illustrates our DIA-specific profile and outlines its usage in the context of DIA performance engineering and deployment. For DIA performance engineering, we rely on the Apache Hadoop technology, while for DIA deployment, we leverage the TOSCA language. We conclude that the proposed profile offers a powerful language for data-intensive software and systems modeling, quality evaluation and automated deployment of DIAs on private or public clouds.

Appendices

Appendix A: MARTE and DAM profiles

MARTE [64] is a standard profile that extends UML for the performance and schedulability analysis of a system. MARTE consists of three main parts: MARTE Foundations, MARTE Design Model and MARTE Analysis Model. The Analysis Model is of our interest since it enables the QoS assessment by allowing the definition of QoS metrics and properties. The Analysis Model consists of a Generic Quantitative Analysis and Modelling (GQAM) profile and its specialization, the Performance Analysis and Modelling (PAM) profile. In addition to this, two other features are also important for our DIA profile.

Fig. 24

Transformation of Apache Hadoop specific stereotypes to Petri nets

The first one is that MARTE enables the specification of quantitative non-functional properties (NFP) in UML models through its Value Specification Language (VSL). The VSL is useful for specifying the values of constraints, properties and stereotype attributes, particularly related to NFPs. Moreover, VSL allows to express basic types, data types, values (such as time and composite values), as well as variables, constants and expressions. This means that using VSL we can define complex metrics and requirements to express, for example, response times, utilizations or throughputs. MARTE also defines a library of primitive data types, a set of predefined NFP types and units of measures. Hence, our DIA profile inherits the VSL altogether.

For understanding the VSL expressions that appear in this paper, it is of interest to briefly recall its syntax. An example of VSL expression for a host demand tagged value of type NFP_Duration is:

This expression specifies that the Reducing activity in Fig. 13 demands 6 (1) milliseconds (2) of processing time, whose mean value (3) is obtained from an estimation in the real system (4). We could replace, for example, the value 6 for a variable $$host\_dem$ to parameterize the analysis of the model with different values for this host demand.

The second feature is that the DAM [10] profile specializes MARTE-GQAM for dependability analysis (i.e., availability, reliability, safety and maintainability). Consequently, the DAM profile also inherits the VSL. As MARTE, DAM consists of a library and a set of extensions to be applied at model specification level. Our DIA profile inherits DAM with the purpose of addressing reliability analysis for DIA.

Appendix B: DIA Profile Library

In this Appendix, we present the DIA library. The library defines the data types, basic and complex, used in the attributes of the stereotypes proposed for the three abstraction levels, DPIM, DTSM and DDSM. Basic types appear in Fig. 22, while complex ones in Fig. 23. From DAM, we have imported the DAM Library [10], which also imports the MARTE Library [64].

Appendix C: TOSCA

TOSCA provides a flexible and highly extensible DSL for modeling resources and software components. TOSCA blueprints are executable IasC composed of node templates and relationships, defining the topology of a hardware/software systems. Node templates and relationships are instances of node types and relationship types, that are either normative (i.e., defined in the standard), provided by the specific engine that executes a blueprint (the orchestrator), or an extension of one of the above, such as in our case, with DIA-specific node and relationship types. Node types are essentially used to describe hardware or virtual resources (machines or VMs) and software components. Relationship types predicate on the association between node types. For instance, a TOSCA node type representing Wordpress CMS must be associated with a node type presenting VMs through the relationship hosted_On. Each node type and relationship type also enables specifying interfaces, which are composed of operations that have to be carried out at specific stages of the deployment orchestration. Typical examples of interface operations include installing, configuring or starting of components, and may take form of Python/bash scripts, or pointers to Chef recipes. Node and relationship templates are free to provide their own interface operations, extending or overriding behavior defined in the corresponding types. TOSCA is being supported by a number of orchestrators that, given a TOSCA blueprint and all node and relationship types used there, are able to execute it deploying the corresponding system and managing its lifecycle. Examples of such orchestrators are Cloudify,¹⁸ ARIA TOSCA,¹⁹ Indigo,²⁰ Apache Brooklyn²¹ or ECoWare [6].

Table 4 Research questions on profile usability

Appendix D: Transformation of a DTSM design to a performance model

Stochastic Well-formed Nets (SWN) [16] are a modeling formalism suitable for performance analysis purposes. A SWN model is a bipartite graph formed by places and transitions. Places are graphically depicted as circles and may contain tokens. A token distribution in the places of a SWN, namely a marking, represents a state of the modeled system. The dynamic of the system is governed by the transition enabling and firing rules, where places represent pre- and post-conditions for transitions. In particular, the firing of a transition removes (adds) as many tokens from its input (output) places as the weights of the corresponding input (output) arcs. Transitions can be immediate, those that fire in zero time; or timed, those that fire after a delay which is sampled from a random variable with a given probability distribution function. Immediate transitions are graphically depicted as black thin bars, while timed ones are depicted as white thick bars. Tokens may also have an associated color, i.e., a data type, which enriches the expressiveness of the net and restricts the movement of tokens to compatible places and transitions.

Figure 24 depicts a schema of how Apache Hadoop stereotypes in UML-profiled models (left) are transformed into an analyzable model such as a SWN (right). Each stereotype is transformed into a sub-net by taking into account the information contained in the tags. For each transformation pattern in the Figure, the part of the Petri net inside the blue box corresponds to the part that the transformation creates. The part of the Petri net outside the blue box corresponds to referenced parts, which are in turn created by other stereotypes. Figure 24 depicts only the specific non-functional annotations for Apache Hadoop; the functional part of the UML diagram is transformed according to the works in [33, 70]. Eventually, all the sub-nets are composed into a single closed Petri net such as in Fig. 16.

A Hadoop cluster accepts several categories of users, whose jobs are probably subdivided into a different number of map–reduce tasks or have assigned a different number of hardware resources. Every user $<i>$ has $\$nC_i$ jobs waiting in the scheduler queue. Hadoop scheduler launches periodically a new job at a given $\$rate$ following a scheduling policy defined by the scenario (e.g., a shared common FIFO queue for all users). By default, our transformation assumes an independent FIFO queue for each user and always guarantees to take a job of each user.

Jobs are labeled with the user they belong to (loop $<i>$-$<i+1>$ in the net, where $<i>$ represents each user). The scheduler waits for the assignment of resources to all tasks in the reduce phase of the precedent job $<i>$ before launching the next job $<i+1>$ (inhibitor arc section). This scheduling allows both concurrency among jobs and giving priority over resources to precedent jobs. Job $<i>$ is divided in $\$m_i$ map tasks and $\$r_i$ reduce tasks, that run simultaneously in up to $\$p_i$ cores ($\sum _{i=1}^{n} \$p_i \ge \$host$, being $\$host$ the total number of cores in the cluster). We use the notation $<i>$ for expressing the color of a token. For instance, each user is represented by a different color in the SWN. Notation $\$m_i$ is used for expressing numerical values; for instance, the number of map tasks in which a job of type $<i>$ is divided.

Appendix E: Usability of the profile

The validation of the DIA profile has been carried out so far from the point of view of its adequacy to solve the QoS assessment and the deployment. However, we consider also important to learn about the usability of the profile, in terms of easiness of use for engineers. It uses to happen that tools, although offering the required functionalities, do not reach their expectations until a degree of maturity is accepted at this regard.

The DIA profile has been used by engineers in four organizations: Prodevelop [38], ATC [5], BluAge [12] and XLAB R&D [71]. We have prepared eight questions, see Table 4, for a total of eight engineers, who have extensively used the DIA profile in the context of the DICE project to carry out industrial applications. From the answers, we see that the profile has been useful for the engineers, specially for the automatic deployment. However, the main lack refers to the Papyrus implementation (see question #6) that also constraints the profile implementation. In fact, the advice of the engineers (see question #8) referred to improve the Papyrus implementation of the profile.