Motion Style Transfer in Correlated Motion Spaces

Abstract

This paper presents a methodology for transferring different motion style behaviors to virtual characters. Instead of learning the differences between two motion styles and then synthesizing the new motion, the presented methodology assigns to the style transformation the motion’s distribution transformation process. Specifically, in this paper, the joint angle values of motion are considered as a three-dimensional stochastic variable and as a set of samples respectively. Thus, the correlation between three components can be computed by the covariance. The presented method imports covariance between three components of joint angle values, while calculating the mean along each of the three axes. Then, by decomposing the covariance matrix using the singular value decomposition (SVD) algorithm, it is possible to retrieve a rotation matrix. For fitting the motion style of an input to a reference motion style, the joint angle orientation of the input motion is scaled, rotated and transformed to the reference style motion, therefore enabling the motion transfer process. The results obtained from such a methodology indicate that quite reasonable motion sequences can be synthesized while keeping the required style content.

Appendix

Here the definition of the components used in Eq. 2 is presented. Specifically, the matrices of $T_{ref}$, $T_{in}$, $R_{ref}$, $R_{in}$, $S_{ref}$, and $S_{in}$ denote the translation, rotation and scaling derived from the reference style and the input motion respectively. They are solved as:

$$\begin{aligned} T_{ref} = \begin{bmatrix} 1&0&0&\bar{X}_{ref} \\ 0&1&0&\bar{Y}_{ref} \\ 0&0&1&\bar{Z}_{ref} \\ 0&0&0&1 \end{bmatrix} \end{aligned}$$

(3)

$$\begin{aligned} T_{in} = \begin{bmatrix} 1&0&0&-\bar{X}_{in} \\ 0&1&0&-\bar{Y}_{in} \\ 0&0&1&-\bar{Z}_{in} \\ 0&0&0&1 \end{bmatrix} \end{aligned}$$

(4)

$$\begin{aligned} S_{ref} = \begin{bmatrix} \bar{\lambda }^X_{ref}&0&0&0 \\ 0&\bar{\lambda }^Y_{ref}&0&0 \\ 0&0&\bar{\lambda }^Z_{ref}&0 \\ 0&0&0&1 \end{bmatrix} \end{aligned}$$

(5)

$$\begin{aligned} S_{in} = \begin{bmatrix} 1/\sqrt{\bar{\lambda }^X_{in}}&0&0&0 \\ 0&1/\sqrt{\bar{\lambda }^Y_{in}}&0&0 \\ 0&0&1/\sqrt{\bar{\lambda }^Z_{in}}&0 \\ 0&0&0&1 \end{bmatrix} \end{aligned}$$

(6)

$$\begin{aligned} R_{ref}=U_{ref} \end{aligned}$$

(7)

$$\begin{aligned} R_{in}=U_{in}^{-1} \end{aligned}$$

(8)