Abstract
Ultrasound shear wave elastography (SWE) techniques have been very useful for the analysis of tissue rheological properties, but there are still obstacles for robust evaluation of viscoelastic tissue properties. In this proof-of-concept study, we investigate whether convolutional neural networks (CNN) are capable of retrieving the elasticity and viscosity parameters from simulated shear wave motion images. Staggered-grid finite difference simulations based on a Kelvin-Voigt rheological model were used to generate data for this study. The wave motion datasets were created using Kelvin-Voigt shear elasticity values ranging from 1-25 kPa, shear viscosities ranging from 0-10 Pa·s, and two different push profiles using f-numbers of 1 and 2. The CNN architectures, optimized using mean squared error loss, were then trained to retrieve a specific viscoelastic parameter. Both elasticity and viscosity values were successfully retrieved, with regression R2 values above 0.99 when correlating the estimated mechanical properties versus the true mechanical properties. The CNN performance was also compared to estimation of shear elasticity and viscosity from fitting dispersion curves estimated from two-dimensional Fourier transform analysis. The results demonstrated that the CNN models were robust to noise, vertical position and partially to f-number. The architecture was proven to be robust to multiple push profiles if trained properly. The CNN results showed higher accuracy over the full viscoelastic parameter range compared to the Fourier-based analysis. The overall results showed the CNNs’ potential to be an alternative to complex mathematical analyses such as Fourier analysis and dispersion curve estimation used currently for shear wave viscoelastic parameter estimation.
Keywords: Shear wave elastography (SWE), ultrasound, acoustic radiation force, machine learning, convolutional neural networks
I. Introduction
Ultrasound imaging is one of the main imaging modalities used in healthcare. It is widely used for multiple applications due to its low cost, relative portability and safety [1]. One of the applications that has developed popularity over the last several years is ultrasound shear wave elastography (SWE). SWE typically uses focused ultrasound beams to produce acoustic radiation force (ARF) that pushes within the tissue [2], [3]. When the ARF ceases, the locally perturbed tissue returns to its equilibrium position, but also generates waves traveling perpendicular to the push direction. These shear waves propagate through the tissue, and the motion is measured using high frame rate ultrasound techniques [4], [5]. The wave motion can be analyzed to extract information about the wave velocity and attenuation. These parameters can also be used to characterize rheological properties of the tissue such as elasticity and viscosity.
The estimation of rheological parameters is dependent on mathematical models, such as the Kelvin-Voigt and Maxwell models [6], [7]. To date, methods have been developed to directly measure the wave velocity and attenuation of shear waves as they vary with frequency [7]-[15]. However, the shear wave attenuation can be difficult to directly measure [11]. With the wave velocity dispersion information alone, it is possible to estimate viscoelastic properties for different rheological models [6]-[9], [16], [17]. Most of these methods involve Fourier analysis to extract dispersion curves that are used for fitting to the rheological models. Each of the approaches that have been studied have different levels of bias and variability. Additionally, different aspects of the SWE acquisition such as the ARF distribution and beam shape and measurement noise can affect the wave motion and confound these methods [18]-[20]. Taken together, there is still a need for robust methods for viscoelastic parameter evaluation from measured shear wave motion.
Machine learning (ML) is the computer science field dedicated to creating algorithms capable of modeling complex datasets, and performs tasks such as prediction, regression or clustering. These methods commonly have high computational costs. In recent years, there has been substantial progress on both algorithms and computational power to perform data training and predictions in feasible time with available computational resources. ML can identify patterns within datasets that otherwise would be very difficult to fit by standard statistical models or by evaluating each feature on its own. The use of ML might enable SWE to reach higher applicability, by naturally modeling the intrinsic complexity of viscoelastic tissues [21]-[24].
Artificial Neural Networks (ANN), or simply Neural Networks (NN), were inspired by the central nervous system structure and its physiology. They are composed by nodes, or neurons, interconnected into multiple layers, from input to output. Each node has adjusted weights and activation functions that simulate the synaptic reinforcement plasticity. The adjustment of the weights is implemented by solving the convex problem that minimizes the loss of the networks, which evaluates the prediction assigned by the network against the ground truth output [25], in a process called supervised learning.
The convex problem is solved using numerical approaches, such as gradient descent, which are iterative, and therefore, can take time and large amounts of data. The process of weight adjustment is called optimization and it is an area of research of its own. Some of the most common optimizers used for neural networks are Stochastic Gradient Descent [26] and Adam [27].
Convolutional Neural Networks (CNN), as other NN architectures, were also inspired by neurophysiology. The architecture replicates the layer-based structure from the human visual system, where the convolutional layers work as the receptive fields present in the brain, while the max-pooling layers act as the lateral geniculate nucleus by reducing the level of information and extracting the relevant features for the task in hand [28]. The input is convolved with the feature weights to produce a feature map which is processed by a non-linear activation function (rectified linear units, traditionally). The pooling layers then reduce the spatial resolution of the feature maps to achieve spatial invariance to input distortions and translations. Max-pooling layers aggregate and propagate the maximum value in a receptive field to the next layers. After stacking multiple convolutional and pooling layers, a fully connected layer is implemented to interpret the feature representations extracted by previous layers, much like the human visual cortex. The fully connected layer is a traditional NN, with different output layer activation functions and loss functions, depending on the task [28]. The CNN basic schema is exemplified in Fig. 1. CNNs have been extensively used for image recognition tasks with high performance and generalizability [22], [28].
In this proof-of-concept study, we propose to analyze, in a controlled setting, the capabilities of ML to model wave propagation and retrieve elasticity and viscosity values from wave motion images simulated using the staggered-grid finite-difference (SGFD) approach to implement a Kelvin-Voigt (KV) material model [29]-[31]. Due to the two-dimensional (2D) image characteristics of the task, the CNN architecture was deemed the best for the objectives of this proof-of-concept. We also compared the ML performance to the two-dimensional Fourier transform (2D FT) analysis approach for evaluating dispersion curves and fitting to the KV theoretical dispersion curve [6], [7], [16].
The modeling of viscoelastic wave propagation is useful for understanding wave phenomena in complex media. Finite-difference modeling approaches have great importance for geophysical exploration, reservoir engineering, and military applications because they provide complete wavefield responses at reasonable computational cost [30], [32]. The SGFD method was used to simulate wave propagation in viscoelastic homogeneous media. SGFD adopts a finite difference operator with fixed-order accuracy to calculate spatial derivatives for a homogeneous or heterogeneous medium. The method’s accuracy, efficiency and stability are improved by the use of staggered grid for spatial discretization [30], [31].
II. Methods
For this study, the simulations were created using SGFD simulations implemented in MATLAB (Mathworks, Natick, MA, USA) and computed on GTX 1080 and RTX 2080Ti GPU cards (Nvidia, Santa Clara, CA, USA). Each simulation took approximately 1-8 hours of computing time depending on the GPU card used. The SGFD simulations implement a KV material in the Navier-Stokes equations to calculation the motion caused by acoustic radiation force from ultrasound push beams. The shear elasticity and viscosity ranges for the KV model were chosen so they would cover a wide range of values for normal and pathological soft tissues [33]. The shear elasticity, μ1, ranged from 1-25 kPa with increments of 1 kPa, and shear viscosity, μ2, ranged from 0-10 Pa·s with increments of 0.5 Pa·s. The 25 values of shear elasticity and 21 values of shear viscosity yield 525 unique combinations of viscoelastic parameters. The medium is assumed to be homogeneous, isotropic, linear, and incompressible [29].
We used two different ARF excitation focus profiles with f-numbers (F/N = focal length/width of aperture) 1 and 2 (Fig. 2) with a focal distance of zf = 2.5 cm, adding up to 1050 simulations. These values of F/N are used frequently in practice and provided cases of a very focused beam (F/N = 1) and a less focused beam (F/N = 2). The ARF was simulated using FOCUS using the model of an L7-4 linear array transducer (Philips Healthcare, Andover, MA) [33], [34]. The element pitch was set to 0.0308 cm, the element kerf was 0.0025 cm, the element height was 0.7 cm, the elevation focus was 2.5 cm, and the frequency was 4 MHz. The medium was modeled using a density of 1000 kg/m3, speed of sound of 1540 m/s, and ultrasound attenuation of 0.5 dB/cm/MHz. A focal depth of 2.5 cm was used and for F/N = 1 and 2, such that 82 and 40 elements were used, respectively. The pressure was simulated, and the intensity was calculated as it is proportional to the ARF in absorbing soft tissues. For the models, the force was applied for 200 μs. For each combination of μ1, μ2, and F/N, a dataset was produced.
An important factor of finite-difference methods is the model’s boundary conditions. The boundary conditions have to absorb and attenuate the waves without creating artifacts such as reflections. It is also important that boundary conditions do not overly increase computational costs. Perfectly matched layers (PML) are considered the optimal absorbing boundary condition for finite-difference methods, and therefore, were implemented in this study [35].
Another important aspect of SGFD simulations is the Courant-Friederichs-Lewy criterion (CFL), as it describes the conditional stability that allows convergent behavior of the solution. The space-time discretization is dictated by the smallest shear elasticity (μ1) and the highest shear viscosity (μ2) to be simulated, and so, the CFL criterion determines the time increment and thus the number of time steps to achieve a certain simulation length for a given shear wave velocity.
(1) |
where the dimensionless number C is called the Courant number, Vp and Vs are the compressional and shear wave velocities, Δt is the time step, and Δx the spatial step in the x dimension. In this study’s simulations, the Δx and Δz (spatial step in the z dimension) were set to 0.1 mm, in a field of 44 mm width and 32 mm height. To optimize the simulation run time, two different values of C were used in the simulations depending on the media properties: 0.2 if μ1 > 5 kPa and 0.05 if μ1 ≤ 5 kPa and μ2 ≥ 4 Pa·s. The simulation Δt was then calculated based on Eq. (1), for a total period of 25 ms. To meet the conditions described above, the temporal resolution was very high (5-7 MHz). After completion, the simulated data was resampled in time to 16 kHz, to ensure that all models would have the same time discretization. This sampling frequency could be reached with interpolated data that would be upsampled for precise time-of-flight estimates for evaluation of shear elasticity estimations [17].
The three-dimensional simulations (z, x, t) were then cropped and sampled in the z-direction in order to create the 2D x-axis vs. time wave motion dataset for training. The excitation generates two identical wave fronts propagating perpendicular to the push, therefore, the simulation region-of-interest (ROI) used to create the dataset was composed solely by the right hand-side wave front (Fig. 3a). The ROI has a width of 2 cm with its origin at 0.2 cm laterally from the push center, to eliminate the high amplitude motion of the push from the dataset images. The time range adopted was 20 ms. The data that was used for the CNN input was the motion from particular depths in the z-direction. To augment the dataset the right hand-side lateral propagation was used from five different vertical positions relative to the push focus (Fig. 3a), at −0.90, −0.45, 0.00 (center), 0.45 and 0.90 cm (Fig. 3). Another technique implemented to augment the number of samples, was adding different levels of Gaussian noise to the lateral propagation images to generate motion with signal-to-noise ratios (SNRs) consistent to literature, at 20, 10 and 5 dB (Figs. 3b and 3c) [29], [36], [37]. The segmentation and augmentation of the dataset was also implemented in MATLAB. With the five depth positions and three noise levels and 1050 unique combinations of the viscoelastic parameters, the total number of images used for the study was 15,750.
The CNN architecture was implemented in Python (Python Software Foundation, Wilmington, DE, USA), using Keras [38] with Tensorflow (Google Brain, Mountain View, CA, USA) backend, and trained on a Z820 workstation (Hewlett-Packard, Palo Alto, CA, USA) equipped with a GTX 1080 GPU (Nvidia, Santa Clara, CA, USA). The data in each image was normalized by subtracting the minimum amplitude from the whole ROI and dividing all motion amplitude values by the maximum amplitude after subtraction, creating images where all the values had an amplitude between 0 and 1. The labels were also normalized with a normalizing factor of the maximum label value (μ1 = 25 kPa for elasticity and μ2 = 10 Pa·s for viscosity) plus a 20% margin, to leave a margin for model overestimation in the top of the range. The same margin was not used at the bottom of the range (μ1 = 1 kPa for elasticity and μ2 = 0 Pa·s for viscosity), because negative values are not physically possible.
The training, validation and test sets were defined using a group shuffle split of 60/20/20, which means that images derived from the same simulation (μ1, μ2, F/N combination) could not be present in more than one set simultaneously. This ensures that simulation subjects presented to the CNN in the validation and test datasets were never seen during training in any of their augmented versions. Due to the regression characteristics of the task, sigmoid and mean squared error (MSE) were implemented as output layer activation function and loss function, respectively.
The elasticity and viscosity determine different characteristics of the wave propagation. The elasticity is related to the time-of-flight (e.g., slope of the wave motion distribution in the x-t plane, Fig. 3b), whereas the viscosity affects the widening of the wave at a particular location.
Therefore, for each dataset, a pair of CNN models were implemented, one for elasticity regression, and one for viscosity regression. To facilitate the compilation of results, a coding system was applied to name the models and results in terms of what the training/validation and test f-numbers were used (Table I). The letter designates what label was used, elasticity (E) or viscosity (V); the first number specifies the F/N used for training and validation, whereas the second number designates the F/N used for testing.
TABLE I.
Pair code | Training F/N (number of images) |
Validation F/N (number of images) |
Test F/N (number of images) |
---|---|---|---|
E11, V11 | 1 (4725) | 1 (1575) | 1 (1575) |
E12, V12 | 1 (4725) | 1 (1575) | 2 (7875) |
E22, V22 | 2 (4725) | 2 (1575) | 2 (1575) |
E21, V21 | 2 (4725) | 2 (1575) | 1 (7875) |
E1+2, V1+2 | 1 & 2 (9450) | 1 & 2 (3150) | 1 & 2 (3150) |
The CNN models trained, validated and tested on data from both f-numbers were denoted by the X1+2 designation. The CNN models were optimized for multiple combinations of 1, 2 and 3 convolutional layers; 2, 3 and 4 dense layers; 16 and 32 nodes per layer (convolutional and dense); 0.001, 0.0005 and 0.0001 learning rates; totaling 54 models. Tensorboard (Google Brain, Mountain View, CA, USA) was used to monitor and evaluate the models during the hyperparameter grid search. By evaluating the validation loss and mean absolute error (MAE) during 200 epochs, it is possible to determine when the learning reached an asymptote or demonstrated overfitting. MAE is defined by the equation:
(2) |
where N is the number of test subjects, is the estimated value of the subject and xi is the true value of the parameter. Each 200-epoch training session took approximately 1 hour of GPU computing time on the GTX 1080 GPU card. For each grid search, the top 20% CNN models with the lowest validation MAE were selected and re-trained for a subsequent round of evaluation until one hyperparameter set remained (Fig. 4). This approach was used to mitigate hyperparameter overfitting, because each iteration of cross evaluation is performed on different random group splits. The final model was then trained, also for 200 epochs, using the optimal hyperparameters. Checkpoint recall was used to store the best model generated and prevent overfitting. The models were then evaluated against the test set.
The datasets were also evaluated using 2D FT analysis and the estimated dispersion curves were fit to the theoretical Kelvin-Voigt model to obtain the viscoelastic parameters. This analytical approach was implemented in MATLAB. The 2D fast Fourier transform was performed with a zero padding factor of 4096 for optimum frequency and wavenumber resolution of the frequency domain representation (k-space). The dispersion curves were than estimated from the k-space using the relationship cp(ωp) = ωp/kmax, where cp is the phase velocity at ωp frequency from 100-600 Hz, and kmax is the wavenumber of maximum k-space energy at ωp. Additionally, cp values below 0 m/s and above 10 m/s were discarded from further analysis. The dispersion curves were then fit to the theoretical Kelvin-Voigt model (Eq. 3), with the density ρ = 1000 kg/m3, to obtain μ1 and μ2 values. The estimated values were then compared to the CNN results.
(3) |
III. Results
The testing results were evaluated using the MAE between the true value of the viscoelastic property and the CNN prediction output. A linear regression was performed to compare the results from the 2D FT analysis as well as the CNN prediction to the identity line, and the slope, intercept and R2 of the regression line were also calculated and reported. The optimal hyperparameters were also noted and reported in Tables II and III for the elasticity and viscosity CNN models, respectively. All elasticity models trained and tested on the same f-numbers showed very good fitting with MAE below 0.079 kPa with slopes and intercept of approximately one and zero, respectively. The viscosity models trained and tested on the same f-numbers also showed very good agreement with MAE below 0.091 Pa·s with slopes and intercept of approximately one and zero, respectively. Alternatively, CNN models tested on f-numbers different from the training/validation F/N had diminished performance with MAE of at least 1.542 Pa·s (V12) and as high as 2.834 kPa for E12.
TABLE II.
CNN | ||||||||
---|---|---|---|---|---|---|---|---|
Result Code | Hyperparameters | Results | ||||||
Convolutional Layers | Dense Layers | Layer Size | Learning Rate | MAE, kPa | Slope | Intercept | R2 | |
E11 | 3 | 4 | 32 | 0.0005 | 0.051 | 1.000 | −0.001 | 1.000 |
E12 | 2.834 | 0.666 | 4.951 | 0.836 | ||||
E22 | 2 | 2 | 32 | 0.0001 | 0.079 | 1.001 | −0.033 | 1.000 |
E21 | 1.663 | 0.848 | 0.547 | 0.958 | ||||
E1+2 | 4 | 4 | 32 | 0.0005 | 0.065 | 0.998 | 0.021 | 1.000 |
2D FT | ||||||||
E1 | 2.380 | 0.936 | 0.404 | 0.762 | ||||
E2 | 2.211 | 0.901 | 1.252 | 0.775 |
TABLE III.
CNN | ||||||||
---|---|---|---|---|---|---|---|---|
Result Code | Hyperparameters | Results | ||||||
Convolutional Layers | Dense Layers | Layer Size | Learning Rate | MAE, Pa·s | Slope | Intercept | R2 | |
V11 | 4 | 3 | 32 | 0.0001 | 0.065 | 0.997 | 0.007 | 0.999 |
V12 | 1.644 | 0.678 | −0.030 | 0.856 | ||||
V22 | 3 | 2 | 32 | 0.0005 | 0.052 | 0.998 | 0.001 | 0.999 |
V21 | 1.611 | 1.046 | 1.296 | 0.885 | ||||
V1+2 | 3 | 2 | 32 | 0.0001 | 0.069 | 1.003 | −0.015 | 0.999 |
2D FT | ||||||||
V1 | 3.173 | 1.580 | −0.022 | 0.693 | ||||
V2 | 2.925 | 1.593 | −0.240 | 0.695 |
The errors for each CNN model were also aggregated and plotted versus the true value (Fig. 5). The data were reported with boxplots where the red line represents the median, and the edges of the box represent the interquartile range (IQR, 25th and 75th quantiles). The whiskers represent 1.5*IQR above or below the 75th or 25th quantile, respectively. Values denoted by markers outside of whiskers are denoted as outliers. For cases E11, E22, and E1+2, the level of bias was minimal over the range of true shear elasticity. For E12, μ1 was overestimated when the true μ1 < 15 kPa, and underestimated for larger values of μ1. For E21, a consistent underestimation of μ1 was observed. In the cases of V11, V22, and V1+2, the bias was relatively low, where the MAE was less than 0.069 Pa·s. For V12, μ2 was consistently underestimated and for V21 μ2 was consistently overestimated.
When compared to the 2D FT analysis, the CNN results were found to be significantly more reliable. The Fourier analysis showed reduced performance at higher viscosity and lower elasticity combinations, driving elasticity underestimation and viscosity overestimation. In these types of materials, the ARF excitation was not able to generate energy at higher frequencies (300-600 Hz), and therefore it limited the quality of Kelvin-Voigt fitting. Although the linear regression technique was able to retrieve acceptable values for slope and intercept for the elasticity estimations, the errors still drove MAE values above 2.2 kPa, and R2 below 0.78. The viscosity estimations were more inferior to the performance achieved with the CNNs, with slopes above 1.5, MAE above 2.9 Pa·s and R2 below 0.7.
The errors were also plotted versus vertical position in Fig. 6. Similar trends were observed for these additional parameters as compared to those discussed above. The prediction of μ1 was largely insensitive to the vertical position of the wave motion used, though the results for model E1+2 exhibited more outliers. For E12, the bias is positive (average of 0.14 kPa), and 1.5*IQR of over 0.63 kPa showing a very large variability, but insensitive versus vertical position. For E21, there was a larger negative bias for data from the pre-focal locations (−0.27 kPa at −0.90 cm and −0.1 kPa at the focus), while larger numbers of outliers for post-focal datasets. For V11, V12, and V1+2, the bias was small, but the number of outliers slightly increased for the V1+2 case. The errors in μ2 for V12 were largely insensitive to the position, though for V21 the error increased both in bias and variability as the data farther from the focus was used, with 0.1 ± 0.18 kPa at the focus, 0.15 ± 0.22 Pa·s at −0.90 cm and 0.28 ± 21 Pa·s at 0.90 cm. When compared to the 2D FT analysis, the CNN is more reliable with respect to the depth position, when trained with the proper f-number. The same sensitivity to off-focus position was also observed using the 2D FT analysis for F/N = 1 (Fig. 6 – E1-2D FT and V1-2D FT).
The errors for the models as a function of noise level are shown in Fig. 7. The model results for the elasticity prediction were not largely affected by the level of noise. The cases of E1+2 and E21 did demonstrate a larger number of outliers. The models trained/validated and tested with different f-numbers showed bias invariance to the noise levels, nevertheless, E11 showed an increase of 21% and 82% in 1.5*IQR from 20 dB to 10 dB and 10 dB to 5 dB, respectively, while V11 showed an increase of 60% and 38% in 1.5*IQR from 20 dB to 10 dB and 10 dB to 5 dB, respectively. The results for model V1+2 showed more outliers than V11 and V22 models. The 2D FT analysis showed a larger number of outliers, when compared with the CNN. While the number of outliers were relatively constant, the 2D FT results were similar across different levels of noise.
For models trained, validated and tested with both f-numbers, the errors were also compiled by f-number (Fig. 8). No relevant difference was found between the errors for both elasticity and viscosity. Because all the data is aggregated, there are a high number of outliers present, but the error has a similar overall range for all models and F/N values (Tables II and III). The outliers cover a similar range for both F/N values for V1+2. Models trained and tested on the same f-numbers generally showed very low absolute errors with no relevant bias and standard deviation of less than 0.5 kPa or 0.5 Pa·s throughout the true value range.
Models E21 and V12 showed a pattern of higher bias and IQR with the increase of true value, whereas E12 and V21 showed more complex patterns of error across the true value range. In some cases, even with the overestimation margin applied to the label normalization, the higher value estimations were saturated and drove the bias negatively, e.g. E11 and V11. It is also notable that model V1+2 presented a higher level of outliers and positive bias at 0 Pa·s.
IV. Discussion
The results shown are encouraging and exhibit the potential of the use of convolutional neural networks in ultrasound SWE for viscoelastic characterization. The results demonstrated that the CNN models were robust to noise, vertical position and partially to F/N. When tested on datasets with the same excitation F/N for training and testing, the elasticity and viscosity estimations were very good and showed robustness to noise and vertical position.
The CNN models were marginally reliable when presented with test data from a different F/N than used for the training, with a trend of overestimation for E12 and V21, and underestimation for E21 and V12. Vertical position showed some level of variability in models trained with data using F/N = 2 and tested on data with F/N = 1 (X21), possibly due to the planar characteristics of higher F/Ns, when compared to a lower F/N, which produced more curved wave fronts.
Higher f-numbers have less focused beams that might cause wider lateral shear wave profiles that could simulate the widening effect of viscosity in the wave propagation. When trained on data using F/N = 1 and tested on data when F/N = 2 (X12), the results showed a consistent underestimation profile increasing linearly with the true viscosity (V12, Fig. 5), while the opposite effect occurred with V21, showing consistent viscosity overestimation. The V21 overestimation is exaggerated with the distance from the focus, as F/N = 1 has a more curved intensity profile to produce the wave front while F/N = 2 has a more planar wave front (Fig. 2). This behavior is well exemplified in Figs. 6 and 9. This characteristic is important because it suggests the possibility of reliable estimation correction for cases of F/N for which the model was not trained.
When different F/N values were used, the elasticity estimations showed the opposite behavior from the viscosity, overestimation for E12 and underestimation for E21. It is possible that the more curved intensity profile in the z-direction used for the ARF push beam, and the presence of more pre-focal and post-focal intensity for F/N = 1 also leads to errors in elasticity estimation, as it might change the perceived slope of the wave motion (Fig. 10).
Similarly to viscosity, the elasticity showed correlation between absolute bias and the distance to push focus when trained with F/N = 2 and tested on F/N = 1 (X21). Models trained with F/N = 2 are tuned to the planar wave front characteristics, therefore, when presented with data from a lower F/N for testing, show higher levels of error off-center (Fig. 6, V21 and E21).
Noise did not affect the estimations significantly in any of the scenarios tested, which suggests the robustness of the technique to SNR changes.
When trained with both f-numbers the CNNs were able to overcome the shortcomings discussed above and performed similarly as the pairs trained, validated, and tested with a single F/N. The results indicate that the CNNs can discern between wave front profiles with high accuracy, as long as they are trained properly.
Overall, this proof-of-concept study showed that the CNN models were able to successfully retrieve the viscoelastic properties from the wave motion images. The approach proposed required minimal preprocessing, without the employment of filters or transforms. The conventional 2D FT analysis approach showed inferior performance on the same datasets, with worse MAE and R2 levels found compared to those found with CNN regression, even in cases where the CNN was trained and tested with different f-numbers. The 2D FT analysis was especially affected by materials where the effects of the viscosity was more prominent compared to the elasticity (e.g., high μ2 and low μ1), whereas the CNN did not show the same behavior and was reliable over the full viscoelastic parameter range. As noted in the Results, in the cases where the viscosity effects were substantial, energy at higher frequencies was diminished and curve fitting was more error prone. These findings with the CNN models are particularly encouraging, because changes in viscoelastic properties are related to pathological changes to the tissue, e.g. fibrotic tissue is correlated with higher μ1 [10], [16]. Therefore, patients and practitioners can benefit from ML classification of tissue state based on SWE, which is an affordable and non-invasive measurement.
Although encouraging, the methods discussed still need further investigation. The method needs to be extended to allow for heterogeneous cases and other noise characteristics. The practical use of ML in ultrasound SWE has specific challenges such as the noise due to ultrasound detection and physiological motion, the standardization of images, and simulation model limitations that will have to be addressed in future work.
In this study, we used simulated motion from the SGFD based on a Kelvin-Voigt rheological model. Although it has been shown in literature, in multiple works, that the Kelvin-Voigt model does describe shear wave velocity dispersion over the typical elastography analysis ranges of frequency, such as 100-600 Hz [39]-[42], the use of the KV model on SGFD simulations might impair the use of simulations for ex vivo and in vivo transfer learning as not all tissues may be well-described by a KV rheological model. We also chose to use the 2D x-t wave motion images to directly estimate the values of μ1 and μ2. Other ML approaches or NN architectures could be utilized to estimate the dispersion curves for fitting to different rheological models.
We did not incorporate the ultrasound detection in these simulated data, which may add additional bias, but this proof-of-concept approach shows the promise of using shear wave motion and the overall shape of the wave fronts to estimate the viscoelastic properties. We did explore adding Gaussian noise to the wave motion data, and the models were largely robust to different noise levels. Higher levels of noise could be explored in future work. Physiological motion will also introduce noise as well that is not accounted for in this work. Each of the images used for these models used a fixed range of lateral space and measurement time. For in vivo implementation, this would have to be standardized for a given application or transducer. We also limited the range of depths over which we made the evaluations. Typically, viscoelastic measurements are made within or near the depth-of-focus of the ARF push beams, but an evaluation of parameter estimation versus depth will be investigated in our ongoing work.
Ultrasound SWE implementations have a wide variety of characteristics, such as F/N, transducer profile, motion detection approaches, all of which with specific use cases, strengths and weaknesses, and therefore, need to be evaluated separately. Ideally, for in vivo implementation, an acquisition approach with the aforementioned parameters would need to be thoroughly modeled and likely used in the training and validation and later testing for a robust technique. The entire pipeline including ARF push beam modeling, wave motion modeling with different material models, ultrasound detection, and motion estimation and filtering would need to be incorporated to all for transfer learning with experimental data from phantoms or ex vivo and in vivo soft tissues. This work represents a first step to demonstrate the promise of this type of approach as well as identify some of the next steps to be taken in this technology development.
V. Conclusions
In this study, we proposed to evaluate the capabilities of CNNs to model wave propagation and retrieve elasticity and viscosity values from wave motion images sourced from staggered-grid finite-difference (SGFD) simulations. A total of 1050 SGFD models were simulated using a wide range of elasticity and viscosity values for two different values of excitation F/N. The dataset was then augmented using five different vertical positions to generate the spatiotemporal images and using three different levels of noise. The dataset was used to train, validate and test CNN architectures configured for data regression using mean squared error as loss function. The viscoelastic parameters were also estimated using the conventional 2D Fourier analysis approach with Kelvin-Voigt dispersion curve fitting and compared against the CNN results.
The overall results showed the CNNs’ potential to be an alternative to complex mathematical analyses such as Fourier analysis and dispersion curve estimation used currently for shear wave viscoelastic parameter estimation. The CNN approach does not impose the biases and variabilities associated with these methods and requires minimal amount of preprocessing, such as filters and transforms. The methods proposed might enable simpler and more reliable non-invasive evaluation of tissue injuries that alter rheological parameters such as liver and kidney fibrosis. The use of transfer learning is also a possibility to enable ex vivo and in vivo viscoelastic estimations [43], and this will be pursued in ongoing work.
More evaluations regarding other f-numbers are necessary, although higher values will yield even more planar wave fronts and should provide easier translation to other CNN models. The initial robustness to noise was also encouraging as it is a strong characteristic of ultrasound applications, but other noise settings need to be evaluated. This work serves as a proof-of-concept to demonstrate the use of CNN models for estimation of viscoelastic properties directly from shear wave motion. We will continue to develop this technology to test on experimental data from phantoms and soft tissues.
Ultrasound shear wave elastography can measure mechanical properties of tissues
Staggered-grid finite difference simulations of viscoelastic media were used
Convolutional Neural Net architectures were optimized using mean squared error loss
The CNN architectures were able to retrieve accurate elasticity and viscosity values
Robust results with different push profiles are obtained if CNNs are trained properly
Acknowledgments
This work was supported in part by the Ministry of Science and Higher Education of Poland under agreement no. 0177/E-356/STYP/13/2018. The work was also supported in part by grant R01DK092255, from the National Institutes of Health. The content is solely the responsibility of authors and does not necessarily represent the official views of the National Institute of Diabetes and Digestive and Kidney Diseases or the National Institutes of Health.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Conflict of Interest Statement
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References
- [1].Gennisson J-L, Deffieux T, Fink M, and Tanter M, “Ultrasound elastography: Principles and techniques,” Diagnostic and Interventional Imaging, vol. 94, no. 5, pp. 487–495, May 2013, doi: 10.1016/j.diii.2013.01.022. [DOI] [PubMed] [Google Scholar]
- [2].Urban MW, “Production of acoustic radiation force using ultrasound: methods and applications,” Expert Rev Med Devices, vol. 15, no. 11, pp. 819–834, 2018, doi: 10.1080/17434440.2018.1538782. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [3].Shiina T et al. , “WFUMB guidelines and recommendations for clinical use of ultrasound elastography: Part 1: basic principles and terminology,” Ultrasound Med Biol, vol. 41, no. 5, pp. 1126–1147, May 2015, doi: 10.1016/j.ultrasmedbio.2015.03.009. [DOI] [PubMed] [Google Scholar]
- [4].Montaldo G, Tanter M, Bercoff J, Benech N, and Fink M, “Coherent plane-wave compounding for very high frame rate ultrasonography and transient elastography,” IEEE Trans Ultrason Ferroelectr Freq Control, vol. 56, no. 3, pp. 489–506, March. 2009, doi: 10.1109/TUFFC.2009.1067. [DOI] [PubMed] [Google Scholar]
- [5].Song P et al. , “Two-dimensional shear-wave elastography on conventional ultrasound scanners with time-aligned sequential tracking (TAST) and comb-push ultrasound shear elastography (CUSE),” IEEE Trans Ultrason Ferroelectr Freq Control, vol. 62, no. 2, pp. 290–302, February. 2015, doi: 10.1109/TUFFC.2014.006628. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Chen S, Fatemi M, and Greenleaf JF, “Quantifying elasticity and viscosity from measurement of shear wave speed dispersion,” J. Acoust. Soc. Am, vol. 115, no. 6, pp. 2781–2785, June. 2004, doi: 10.1121/1.1739480. [DOI] [PubMed] [Google Scholar]
- [7].Catheline S et al. , “Measurement of viscoelastic properties of homogeneous soft solid using transient elastography: An inverse problem approach,” The Journal of the Acoustical Society of America, vol. 116, no. 6, pp. 3734–3741, December. 2004, doi: 10.1121/1.1815075. [DOI] [PubMed] [Google Scholar]
- [8].Chen S et al. , “Shearwave dispersion ultrasound vibrometry (SDUV) for measuring tissue elasticity and viscosity,” IEEE Trans Ultrason Ferroelectr Freq Control, vol. 56, no. 1, pp. 55–62, January. 2009, doi: 10.1109/TUFFC.2009.1005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [9].Deffieux T, Montaldo G, Tanter M, and Fink M, “Shear wave spectroscopy for in vivo quantification of human soft tissues viscoelasticity,” IEEE Trans Med Imaging, vol. 28, no. 3, pp. 313–322, March. 2009, doi: 10.1109/TMI.2008.925077. [DOI] [PubMed] [Google Scholar]
- [10].Nightingale KR et al. , “Derivation and analysis of viscoelastic properties in human liver: impact of frequency on fibrosis and steatosis staging,” IEEE Trans Ultrason Ferroelectr Freq Control, vol. 62, no. 1, pp. 165–175, January. 2015, doi: 10.1109/TUFFC.2014.006653. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [11].Nenadic IZ et al. , “Attenuation Measuring Ultrasound Shearwave Elastography and in vivo application in post-transplant liver patients,” Phys Med Biol, vol. 62, no. 2, pp. 484–500, January. 2017, doi: 10.1088/1361-6560/aa4f6f. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [12].Bernard S, Kazemirad S, and Cloutier G, “A Frequency-Shift Method to Measure Shear-Wave Attenuation in Soft Tissues,” IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, vol. 64, no. 3, pp. 514–524, March. 2017, doi: 10.1109/TUFFC.2016.2634329. [DOI] [PubMed] [Google Scholar]
- [13].Budelli E et al. , “A diffraction correction for storage and loss moduli imaging using radiation force based elastography,” Phys Med Biol, vol. 62, no. 1, pp. 91–106, July 2017, doi: 10.1088/1361-6560/62/1/91. [DOI] [PubMed] [Google Scholar]
- [14].Kijanka P and Urban MW, “Local Phase Velocity Based Imaging: A New Technique Used for Ultrasound Shear Wave Elastography,” IEEE Trans Med Imaging, vol. 38, no. 4, pp. 894–908, 2019, doi: 10.1109/TMI.2018.2874545. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [15].Kijanka P and Urban MW, “Two-Point Frequency Shift Method for Shear Wave Attenuation Measurement,” IEEE Trans Ultrason Ferroelectr Freq Control, vol. 67, no. 3, pp. 483–496, 2020, doi: 10.1109/TUFFC.2019.2945620. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [16].Chen X et al. , “Quantification of liver viscoelasticity with acoustic radiation force: a study of hepatic fibrosis in a rat model,” Ultrasound Med Biol, vol. 39, no. 11, pp. 2091–2102, November. 2013, doi: 10.1016/j.ultrasmedbio.2013.05.020. [DOI] [PubMed] [Google Scholar]
- [17].Palmeri ML, Wang MH, Dahl JJ, Frinkley KD, and Nightingale KR, “Quantifying hepatic shear modulus in vivo using acoustic radiation force,” Ultrasound Med Biol, vol. 34, no. 4, pp. 546–558, April. 2008, doi: 10.1016/j.ultrasmedbio.2007.10.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [18].Urban MW, Chen S, and Greenleaf JF, “Error in Estimates of Tissue Material Properties from Shear Wave Dispersion Ultrasound Vibrometry,” IEEE Trans Ultrason Ferroelectr Freq Control, vol. 56, no. 4, pp. 748–758, April. 2009, doi: 10.1109/TUFFC.2009.1097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [19].Deffieux T, Gennisson J, Larrat B, Fink M, and Tanter M, “The variance of quantitative estimates in shear wave imaging: Theory and experiments,” IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, vol. 59, no. 11, pp. 2390–2410, November. 2012, doi: 10.1109/TUFFC.2012.2472. [DOI] [PubMed] [Google Scholar]
- [20].Deng Y, Rouze NC, Palmeri ML, and Nightingale KR, “On System-Dependent Sources of Uncertainty and Bias in Ultrasonic Quantitative Shear-Wave Imaging,” IEEE Trans Ultrason Ferroelectr Freq Control, vol. 63, no. 3, pp. 381–393, March. 2016, doi: 10.1109/TUFFC.2016.2524260. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [21].Ahmed S, Kamal U, and Hasan Md. K., “DSWE-Net: A deep learning approach for shear wave elastography and lesion segmentation using single push acoustic radiation force,” Ultrasonics, vol. 110, p. 106283, February. 2021, doi: 10.1016/j.ultras.2020.106283. [DOI] [PubMed] [Google Scholar]
- [22].Treacher A et al. , “Deep Learning Convolutional Neural Networks for the Estimation of Liver Fibrosis Severity from Ultrasound Texture,” Proc SPIE Int Soc Opt Eng, vol. 10950, February. 2019, doi: 10.1117/12.2512592. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [23].Wang K et al. , “Deep learning Radiomics of shear wave elastography significantly improved diagnostic performance for assessing liver fibrosis in chronic hepatitis B: a prospective multicentre study,” Gut, vol. 68, no. 4, pp. 729–741,2019, doi: 10.1136/gutjnl-2018-316204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [24].Zhang Q et al. , “Deep learning based classification of breast tumors with shear-wave elastography,” Ultrasonics, vol. 72, pp. 150–157, 2016, doi: 10.1016/j.ultras.2016.08.004. [DOI] [PubMed] [Google Scholar]
- [25].Pagariya R and Bartere M, “Review Paper on Artificial Neural Networks,” International Journal of Advanced Research in Computer Science, vol. 4, no. 6, 2013, doi: 10.26483/ijarcs.v4i6.1699. [DOI] [Google Scholar]
- [26].Robbins HE, “A Stochastic Approximation Method,” 2007, doi: 10.1214/aoms/1177729586. [DOI] [Google Scholar]
- [27].Kingma DP and Ba J, “Adam: A Method for Stochastic Optimization,” arXiv:1412.6980 [cs], January. 2017, Accessed: Jan. 28, 2020. [Online]. Available: http://arxiv.org/abs/1412.6980. [Google Scholar]
- [28].Rawat W and Wang Z, “Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review,” Neural Computation, vol. 29, no. 9, pp. 2352–2449, June. 2017, doi: 10.1162/neco_a_00990. [DOI] [PubMed] [Google Scholar]
- [29].Kijanka P and Urban MW, “Dispersion curve calculation in viscoelastic tissue-mimicking materials using non-parametric, parametric, and high-resolution methods,” Ultrasonics, vol. 109, p. 106257, January. 2021, doi: 10.1016/j.ultras.2020.106257. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [30].Hustedt B, Operto S, and Virieux J, “Mixed-grid and staggered-grid finite-difference methods for frequency-domain acoustic wave modelling,” Geophys J Int, vol. 157, no. 3, pp. 1269–1296, June. 2004, doi: 10.1111/j.1365-246X.2004.02289.x. [DOI] [Google Scholar]
- [31].Zeng YQ and Liu QH, “A staggered-grid finite-difference method with perfectly matched layers for poroelastic wave equations,” The Journal of the Acoustical Society of America, vol. 109, no. 6, pp. 2571–2580, June. 2001, doi: 10.1121/1.1369783. [DOI] [PubMed] [Google Scholar]
- [32].Virieux J, “P-SV wave propagation in heterogeneous media: Velocity-stress finite-difference method,” GEOPHYSICS, vol. 51, no. 4, pp. 889–901, April. 1986, doi: 10.1190/1.1442147. [DOI] [Google Scholar]
- [33].Chen D and McGough RJ, “A 2D fast near-field method for calculating near-field pressures generated by apodized rectangular pistons,” J Acoust Soc Am, vol. 124, no. 3, pp. 1526–1537, September. 2008, doi: 10.1121/1.2950081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [34].Zeng X and McGough RJ, “Evaluation of the angular spectrum approach for simulations of near-field pressures,” J Acoust Soc Am, vol. 123, no. 1, pp. 68–76, January. 2008, doi: 10.1121/1.2812579. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [35].Drossaert FH and Giannopoulos A, “A nonsplit complex frequency-shifted PML based on recursive integration for FDTD modeling of elastic waves,” GEOPHYSICS, vol. 72, no. 2, pp. T9–T17, January. 2007, doi: 10.1190/1.2424888. [DOI] [Google Scholar]
- [36].Kijanka P, Qiang B, Song P, Amador Carrascal C, Chen S, and Urban MW, “Robust Phase Velocity Dispersion Estimation of Viscoelastic Materials Used for Medical Applications Based on the Multiple Signal Classification Method,” IEEE Trans Ultrason Ferroelectr Freq Control, vol. 65, no. 3, pp. 423–439, 2018, doi: 10.1109/TUFFC.2018.2792324. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [37].Amador Carrascal C, Chen S, Manduca A, Greenleaf JF, and Urban MW, “Improved Shear Wave Group Velocity Estimation Method Based on Spatiotemporal Peak and Thresholding Motion Search,” IEEE Trans Ultrason Ferroelectr Freq Control, vol. 64, no. 4, pp. 660–668, April. 2017, doi: 10.1109/TUFFC.2017.2652143. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [38].Chollet F, Keras. GitHub, 2015. [Google Scholar]
- [39].Chen S et al. , “Assessment of liver viscoelasticity by using shear waves induced by ultrasound radiation force,” Radiology, vol. 266, no. 3, pp. 964–970, March. 2013, doi: 10.1148/radiol.12120837. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [40].Amador C, Urban M, Kinnick R, Chen S, and Greenleaf JF, “In vivo swine kidney viscoelasticity during acute gradual decrease in renal blood flow: pilot study,” Rev Ing Biomed, vol. 7, no. 13, pp. 68–78, January. 2013. [PMC free article] [PubMed] [Google Scholar]
- [41].Amador C, Kinnick RR, Urban MW, Fatemi M, and Greenleaf JF, “Viscoelastic tissue mimicking phantom validation study with shear wave elasticity imaging and viscoelastic spectroscopy,” in 2015 IEEE International Ultrasonics Symposium (IUS), Oct. 2015, pp. 1–4, doi: 10.1109/ULTSYM.2015.0287. [DOI] [Google Scholar]
- [42].Aristizabal S, Amador C, Qiang Bo, Nenadic IZ, Greenleaf JF, and Urban MW, “Viscoelastic characterization of transverse isotropic tissue mimicking phantoms and muscle,” in 2014 IEEE International Ultrasonics Symposium, Sep. 2014, pp. 228–231, doi: 10.1109/ULTSYM.2014.0058. [DOI] [Google Scholar]
- [43].Weiss K, Khoshgoftaar TM, and Wang D, “A survey of transfer learning,” J Big Data, vol. 3, no. 1, p. 9, May 2016, doi: 10.1186/s40537-016-0043-6. [DOI] [Google Scholar]