A Case Study Exploring Data Synthesis Strategies on Tabular vs. Aggregated Data Sources for Official Statistics

Abstract

In this paper, we investigate different approaches for generating synthetic microdata from open-source aggregated data. Specifically, we focus on macro-to-micro data synthesis. We explore the potential of the Gaussian copulas framework to estimate joint distributions from aggregated data. Our generated synthetic data is intended for educational and software testing use cases. We propose three scenarios to achieve realistic and high-quality synthetic microdata: (1) zero knowledge, (2) internal knowledge, and (3) external knowledge. The three scenarios involve different knowledge of the underlying properties of the real microdata, i.e., standard deviation, and covariate. Our evaluation includes matching tests to evaluate the privacy of the synthetic datasets. Our results indicate that macro-to-micro synthesis achieves better privacy preservation compared to other methods, demonstrating both the potential and challenges of synthetic data generation in maintaining data privacy while providing useful data for analysis.

The views expressed in this paper are those of the authors and do not necessarily reflect the policy of Statistics Netherlands.

A Appendix

1.1 A.1 Dataset Rules

We explain the types of relationships present in the dataset and the rules the synthesizer must account for. We divide the rules into five categories: non-negative rules, summation rules, equality rules, inequality rules, and if-then rules. The non-negative rule imposes that the variable must be greater or equal to zero (e.g. number of workers and various expenses). For the summation rule, the sum of certain variables must equal another variable per definition. Think of the total expense variable being the sum of individual expenses, or the net revenue being a summation of sales and costs. The equality rule suggests that specific variables must be equal to another variable ($=$), while the inequality rule imposes that some variables must be lesser or equal ($\le$) or greater or equal ($\ge$) than other variables. This is the case of repeated variables in the survey (equality rules), and the variable of the number of workers being greater or equal to the full-time equivalent workers (inequality rule). The if-then rules impose the same rules as the previous ones, with the addition of requiring a certain condition to be satisfied (as some rules can only exist if a variable meets a certain value).

1.2 A.2 Extra Analytical Validity Results

For micro-to-micro data synthesis, we measure the KS-Complement metric, which assesses the similarity between a real variable and its synthetic counterpart based on their marginal distributions. Our results are provided in Tables 3 and 4 (Fig. 5).

Table 3. KS-Complements macro-to-micro scaled data

Table 4. KS-Complements macro-to-micro

Fig. 5.

Univariate distribution of the gk_SBS variable

1.3 A.3 Residuals from the Regression Models

We analyze the performance of the synthetic data by examining the residuals of the linear regression model. Figure 6 shows the residual plots for the models trained on the real data, CART synthetic data, and CTGAN synthetic data. The residuals represent the differences between the observed and predicted values.

In the residual plot for the model trained on the real data (Fig. 6a), the residuals are evenly distributed around zero, indicating a good fit of the model. In contrast, the residual plot for the CART synthetic data (Fig. 6b) shows a similar distribution, though with a slightly larger variance, suggesting that the CART synthetic data maintains the relationships in the real data fairly well. The residual plot for the CTGAN synthetic data (Fig. 6c) reveals a different pattern. The residuals are more spread out and exhibit a clear upward trend, indicating that the CTGAN model has a downward bias and struggles to capture the variability in the real data. This pattern suggests that the CTGAN synthetic data may not be as effective in preserving the underlying relationships present in the real data. Overall, these residual plots highlight that the CART synthetic data provides a better approximation of the real data compared to the CTGAN synthetic data, aligning with the findings from the prediction accuracy analysis.

Fig. 6.

Residual plot, with the prediction model trained on the real and micro-to-micro synthetic datasets

The residual plots for the synthetic data generated using the macro-to-micro approaches are shown in Fig. 7. In Scenario 1 (Fig. 7a), where no knowledge of the real data was used, the residuals are widely scattered and show a large deviation from zero. This indicates poor accuracy in the synthetic data, as the model fails to capture the true relationships present in the real data. Scenario 2 (Fig. 7b), which incorporates internal knowledge, shows an improvement in the residuals compared to Scenario 1. The residuals are closer to zero, suggesting that the synthetic data better approximates the real data, although there are still noticeable deviations. Scenario 3 (Fig. 7c), which leverages external knowledge, exhibits the most favorable residuals among the three scenarios. The residuals are tightly clustered around zero, indicating a higher accuracy in the synthetic data. This demonstrates that utilizing external insights significantly enhances the quality of the synthetic data generation process.

Fig. 7.

Residual plot on the synthetic data generated using macro-to-micro scenarios with scaled data.