SIMPLE AND ROBUST MULTIVARIATE ESTIMATOR FOR COMPLEX SURVEYS

SIMPLE AND ROBUST MULTIVARIATE ESTIMATOR FOR COMPLEX SAMPLE SURVEYS WITH HIGH DIMENSIONS

Submitted to the Journal of Survey Statistics and Methodology 

JSSAM-2025-138 

August 18, 2025

DOI: 10.13140/RG.2.2.30545.44641

This article introduces a new category of model‑dependent methods: the Generalized Multivariate Difference Estimator (GMDe). GMDe is a multivariate composite estimator. It minimizes the variance of the population estimate for each study variable by gaining strength from population estimates or census figures for auxiliary variables. The degree of variance reduction depends, in part, on the strength of the correlations between the auxiliary variables and each study variable. GMDe is unbiased even with a misspecified model, while inequality constraints mitigate outliers and reduce overfitting.

GMDe is a simple estimator for complex designs, including multi-phase and multi-stage sampling, rotating panels, repeated measures, supplemental sampling frames and found data. High dimensional vectors of population estimates, for both study variables and auxiliary variables, can include partitions for multiple time periods, domains, and small areas. A chained version of GMDe is a simple method for multi level designs. Auxiliary variables can include predictions from deterministic models, which reduce variances and smooth time series estimates; and analyses of residuals reveal unexpected deviations of model predictions from design based population estimates. Pseudo estimators compute sums, differences, products, and ratios between GMDe population estimates. The multivariate structure improves statistical efficiency and simplifies variance estimators.

A hybrid version of GMDe forms a new class of model-assisted estimators. Parameterization of the optimal GMDe coefficient matrix uses the design based estimate of the population covariance matrix between the study variables and auxiliary residuals. This requires a matrix inversion that can be infeasible, especially with high dimensions; but a recursive GMDe algorithm reliably computes a matrix pseudo inverse, including efficient stepwise selection among auxiliary variables. 

GMDe anticipates a future that increasingly requires simple, robust, flexible, and adaptable methods for complex sample surveys that assimilate high dimensional auxiliary data from unconventional sources. These attributes have strong appeal to developers of sample surveys and users of statistical products.

The Generalized Multivariate Difference estimator (GMDe) is a new paradigm in model dependent estimation methods. It is fundamentally different than conventional model assisted and model based estimators. GMDe is an optimal multivariate generalization of the classic composite estimator. Its multivariate structure simplifies variance estimators, and it is more efficient than univariate estimators. GMDe is adaptable to changes in the sampling design, analysis priorities, auxiliary variables, measurement technologies, population processes, and funding. The simplicity and robustness of GMDe reduces risks with sample survey designs that were previously considered impractical because of their complexity.

The Generalized Multivariate Difference Estimator is a new estimation paradigm. It is an innovative alternative to conventional model assisted and model based estimators. GMDe does not adjust PSU sampling weights, which makes it simpler than other estimators for complex sample surveys. 

GMDe assimilates high dimensional time series of auxiliary information from diverse sources, including administrative records, remote sensing, model predictions, supplemental sampling frames, and found data; and GMDe readily adapts to changes in those sources over time. 

GMDe relaxes the minimum variance criterion to apply simple equality and inequality constraints, none of which require analyst intervention; constraints reduce risks from model misspecification, data anomalies and overfitting. 

The model assisted version of GMDe uses design based survey estimates to parameterize the optimal coefficient matrix. GMDe is a linear function that simplifies variance estimation in complex survey designs. 

As unconventional sources of auxiliary data proliferate, as demand for more frequent, detailed, and granular information grows; and as resources to conduct surveys become more limited; the need for alternatives to conventional methods will increase. 

GMDe enables statisticians to confidently adopt more complex and adaptable survey designs that fulfill this need. 

GMDe is simple, robust, and efficient, which has strong appeal to developers of sample surveys and users of the resulting statistics.