Debiased and targeted machine learning (ML) estimators have grown increasingly popular for their ability to incorporate flexible, data-adaptive nuisance parameter estimation while providing valid inference for a target estimand. However, for many common estimands, the finite-sample behavior of debiased ML estimators can be unstable under certain data-generating mechanisms - a consequence, for instance, of inverse propensity or density weighting. Targeted learning estimators can offer improved finite-sample performance, but not uniformly so; they share identical first-order asymptotic behavior with debiased ML, and for certain estimands require iterative algorithms that may fail to converge. Through a careful study of these instabilities, we identify a weakness of debiased ML estimators, for which we propose a general remedy, termed stabilized debiased ML. It introduces regularization precisely where erratic behavior arises. The regularization is designed to shrink at a sufficiently fast rate to leave first-order asymptotic properties intact under conditions that will be discussed. The approach applies to any estimand, including infinite-dimensional target parameters. It delivers competitive empirical performance with standard debiased and targeted ML estimators in information-rich settings, but substantial reductions in variance and mean squared error with negligible impact on bias - often reducing finite-sample bias as well - in settings where standard debiased and targeted ML estimators struggle, such as with extreme inverse propensity score weights.
