Scaling Conditional Autoencoders for Portfolio Optimization via Uncertainty-Aware Factor Selection
Abstract
Conditional Autoencoders (CAEs) offer a flexible, interpretable approach for estimating latent asset-pricing factors from firm characteristics. However, existing studies usually limit the latent factor dimension to around K=5 due to concerns that larger K can degrade performance. To overcome this challenge, we propose a scalable framework that couples a high-dimensional CAE with an uncertainty-aware factor selection procedure. We employ three models for quantile prediction: zero-shot Chronos, a pretrained time-series foundation model (ZS-Chronos), gradient-boosted quantile regression trees using XGBoost and RAPIDS (Q-Boost), and an I.I.D bootstrap-based sample mean model (IID-BS). For each model, we rank factors by forecast uncertainty and retain the top-k most predictable factors for portfolio construction, where k denotes the selected subset of factors. This pruning strategy delivers substantial gains in risk-adjusted performance across all forecasting models. Furthermore, due to each model's uncorrelated predictions, a performance-weighted ensemble consistently outperforms individual models with higher Sharpe, Sortino, and Omega ratios.
Summary
This paper addresses the challenge of scaling Conditional Autoencoders (CAEs) for portfolio optimization. Traditional CAEs are limited to a small number of latent factors (K≈5) to avoid performance degradation. The authors propose a novel framework that combines high-dimensional CAEs (up to K=50) with an uncertainty-aware factor selection procedure. They employ three models for quantile prediction: zero-shot Chronos (ZS-Chronos), gradient-boosted quantile regression trees (Q-Boost), and an I.I.D bootstrap-based sample mean model (IID-BS). For each model, factors are ranked by forecast uncertainty, and the most predictable subset is selected for portfolio construction. This pruning strategy significantly improves risk-adjusted performance. Furthermore, a performance-weighted ensemble of these uncorrelated models consistently outperforms individual models, achieving higher Sharpe, Sortino, and Omega ratios. The methodology involves a two-stage process: first, extracting latent factor portfolios using a CAE trained on firm-level characteristics and asset returns. Second, forecasting the time series of each latent factor using the three models, with forecast uncertainty guiding factor selection. The CAE models asset returns as a function of firm characteristics and latent factors. The parameters are estimated by minimizing the cross-sectional pricing loss. The forecasting models generate point and quantile predictions, and forecast uncertainty is defined as the average absolute deviation of quantile forecasts from the central prediction. The most predictable factors are used to construct a tangency portfolio, which is then mapped to asset weights. The empirical analysis uses monthly returns data for the 2000 largest US stocks from February 1962 through December 2024. The results demonstrate that uncertainty-aware factor selection leads to substantial improvements in portfolio performance, and an ablation study validates the ZS-Chronos model's performance by ensuring no data leakage.
Key Insights
- •The paper introduces a novel uncertainty-aware factor selection framework that integrates high-dimensional latent portfolios extracted via CAEs with predictive signals from multiple advanced forecasting models (ZS-Chronos, Q-Boost, IID-BS).
- •High-dimensional CAEs, when coupled with forecast-driven factor selection, significantly outperform conventional low-dimensional factor models, achieving higher Sharpe, Sortino, and Omega ratios while maintaining maximum drawdowns below 10%. For example, Ensemble (B) achieves a Sortino ratio of 4.22 and an Omega ratio of 5.43.
- •Forecasts generated by zero-shot pretrained models such as Chronos and quantile gradient-boosted regression trees provide complementary predictive signals that substantially augment the baseline IID-based predictions.
- •A performance-weighted ensemble of the three forecasting models consistently outperforms individual models, demonstrating the benefit of combining diverse predictive signals. Ensemble (A) achieves a Sharpe ratio of 2.20.
- •The authors address the practical limitation of assuming future optimality of the number of latent factors by introducing an adaptive selection procedure (adaptive k*) based on a temporally regularized log-sum-exp optimization of the Sortino ratio.
- •An ablation study is performed to rule out any data leakage concerns related to the pretrained ZS-Chronos model, confirming that its performance is not driven by hidden biases. The model is evaluated using synthetic latent factors specifically generated within the framework.
- •The framework establishes a theoretical foundation for uncertainty-based selection, where predictive uncertainty acts as a sufficient statistic for the expected degradation of portfolio utility under model misspecification.
Practical Implications
- •The research provides a practical framework for portfolio optimization that can be implemented by financial institutions and investment managers.
- •The use of uncertainty-aware factor selection can improve the risk-adjusted performance of portfolios by focusing on the most predictable factors. Practitioners can use the proposed methodology to construct more robust and efficient portfolios.
- •The paper highlights the potential of using pre-trained time-series foundation models (like Chronos) in financial forecasting and portfolio optimization. This encourages further exploration and application of such models in finance.
- •The adaptive factor selection procedure allows for dynamic adjustment of the number of latent factors based on market conditions and past performance, making the framework more practical and adaptable.
- •Future research can focus on exploring other forecasting models and uncertainty quantification techniques, as well as applying the framework to different asset classes and investment strategies.