Better by design: Why human choices matter for return predictions via machine learning

The paper by Minghui Chen, Matthias Hanauer, and Tobias Kalsbach, titled ‘Design choices, machine learning, and the cross-section of stock returns’, identifies several key design choices researchers have to make when training ML models. For instance, when setting the prediction (target) variable, should the researcher employ the excess return over the risk-free rate or the abnormal return relative to the market? Is it better to use a continuous target variable or are categories, such as outperformers vs. underperformers, preferable? Is it better to train models based on a rolling window that leads to more adaptive models, or are models based on expanding windows superior, thanks to the availability of more training data?

To assess the importance of such choices, the authors identify seven such key design choices and examine all the ensuing possible combinations, resulting in a total of 1,056 ML models. In this way, the study trains each model on a common set of signals (features) for the US stock market and evaluates their out-of-sample performance using hypothetical top-minus-bottom decile portfolios.

Figure 1 reveals that portfolio returns vary substantially across different model designs, with monthly mean returns ranging from 0.13% to 1.98% and annualized Sharpe ratios ranging from 0.08 to 1.82.¹ This variation highlights the substantial impact of human design choices on the performance of ML strategies.

Having documented the substantial variation in the performance of ML models, the study also provides actionable guidance for ML model design:

• Ensembles of ML models typically outperform individual algorithms.
  

• The choice of target variable depends on the investment objective:
  o For identifying relative winners and losers among stocks, predicting stock returns over the market rather than the risk-free rate is better.
  o If the goal is to achieve high market-risk-adjusted returns, CAPM beta-adjusted returns are better.

• Non-linear ML models are more likely to outperform their linear counterparts when:
  o using abnormal returns relative to the market as the target variable,
  o employing continuous target returns, or
  o adopting expanding training windows.

Footnote

¹Please note that these are hypothetical gross returns for long-minus-short strategies that do not consider any transaction costs. We investigated the impact of transaction costs on ML strategies in our study ‘The term structure of machine learning alpha’.