Leave-one-race-out validation
A way of testing a model in which each race is predicted by a model that never saw that race in training.
To score race N, the model is trained on every other race and then asked to predict N. Repeat for every race. The result is an honest estimate of how the model performs on races it has not seen, which is the only performance that matters.
RaceModel's published backtest is validated this way, race by race. It is slower than a single train/test split but it removes the temptation to pick a favourable split.