Episode 031: Is your out of sample testing even valid?

Strategy validation is a crucial part of live trading.

Get it wrong and the results can be disastrous.

Traders rely on various techniques to validate strategies, including out of sample testing, but how do we know if the approaches we’re using are even valid?

In today’s podcast episode we discuss a series of questions submitted by trader Sam.

Sam raises a number of interesting points about out of sample testing that all traders need to consider, including:

How much out of sample data is sufficient to identify a stable relationship,
How do you know that what worked in Out of Sample testing will continue to work,
Is it even possible to avoid curve-fitting,
Why you should NEVER use trading techniques as crutches,
Plus much more.

You can listen to it now.

Happy trading,

Tomas and Andrew

“Even a small improvement is an improvement.” - Tomas Nesnidal Share on X “The truth is, we’re always curve fitting. Curve fitting isn’t an evitable thing in trading and it’s not always a bad thing.” - Tomas Nesnidal Share on X “In trading, everything is a statistical probability, not a statistical certainty. So, you need to learn how to live with this uncertainty.” - Tomas Nesnidal Share on X “Your foundation always needs to be super strong and go through the most demanding robustness testing procedures you can imagine.” - Tomas Nesnidal Share on X

Resources:

Tired of unexpected losses? Find out how to start smashing frustrating false breakouts here.