Backtesting Your Trading Strategy
There are a number of challenges with backtesting, which need to be addressed for reliable results.
Survivorship bias basically means you’re basing results on something that may not have existed back then. Let’s assume that you want to trade the All Ordinaries, in Australia –which is the top five hundred stocks. The All Ordinaries today is very different from what the All Ordinaries was back in the late 90’s. Back then you had HIH, that’s gone bankrupt. You’ve got to include HIH in your testing to see how your strategy would have done. Different strategies are going to operate in different ways. For example, most stocks that delist in Australia, tend to trail down for a sustained period of time. We discuss this in ‘Unholy Grails’. Stock after stock that ends up going bankrupt start by trending down. HIH was in a downtrend for a good year and a half. A trend following strategy won’t tend to be involved with those kinds of stocks, but you have to put them in your back testing to understand what impact they would have had on the backtest.
Back in the 90’s many of the tech stocks in the U.S. split each year. I think Microsoft split almost every year –year after year. If you build a system that has some kind of price limit around it –say your system says “I only want to trade stocks below fifty dollars”, and if you don’t take those stock splits into account, you will actually get Microsoft in your backtests when in the real world, that would not have been possible. Little things like that make a difference.
We do a lot of testing using CPI in adjusted volume. If you have a strategy that you build today, requiring a minimum turnover of a million dollars –the market today is substantially larger than it was back in 1990. Back in 1990 there may not have been many stocks with that kind of turnover. We’ll go back and set a base date, and then we’ll adjust the volume by the CPI, every year up until the current year.
You can get quite advanced with this stuff. Is it necessary? Well, if your system is relatively robust then less testing is actually needed but it’s always good to have a look at these things.
To learn more about backtesting and validating strategies take our Beginners Guide to Building Trading Systems course.
