Backtest Results

[ChrisThong]

Hi All,

Below is an issue came up in my backtest recently.

On 5/7/2021, I ran a backtest from 1/1/2005 to 5/7/2021 for my MR strategy, and I got the following results:

1. CAR: 21.02
2. MDD: 17.64
3. MAR: 1.19
4. Trades: 5090

On 9/8/2021, I use the same AFL file and ran a backtest from 1/1/2005 to 5/7/2021 (same period as above), and I got the following results:

5. CAR: 20.67
6. MDD: 19.08
7. MAR: 1.08
8. Trades: 5092

Although the variance are minimal, I wonder why?

I checked the settings in AB and all is good. Perhaps, it is data issue. So, I contacted Norgate Data and they told me that it is not data issue. Further, they suggest that the settings in AB are the culprit.

Has anyone come across issue like the above?

Regards,

Chris

[GlenPeake]

Hi Chris,

Hard to say exactly…. it could be 1 of many different settings/options selected/de-selected etc e.g. selecting “Pad and Align” while running a Rotational system backtest and forgetting to turn it off etc…can skew the trades etc….

You could start by reviewing the REPORTS folder and checking to see the output of each of the backtests from the 2 different days….. there will be a bunch of HTML and other files…. have a look at the “settings.html” of each backtest folder and compare….. and see if maybe there is a difference between the 2 backtest/settings etc….

(Report folder location)
C:Program FilesAmiBrokerReports

If nothing is obvious…. if you have a 2nd PC running Amibroker etc… you could run the backtest on the 2nd PC and see if there’s a difference etc…. this will rule in/out the possibility of a database corruption on you ‘main PC’.

[ChrisThong]

Thanks Glen.

I printed the trade logs of both backtests and cross checked the trades. I did not check all trades as there were 5092 trades. Comparison of the return tables (both old and new) found that in 2018 the old backtest shows a MDD of 17.6 whereas the new backtest shows a MDD of 19. Hence, I focus on the trades for 2018, which was 10 pages of trades, and the culprits are BBWI (was known as LB last Monday), HCA and EXEL.

These trades were entered in the new backtest, and not in the old backtest. Thanks to Nick, I included them in a custom watchlist and excluded them in a backtest that I ran minutes ago, and the MDD was 17.5. This suggests that Norgate Data has from 5/7/2021 to present made some changes to historical prices.

[Author]

Posts