- This topic is empty.
-
I have discovered a very useful section of the RT software. Trent has been trying to explain it to me for a while now but it didn’t click until a day or so ago.
I’ll see if I can explain.
In Amibroker if we don’t use the Selection Bias CBT code, then the number of setups for a system that get passed into the backtester can be large.
Let’s say for example we test a simple system that buys if the C > MA(C,20) and run it on R1000 with 20 positions at 20%. You can imagine the day after a big up day there could be 700 odd stocks that fill that requirement and they all get passed to the backtester. You get results from the 20 chosen and run a Monte Carlo to see what effect any of the other combination of 700 stocks would have had on the result. Huge variance!
If you use the CBT code in the same circumstances then you rank the 700 stocks by your chosen method and those 20 are passed to the backtester, so running a Monte Carlo will always give you the same results as the same 20 stocks only are backtested. This is good, but it also means that the ranking method becomes a parameter which means that in effect, the ranking is a form of optimisation. You have chosen the “best” stocks on that particular day and then every other day in the backtest, so it could be argued that the backtest is curve fitted to a certain extent. That is the earworm I planted in Seth’s ear.
What I didn’t think to do with the CBT code was to have it coded so that you can pass more potential entries to the backtester than you would actually take. So in this example you could pass 40 ranked by your chosen method, and fill 20. In other words add back a controlled amount of selection bias back into the mix. This removes the over optimisation element from the CBT code backtests.
RealTest does this, and allows you to analyse the results. The results are very interesting. It adds a whole new level of adaptability to the system. I have found that you can run with less leverage and get similar results. Because the leverage is smaller, the -MDD is smaller, but the CAGR remains at a similar level because you can manipulate things to allow you to cause your system to take more trades. If your system has an edge to it, then that edge is allowed to be exposed to more trades in a controlled way.
Food for thought. And apologies right now if I send anyone off down a rabbit hole…Seth
Following from Julian and Seths post in Terry’s jornal.
I have been trading a long/short MOC system for about a month now. I have coded in RT and I am running five positions long and five position short at 10% each. Each day I am placing 15 long and short positions with the API. (Running two Apis, one long one short). Because I am running a cash account and not leveraging I found it not worth while only placing the number of orders for my maximum positions as with the AB CBT code. The exposure was too low so returns weren’t great.Placing more orders than positions does introduce a element of selection bias but as Julian said in RT is easy to rank these orders. Because RT is so fast at back testing you can easily do 1000 MCS simulations to test the possible variance to get an idea of the possible results.
I’ve been keeping a spreadsheet of my daily returns versus the back test on days where there are > 5 entries from my 15 orders. I want to know if on average my returns are going to be worse than the back test when SB is occouring. Because I’m still sceptical if I can achieve the returns that the back test shows. So far it seems on days when there more possible entries than positions you will get the worse end of the distribution of returns.
Example. Last session 4/12
Live:
7 trades total – 2 long 5 short
Return: -1.15%BackTest:
7 trades – 2 long 5 shortTotal possible trades from 15 long order + 15 short orders
15 trades – 2 long 13 shortI do 1000 backtests over 1 bar to see all the possible outcomes from taking different trades.
MAX RETURN: +0.59%
AVG RETURN: -0.85%
MIN RETURN: -2.41%MY RETURN: -1.15%
Remember this is only over one bar, so the distribution of returns can be quite large. The blue line on the picture below is where my returns came in the day. This is 1000 runs over the last bar. I guess the theory is over the long term you will get a random outcome and it will average out. Whether or not this happens in live trading I guess time will tell and is why I am keeping the spreadsheet.
If it turns out that in live trading you get a worse end of the deal. Then you can easily incorporate that into the back tests in real test. To get a better/more accurate back test.
- You must be logged in to reply to this topic.