MOC System Missed Entry Orders

[TerryDunne]

I have used the trade date’s ROC of the low (i.e. post dictive) as my ranking, on the basis that these are the ones likely to be filled first on the day.

[Anonymous]

So doesn’t that create a post-dictive error? How can you know the low of the day before the day is complete?

[TerryDunne]

I think it’s OK. In real life I’ll be sending a lot more transactions each day than can be filled, and the ones that have the lowest low move are likely to be the ones filled…I think?

[Anonymous]

I’m still not sure if I’m following.

If this item in your code is for the purpose of a ranking calculation, does this item then act as the ranking condition within a buy setup from the previous day data? i.e. within a set of entry conditions that form something like…

RANK = ROC(L,1)+1;

Entry1 = Somekindofentrycondition;
Entry2 = Anotherkindofentrycondition;
Entry3 = andjustonemoreentrycondition;
Entry4 = (RANK > 0);

BuySetup = Entry1 AND Entry2 AND Entry3 AND Entry4;

Buy = Ref(BuySetUp,-1) ;

If you are not running it within a buysetup from the previous day data then you’ll definitely have a post dictive error and backtests will never match reality. Also your backtests would be choosing entries from understanding future data, so all testing/optimisation is going to be falsely showing great returns.

Also this would then make the ranking portion of the code completely useless/pointless.

I’m sorry if I am missing something?

[TerryDunne]

Hi Matt,

I don’t think I’ve explained myself very well – here’s the problem I’m trying to solve.

Let’s say you have 20 stocks where your oscillator is below your threshold but to enter you need an intra-day downwards movement (probably some number of ATRs). You might get 5 of the 20 that move down enough to trigger the entry but only have available funds to enter 2 trades.

Whatever ranking used won’t be relevant because the stocks that are entered will be the first two to reach their LIT point, irrespective of whether they are ranked last and second last. The ranking might put the last one to reach the trigger point at the top of the list but in practice you won’t ever get that stock because you’ll be fully invested before it triggers.

My attempted solution to this is to use the ROC of the low on the trade day, the idea being that 2 stocks with the biggest downward movement are the ones likely to reach their trigger point first.

Regards,

Terry

[Anonymous]

It sounds like you are not using the custom backtester code which helps do the ranking such that the very selection bias issue you mention, would be eliminated.

If you are happy to always run with more open orders than positions/funds available and not have a system which you have designed/implemented with the assistance of the custom backtester code, then you will have the selection bias issue.

If that is how you want to run it, then the only thing I could suggest is to try to use as many positions (with smaller allocation each), as possible. One way to check how much an effect this problem presents is to just do something silly like setting your maximum positions to 1000 and % per position to 0.1% then run the backtest, dump all into spreadsheet, subtotal number of orders placed within the same day then insert a histogram and see what it looks like. Perhaps across a 15 year backtest the graph may show there are only 10 or 15 days where you exceeded 50 positions, in which case you could probably just run with a 50 position system and accept that in a handful of these 10 or 15 days your results may not accurately reflect a backtest because you could have gotten more than the 50 fills.

Still, I use the custom backtester code to specifically avoid this issue. From my understanding, the custom backtester code runs the analysis in the first loop to know which symbols on which dates would have satisfied your ranking method through the backtest period, then goes back and actually runs the backtest over these same symbols and dates only against all entry-exit conditions. I guess you could say it is a post-dictive ranking check combined with a traditional non-post-dictive backtest. That is my understanding of it, anyway. Perhaps I’m wrong.

[Nick Radge]

Yep, need to run the custom backtester code.

[TerryDunne]

OK, sounds interesting.

Thanks for the information!

Terry

[JulianCohen]

I created a system for the RUA (3000) identical to my RUI (1000) system with a small tweak of stretch and one parameter. I have been running it alongside my RUI with a small account to test out the actual fills. So far, and this is a very small sample size, the RUI has filled 100% and the RUA has not. I’m not actually counting the fill rate, as at the end of the month I will compare actual returns to backtester returns to see the discrepancy. I’ll keep you posted on my monthly returns on my Progress Journal.

Ironically Actual returns are double backtester returns because many of the missed trades were losers, but this is only from a one week sample size so don’t read too much into it. End of the year will be a better time to judge.

I really think this is the only way to properly test this type of system as it is impossible to know otherwise what trades were filled or not. However I am quite happy to run the systems alongside each other for six months or so in order to evaluate them.

[Anonymous]

Hi guys, just a quick follow-up on this from my side. I have been tracking the slippage on a weekly basis for two MOC systems from earlier this year. I definitely wouldn’t say that the slippage is meaningless. Based on the last 6 months, this slippage is reducing my return by ca. 1/3 relative to what my backtest illustrates on the SPX500. On the Russell 1000 it is worse at ca. 1/2 – in otherwords, my actual return is 50% of my backtest.

The key thing is that its pretty much purely downside risk. The trades that are taken in the backtest but not actually recieved are almost always very profitable.

I will keep tracking at let you know but the reason I raised this topic is that I can see, based on my systems at least, that MOC systems definitely have a performance drag. Enough that it could take a system from tradable to non-tradable in my opinion.

A perfect example this week was CERN on Nov. 29th. A good portion of my system’s weekly return was based on a MOC buy signal on this stock, which was never actually filled in my account.

[JulianCohen]

Dustin you are getting slippage on the R1000? I very rarely get slippage on that.

What is the other universe you are trading?

[Nick Radge]

I also have to agree with Julian – I rarely miss signals on the R-1000. Currently trading 3 strategies on that universe.

That said I missed 1 on Friday which was the low of the day, but considering volume was just 25% of the average daily volume I’ll put it down to an aberration.

[Anonymous]

Yes, I am trading the R1000. The issue for me is that I would agree that I don’t miss many trades, but the ones that I do miss are often the most profitable ones. It may be 3 or 4 trades a month but they can be some of the best ones in the backtest.

Also to confirm, I am talking about a MOC systems that has quite a lot of turnover. For the month of November, I had 81 trades in my R1000 MOC system.

I do this evaluation after every week on a trade by trade basis to see where my system is tracking relative to the backtest.

[Author]

Posts