Reducing Entry Signals

[ScottMcNab]

win rate drops 0.5-1%
ave profit/loss drops approx 0.05%
CAGR drops 2-3%
maxDD increases 1%

getting closer to reality it would seem

Thanks again Said

[JulianCohen]

Here’s a question regarding MCS.

If the selection bias is running at 95% or better, do you think you should run the MCS variance testing on say 10 skip trades, even if the trade volume is high, as you know that the chance of the system not getting filled with the trades you are expecting is low?

[Anonymous]

As mentioned elsewhere on the forum, I shut down my MOC trading pending further research since 3/21, a day when my live results were way worse than anything my MCS was generating due to selection bias during the US session.

In order to test the efficacy of the following code in capturing the scope of my selection bias, I ran a simulation on that day with this code included and compared the live results to my actual trades:

PositionScore = IIf(O < Ref(BuyLim,-1), 200, 100 - abs((BuyPrice-O)/O) * 100); Unfortunately, this did not accurately capture the selection bias I had on that day. Perhaps I am doing something wrong, but selection bias was still much worse in reality than simulated results even including this code.

[Nick Radge]

Perhaps its better to work with a strategy + universe + position size combo that almost entirely eliminates it…

[ScottMcNab]

I think on those few days a year when buy signals greatly outnumber available positions then the results will (by the nature of the system design) have a very high chance of being almost as bad as it can be. In one of my rants…er posts in my journal a while ago I discussed the first time it happened. I was very disturbed. My results came in at 995/1000 (or similar) for the simulations I ran for that day. It resulted in a drop of 1% MORE than the “average” return for that day in the backtests. It has happened 3 additional times since then. For one the results were smack in the average and the other 2 were lower than average for the day but not as disastrous as that first time.

Further work on selection bias has been with the requirement and keep these days to a maximum of 4 times a year. I expect my real CAGR to be 1% lower for every one of those days that occur each year.

Maybe run a few tests on 3/21 (1000 runs each)…check what the “average” is and then what the worst possible and then reduce leverage until the additional loss is bearable and then see if the system metrics are still acceptable ? How often will these occur each year ?

I am now experimenting with combined MOC system (ASX day and SPX night) without any leverage (20 at 5%) as the combined results are looking acceptable with reduced risk….just a thought

[ScottMcNab]

Another option to try perhaps is changing the system to see how it runs with a limit on open buy order
…so open has to be <= buylimit or order cancelled....if there are more buy signals than positions then distribution of returns for the day should be more random (or could maybe account for order that stocks open in within the backtest itself) I have found the drop in buysignals to be significant though would necessitate new API… but if happy with system may be worth it in long run

[Nick Radge]

Quote:

I think on those few days a year when buy signals greatly outnumber available positions then the results will (by the nature of the system design) have a very high chance of being almost as bad as it can be.

Why not go back to those specific days and see what impact they actually have on the overall system results…

[Anonymous]

Nick Radge wrote:

Perhaps its better to work with a strategy + universe + position size combo that almost entirely eliminates it…

if only!

I am able to get there in AU, but so far it has eluded me in the US.

[Nick Radge]

Keep working at it. I just ran a test on my MOC system for the last 3 months. The single run backtest is less than $50 out from the real time result.

[ScottMcNab]

Brent Hause wrote:

Hey guys –

A bit of thought on this discussion – when I see you guys talking about the “worst case scenario” for a MOC system and stating that as a situation in which the worst trades of every day are selected, I think you are looking at the worst case in the wrong way. While it is important to look at these metrics and judge their probability, I think the “worst case scenario” is indeed much worse.

When I think of the worst case scenario, I don’t envision a slow bleed of the equity curve over time as the system slowly diverges from test results, what I imagine is a large gap lower when I have lot of orders staged.

E.g. Say I have a system that only sees selection bias on 5% of days. That said, there are still days when I put in 200 limit orders ahead of the open. My API is set to cancel all remaining orders as soon as 40 are filled, if more than 40 are filled, immediately sell any positions over the 40 threshold.

The nightmare scenario is when the market gaps lower aggressively (think flash crash), way more than 40 positions are bought, but due to volatility and race markets, there is no real bid and the positions over 40 get sold for pennies on the dollar. The likelihood of this scenario is admittedly small, but this is the type of event that could literally wipe out a trading account. This type of scenario could have happened during the ’87 crash, the flash crash of 2010, and Knight’s flash crash in 2012 – I don’t know and I don’t think we have anyone here who traded a MOC system in those markets.

In order to control this risk I am doing two things:

1) Trade my MOC system in a separate account with as much margin as possible. This way if things get real nasty, the bulk of my trading capital will not be exposed. I would rather have my account go negative and be a creditor to my broker (and hence have negotiation options) than see all of my cash get vaporized and have no options.

2) I am considering slicing out orders in batches when I have more than 40 orders. Using the API I can set batches of orders to route, say every 3 seconds. E.g. have first 40 orders in on the open, and slices of 40 orders set every 3 seconds thereafter. This will reduce my risk if there is a big gap on the open (if more than 40 orders get triggered, the remaining batches will not route). Of course I cannot test this, so there may be some performance lag – something I am weighing.

Also, I am trying to decide if this risk just simply rules out MOC systems for me, and I will work to develop short term non-MOC systems to balance out the long term systems (3+ day mean reversion strategies without stops). While the likelihood of this type of scenario playing out is very small, the ramifications are extremely large. My thought is that I want to do everything I can to minimize risk of ruin – that said, I don’t know what the answer is to be honest.

Just my two cents.

I wonder if buying a long dated put options on index 20% below current price would be cheap enough to be a viable insurance policy for those ’87/flash crash scenarios ? Not to make money but simply avoid ruin…

[LEONARDZIR]

Brent,

A couple of observations.
I checked my MOC system on the day of the flash crash and it was slightly positive.
I did get filled with 40 positionbs at 10% my limit.
If you wish you could just avoid placing orders when you know there is a significant gap down day at the open.
Your backtesting includes all the gap down days .

That’s now to say there couldn’t be some catastrophic event that could occur during market orders with multiple long positions and no bids but perhaps in that situation money maybe the least of our worries.

[ScottMcNab]

Was just a thought about a possible low cost hedge Len…can’t hurt to explore the possibilities. If the flash crash had occurred 60 min later it may well have been a very different scenario for MOC systems.. Still the old vanilla style ’87 crashes etc in the future to be expected…

simplest solution is probably not to be leveraged to the stage where an ’87 style crash would be catastrophic…I might break my MOC system from a combined US/AUS account into two separate accounts and accept reduced cagr….at least I know that only 50% of MOC account exposed at any one time…

[LEONARDZIR]

Scott,
I agree with you. I am trading my MOC with 4:1 leverage and sometimes I think I am hiding my head in the sand when it comes to thinking about potential catastrophic losses.

[ScottMcNab]

Len Zir wrote:

Scott,
I agree with you. I am trading my MOC with 4:1 leverage and sometimes I think I am hiding my head in the sand when it comes to thinking about potential catastrophic losses.

Thanks Len…interesting and reassuring .. I have had exactly the same image in my mind with respect to how I have been “dealing” with 4:1 leverage…it bubbles to the surface periodically as the subconscious finally gets through

[RobGiles]

The more I think about 4:1 leverage, especially when backtesting can’t really give you an accurate measure of your worst case scenario why are we trading these systems? Regardless of the probability of an ’87 style, catastrophic event (Which I believe could still happen even though the markets have changed), at some stage one of us will get blown up. This is a very different proposition to a drawdown which a MCS is supposed to measure and therefore give you the opportunity to ask yourself, ‘can you handle it’? This is the process I went through when I originally built my MOC system. I set out to build a system that exploits oversold market conditions by trading regularly, but at all times I wanted to limit my downside to about 30%. I thought the MCS was supposed to answer that question. After hearing the comments in this forum and giving it some more thought, I’m starting to wonder if trading this type of system at this kind of leverage is highly reckless, unless of course the chance of complete account wipe-out in 1 session doesn’t bother you?

[Author]

Posts