Ben’s Journal

[BenOsborn]

Slow week for me this week. I only had 21 fills, which is less than half the last two weeks.

Observations for this week:
– Three trades didn’t close in time for one session. The orders tried to send one second after market close. There were 10 other orders that filled that day that did close in time though. Closed on in aftermarket trading, on in pre-market the next day and one when the market opened.

– I had a partial fill in one session. Unlike Interactive Brokers, Tradestation does not recognize the partial fill to send an OSO order to close just prior to market close. Managed to close in aftermarket trading.

The problem with closing in aftermarket, pre-market or the following day is that Tradestation seems to reserve that available margin / capital and I can’t use it in the next session, even though the trades are closed. This means that the next session I can only have a max of 37 trades. This presents an opportunity cost. What if the 3 trades that I couldn’t enter were the ones that triggered (it has happened before)?

To try and mitigate this I have changed my OSO orders to send the order to close 5 seconds prior to market close. This does come with its own costs. More often than not I end up with a worse price than 2 seconds before or the MOC orders in the backtest. Although I only have limited data to test at this stage. Either way there is a trade off.

All this will hopefully be irrelevant soon as the API type solution is progressing. I have put it on my VPS this morning to start testing it next week. To say I’m excited about this is an understatement!

I am trying ChartVPS for my VPS as I needed more RAM to run it than what Speedy Trading Servers offered (unless I went for their Mega VPS). I have heard that they are good but I will wait and see over the coming month.

[BenOsborn]

In early March I decided to close the remaining trades in the BWTT and move the capital to my MOC systems. This was because the version of WTT had been cobbled together prior to joining the Mentor program from what I could find on internet chat forums, trying to piece together the code myself and throwing in some over optimisation of irrelevant parameters. In short, it wasn’t robust. I may revisit it again sometime in the future, putting into practice the knowledge I have gained here.

From the beginning of March until 28 March I was trading a live MOC system on the Russell 2000 whilst I worked on getting an API type solution developed for Tradestation. This was ready for me to start testing this week. I tested on SIM Monday and then moved to live testing Tuesday as some of the features (such as the MOC orders) do not work on SIM. This week has been a lot of late nights (US opens at 9:30pm here) making sure the system was running, orders were entering etc and early mornings (3:20 am for me) ensuring unfilled orders were cancelled, MOC orders were in place etc.

On Tuesday when I moved to live testing of the API I moved to trading a combined MOC system. I should buy a lotto ticket as pure luck saw me start trading the combined system live on a day where I got an 11% return. Will continue testing with the late nights and early mornings next week to make sure it continues to run as it should.

No problems with ChartVPS so far.

March 2022 Performance

US
BWTT + 0.21% (17.58% drawdown) (discontinued early March)
MOC RUT – 6.7% (discontinued late March)
Combined MOC +9.1% (started late March)

ASX
Growth Portfolio -0.31% (7.26% drawdown)

[BenOsborn]

April has been a below average month for my combined MOC systems so far. Some interesting stats:

– I had a losing streak of 6 days (which is equal to my longest loosing streak in terms of losing days over the 17.25
year backtest period)
– The total length of my current drawdown period has only occurred 4 other times in the past 17.25 year backtest
period (although I am only halfway to the max drawdown period)
– I am now starting to nudge my max DD in my backtest period (only 2% away)
– Although a lot can change in a week (even a day), this is currently looking to be the worst month my system has had
since I started backtesting.

On the positive side, the probability of things turning around gets higher each day!

Whilst I would prefer to be positive rather than negative, I’m not bothered about it at the moment and have had no problems executing my trades each night. Although I’m not sure how I will start to feel if it begins to exceed my backtested max drawdown by a significant amount.

I am aware that it may though because, apart from the fact that the past does not necessarily = the future, there is some uncertainty in my calculations due to combining the systems and my position sizing. So the total number of trades taken by the combined systems, in theory, at times exceeds my available capital. This means that on days where there are a large number of fills for each system in the backtest, I have had to average the results out a little. Chances are in real trading my systems would have caught all the worst trades leading to worse results than the backtest might suggest. Obviously, it could also go the other way and things may even out over time.

I have the option of reducing my position size and reducing this risk but at this stage I am aware of the risk and ok with taking it on. Most of the time the results have been pretty similar, except there was one day where I lost a bit more than what the averaged out backtest result was. Who knows how I will feel in another month if the results vary significantly to what the backtest suggests! I am keeping a close eye on it anyway.

Hopefully that makes sense.

The only other problem that I have been noticing lately is that I have had a few instances of trades being in my exploration and entered in real trading yet the next day they do not appear in the backtest and a different stock might. Has anyone else come across this?

[PaulBailey]

That’s a really good post, Ben – thanks for sharing. I suspect a few of us are in the same boat – although I’m just at the start of my own journey.

[KenHall]

RE: The only other problem that I have been noticing lately is that I have had a few instances of trades being in my exploration and entered in real trading yet the next day they do not appear in the backtest and a different stock might. Has anyone else come across this?

Hi Ben, I’ve had some strange issues like this when developing an overly complicated system that relied on identifying Higher Highs HH, Higher Lows HL etc that I’ve since shelved. (I’ve had trades not taken in real life (but in the backtester), but that’s normally because I had a buy limit order at the Low of the candle.)

I’d encourage you to get to the bottom of the issue as it may indicate an error that invalidates the confidence you have in the strategy.

Couple of suggestions…

Check for any possible post-dictive (forward looking, signal repainting) errors especially in your ranking code

Check for any data integrity inconsistencies.

Use the “Interpretation Window” (Course: module 10), to visually step through the bars (and with bar-replay) and see that indicator results, ranking results are what you expect.

Use the bar-replay feature to step through each day generating explorations / backtests (with _TRACE debugging) to see if it’s consistent in creating the same signals / backtest results. (Initially I never knew that bar-replay essentially truncates your database to the time period at the bar-replay, therefore Amibroker can not see future data and so best simulates what happened on the actual day)

Reach out to Craig to review your code.

[KateMoloney]

When you did your testing did you stress test to exclude the low of the day?
Craig told me about this, it helps give you more realistic results when backtesting.

custom backtesting section (Craigs bit is in red, you just activate the second line and // the first line.

if (LimitPrice < LowPrice) // If Limit Price less than Low
//if (LimitPrice <= LowPrice) // If Limit Price is the low of the day, therefore exclude this signal
sig.Price = -1; // Therefore exclude this signal

[BenOsborn]

Thanks Ken and Kate.

I have gone back and re-run backtests and resolved most of them. There were a number in the last session and I am trying to think back to when I ran the backtest after market close. I am beginning to think that it was probably after the interim data was published but before the final data was published for the day. I think this could be the issue.

There are a few instances that are unresolved, however, they involve the same shares each time – HZNP and AMED. They are picked up and rank in the exploration but are not picked up in the backtest. Will continue with the suggestions to see if I can find out why.

[Nick Radge]

Reach out to Craig

[BenOsborn]

April 2022 Performance

US
Combined MOC -20.9% (ouch!) (-12.84% since late March 2022)

ASX
Growth Portfolio 2.88% (-4.29% since Oct 2021)

[BenOsborn]

US Combined MOC got smoked last night. Largest one day loss since I started trading it (although not the largest one day loss in backtesting). However, my drawdown has now exceeded the max drawdown in backtesting. No risk, no reward I guess!

[BenOsborn]

Another large loss for the US Combined MOC last night; if this keeps going I will have to use my safe word!

Drawdown has now exceeded the drawdown in all of my backtesting by over 10%. If the total drawdown starts to exceed 40% (which it is nearing) I think will start to feel it. At the moment I am so/so about it and trust it will turn around.

In the meantime I guess I just need to keep pushing the buttons and wait it out!

[Nick Radge]

Ben,
I’m away till Sunday, but maybe send me through the code and details and I’ll take a look again.

Nick

[BenOsborn]

May 2022 Performance

US
Combined MOC -15.62% (-26.46% since late March 2022)

ASX
Growth Portfolio 0.46 (-3.85% since Oct 2021)

[BenOsborn]

I haven’t been updating progress much lately as I have been swamped at work.

After speaking with Nick, I found out that my selection bias on the combined MOC systems was a bit high at 11.25%.  I have since played around with the stretch, number of positions, position size, universes, etc to reduce the selection bias. The challenge is balancing reducing drawdown and uncertainty to acceptable levels whilst still maximising profit potential, levels that will be different for all of us.

I had slowly been eeking out small gains over that last few weeks but the last couple of sessions put a little dent in that. Overall, in terms of my re-jigged parameters, I am still sitting below the max drawdown in backtesting (although it is not too  far away).  The system has been in drawdown for 72 days now; this length of drawdown has occurred 10 times in the past 20 odd years so nothing new.  The max period that the system has been in drawdown in backtesting over the last 20 years is 185 days so I might still have a way to go before I see new equity highs.

I have also been trying to develop a rotational system for the Nasdaq.  I have tried various ways and indicators for measuring momentum.  At this stage I am getting CAGR in the mid 20’s and drawdown in the high 30’s.  I have been able to get drawdown into mid 20’s but CAGR then drops down to high teens.

I don’t mind taking on a bit of risk and drawdown in order to try an achieve higher gains, I just want to make sure that the system is robust and there is not something that I am missing.  My thoughts are to try and maximise capital growth early on with a view to tuning things back a little in the future when my accounts are at a larger size. 

Although I would love to be making new equity highs instead of lows, I am still pretty happy with how things are going and it is just a matter of time before the equity highs come.
 

[BenOsborn]

Combined MOC daily results for June so far:
-0.01%
0.53%
-0.29%
0.34%
-0.13%
-0.78%
-3.63%
-5.94%
-0.41%
0.18%
-6.86%

Not looking pretty!  I must admit that last night has hurt the most (even though I have had bigger one night drawdowns) as it took me over 40% drawdown for the combined MOC system.  Selection bias has come into play a couple of times this month; however, results don’t exceed what the backtests shows overall (something which I am keeping an eye on).

Hopefully we end the week with a bounce (either way I will be at the pub tonight)! 

[Author]

Posts