Mike’s (UK) journal

[Anonymous]

Hey Michael, I think you would really like some of the data that Jack Schwager gives in [strike]Market Wizards[/strike] I’m freaking kidding…;
A Complete Guide to the Futures Market

Quote:

“The fact is you can find a favorable illustration for almost any trading system. The mistake is in extrapolating probable future performance on the basis of an isolated and well-chosen example from the past.”

“The moral is simple: Don’t draw any conclusions about a system (or indicator) on the basis of isolated examples. The only way you can determine if a system has any value is by testing it (without benefit of hindsight) over an extended time period for a broad range of markets.”

“As a general principle, it is wise to use the simplest form of a system (i.e., the least number of parameters) that does not imply any substantial deterioration in performance relative to the more complex versions.”

Excerpt From: Jack D. Schwager. “A Complete Guide to the Futures Market: Technical Analysis, Trading Systems, Fundamental Analysis, Options, Spreads, and Trading Principles (Wiley Trading).

Those sound strikingly similar to someone else we know… dang… can’t put my finger on who it is.

He has loads of good info in there, and I probably read half of the book. Some chapters I felt were a bit past the point of me being able to retain helpful information, so I skipped them.

[Anonymous]

Re: different stretch as referenced by Julian above
For what its worth, my system with that atypical stretch calculation has two checks for an uptrend, two volatility checks, ranks by positive ROC, and sets the entry off of the close, not the low. It has no test for a pullback. The only thing that I’m nervous about is that I am not sure why the system makes money…. and that seems to be one of Nick’s pillars, justifiably so. I’m not in dire need to use it at the moment, so I will investigate more in the future.

[Anonymous]

Had my first mismatch in live trading my MOC yesterday, 4 June.

Sent 40 orders to IB as usual.

Had 7 of those hit their Limit Price and fill OK.

Checked against the backtest today – those 7 live trades show in the backtest correctly.

There was an additional trade however showing in the backtest – MORN.

I keep a csv of the Explore I do each day for sending to the API (cuts off at my max pos of 40).

Checked the csv for the day, and MORN not on the list (have to assume was ranked below 40).

COHR was on the API list at position 11, but when I reran the Explore for the day COHR doesn’t appear anywhere – doesn’t make the ranking list at all.

As such, this has allowed MORN to get into the top 40, and so show up in the backtest result.

Rechecked the Ranking calculation manually for COHR, and the result would have it well above MORN in the Ranking Lists.

Manually checked Amibroker, and both COHR and MORN meet BuySetUp criteria on the previous day (3rd June) so both valid as candidates to Buy 4 June.

All other candidates for the day match the API list correctly.

No news items suggesting COHR fallen off the face of the earth, and Norgate prices match publicly available price info.

Just wondering if anyone has seen this sort of situation arise before?

As mentioned, my 1st difference between Amibroker and 7 weeks of live results, so in the scheme of things, not an issue. Just wondering if might indicate areas to check / investigate.

Any thoughts or suggestions gratefully appreciated.

Many thanks

[Nick Radge]

Check the ranking decimals. Take them out to 5 or 6 decimal places.

The other question is whether or not the exploration settings had changed between the two days and/or did you run the exploration incorrectly.

[Anonymous]

If you look at the volume on a minute basis for the 4th, there are some minutes where volume traded less than 100 units for the whole bar.

Could it be a liquidity issue? A “not able to fill the order at the time of your limit price hitting as no volume was being traded” kind of problem?

[Anonymous]

Thanks for the feedback & suggestions.

The afl file I use is designated “Live”, and so I never alter since all original testing finalised and completed.

As such, should be no changes between any 2 days, other than a process mistake I might have made.
I tend to plod through fairly slowly, to make sure I get each step correct.

I run the Explore each day against R1000 (not R1000 Current & Past). I’ve test checked running Explore on both R1000 and R1000 C&P, and no difference.

Ran an Explore of Regular vs Chartist API, and both exactly the same (no COHR on either).

Re-ran an Explore based on 3 June data – every stock matches its ranked position when I compare to the original Explore / csv file for extract to API for 4 June trading – other than COHR which originally ranked at 11 – and MORN which has now crept onto the ranking list at 40, due to COHR falling away.

My original Ranking was at 5 d.p.

I manually re-calculated my original Explore / API extract

Rank # 10 ESTC at 100. 04571
Rank # 11 COHR at 100. 04546
Rank # 12 FND at 100. 04505

Rank # 40 CDNS at 100. 03011

Also calculated MORN at 100. 02856, so legitimate that it should now get onto the Explore list once COHR fell away.

re the volume point Matt – COHR never got close to its target limit price, so I wouldn’t have expected that to be a live trade on 4 June (COHR volume 170,000 / turnover $25.9Mill).

I will do a re- Explore for each day since went live, and compare to my original Explore csv file for that day, and see if any other discrepancies arise.

Never considered to check that before , since my live trades match up with the backtest each day – I run a backtest every day as part of my test process to compare to live.

Will dig a bit further and see if it gets me anywhere.

Thanks again for the suggestions.

[Anonymous]

When you ran the original exploration for 4th June trades on 3rd June data did you wait until after midnight US time on 3rd June to ensure all data produced by Norgate Data was”Final”? You will notice on the information tab on the updater that after the market closes each day at 4pm and then up to midnight that night the data will show as “interim 3” and the like until very late in the US day at which point it will change to “final”. The indices data only change to Final after midnight. I believe this is because Norgate go through a process of verifying all the data for the day, the info on some symbols is completed before others etc.

Perhaps the original “faulty” exploration result was run before the final data for the day was complete for either equities or indices? Maybe a price on interim data was showing the close as $153.53 (and that’s when you ran your original exploration) but only after the final data was out did Norgate clean the data and the close was finally recorded as $153.59.

I make it a habit to not run any explorations or backtests on the day that just closed until well after US midnight. And that’s for both explorations for the following day of trade and also for running a backtests for the day to check if all my trades happened as planned.

And I’m not saying that Norgate distributes incorrect close prices in their interim data before it becomes final, I am just taking a stab in the dark that perhaps interim data isn’t potentially as complete as the final data for the day so perhaps always best to wait until after midnight to make completely sure.

The other thing I have found is that (although I can’t really prove it) I swear that sometimes my indicators or code that I run is sometimes affected by whatever symbol I may happen to be currently displaying in the chart window. For example maybe I haven’t coded well or haven’t used restorepricearrays() or something like this correctly. So to make sure I best can avoid anything currently displayed affecting things I always close Amibroker, update data, open Amibroker, refresh data, close it again a second time after the data refresh, then open again and ensure the displayed symbol on the chart is related to the strategy I am trading. For example if it is my Russell 1000 strategy I will ensure I have $RUI showing when I then run the exploration for the day.

I know these are long shots but things that I do to help remove doubt from “unknowns” in daily processes.

[Anonymous]

Well that’s 5 hours of my life that I wont get back quickly.

Various tests and checks made.

Did as I advised – re-ran explores for last 6 weeks and checked each one against my csv explore extract sent to the API.

Everything 100% OK other than last 3 days, when COHR has stopped showing up in the re-Explore, when it was in the original explore / csv / API loaded to Interactive.

Ran an individual backtest on COHR (1/1/2015 to date) and got “No Results”

Then checked the chart for COHR, and plenty of green BuySetUp arrows, and 16 Blue Entry arrows.
Checked the setups / buys valid, and all were good.
Mmmm.

So the sad part of me said, “why not check all the R1000 stocks to see if any other oddities”.

Using the R1000 (not Current & Past), did an individual backtest on every stock, for 1/1/ 2015 to date.

Did sample checks comparing the backtest result against the stock chart, and that Blue buy arrows matching entries in the backtest results.

Every stock that showed “No Results” in its backtest, checked the chart.

In all cases, except a couple I note below, the reason for No Results was that the stock was relatively recently listed, and so no LTMA in place (one of my entry rules).

There were a few stocks however, that were similar to COHR i.e. “No Results” showing on the backtest, yet the chart contained Set Up and Entry arrows. Checked these to ensure consistent with system rules, and they were valid.

So my current thinking is, might there be an issue with my data?

The stocks that showed No Results, but charts had entries were;

COHR
ABC
IRM
LSI
MNST
SVC

I then did similar individual backtests on these 6 stocks on 2 other of my MOC systems, and got exactly the same outcome i.e. charts showing trades, but individual backtests showing No Results.

Also checked these other systems against a variety of other R1000 stocks, and individual backtests working fine and in line with charts trades.

I would be very interested and grateful to know if anyone could run individual backtests on any system they have on each of those named stocks, and see if it generates backtest results that match up with what the chart shows.

Thank you

[Nick Radge]

I tested a turnkey system on those symbols back to 2014. Every signal correct.

Have you checked that your dividends are off?

[Anonymous]

Thanks for running those checks Nick.

Dividends switched off – I never have on.

Whilst working through everything yesterday, I did the basic technical “go-to” fix – switched the machine off and restarted to see if that caused a change, but to no effect.

This morning, switch on, and COHR is now working again!
The other 5 stocks remain as No Results.

Reloaded AB (version 6.30.5)

I’ll go back to Amibroker and see if they have any thoughts.

Thanks for the help and ideas.

[GlenPeake]

Hi Mike,

This is a long shot and I don’t think this is your issue judging from the symptoms….but..

Check your EXCLUDE WATCHLIST section in the Filter area.

I have my EXCLUDE MR/MOC list configured there… and on one occasion, this had ‘mysteriously’ flicked over to my ‘current MR trades’…. so the EXPLORE would work fine, however once I updated the “CURRENT MR Trades” watchlist a backtest would then NOT include these trades, because those stocks were in my EXCLUDE LIST (as current trades watchlist)…

Maybe just double check the EXCLUDE section, in case something ‘funky’ is happening there.

[Anonymous]

Thanks for the suggestion Glen.

Double checked, and just reran ensuring Exclude is Off.
It is, and still blocking the rogue stocks.

Last night did a very simple test system – Buy if C > Ref(C,-1). All my “rogue” stocks worked absolutely fine – lots of chart arrows, and matching individual backtests.
So my suggestion that I had a data issue is incorrect.

Will keep looking.

Thank again everyone.

[GlenPeake]

Hi Mike,

I’ve got something for you to check with your Backtest issue…..

(Similar thing happened to me recently on Friday)….

I found a fix, which might also apply for you…

Firstly, to gather a bit more information, run the same backtest that you see the missing trade(s) on…. but run the backtest in “DETAILED LOG” mode

e.g.

Run your ‘troublesome’ backtest again and if you’re hitting the same issue as me, you might see something to the effect of:

<whatever stockcode> not entered because of insufficient funds or wrong position size/value <blah blah blah position size details etc>

If this is the case, add this to the “Backtesting” section of your code

SetOption(“AllowPositionShrinking”,True);

Then go back to normal “TRADE LIST” log mode and run again.

Fingers crossed that this solves the puzzle for you!!! I’m keen to know if it does….

Good luck!

If it doesn’t fix it, post up any error you see in the Detailed Log etc.

[Anonymous]

Hi Glen

many thanks for that detailed suggestion.

Ran the Detailed Log backtest, and no errors appear at all.

I will continue to investigate, but have to also “park” the issue to a degree, whilst trying to develop other systems.

And as I found in another test, the rogue stocks still appear in other system backtests, so my Amibroker hasn’t fallen out of love with these particular stocks.

Thanks so much though for that extra help.

Cheers

[Anonymous]

Yesterday (11 June ) was very red, but also flagged up a number of learning points for me.

Fortunately (at least for yesterday) I am still in test mode (nominal funds).

And my orders have been sent to Interactive today.

Learning points

A) My MOC is 40 positions. So 40 orders sent through to Interactive using the API – fine

Noticed that various trades had been BOT then immediately SLD during the opening minute or so.

Reason I have found is that when setting up the API, I left Maximum Allowed Fills still set at 20. And that worked effectively on the day.

I need to match the API with my system rules i.e. 40 Maximum Allowed Fills.

b) Reconciling backtest for the day against live results raised a point about Max Positions.

My backtest showed 39 trades i.e 1 short of my 40 max.

A number of backtest trades were not activated live due to the API Max Allowed Fills of 20, and
these non activated trades only hit their Limit Price well after the opening few minutes, at which point I was fully filled to the 20 API max.

The issue highlighted though that only 39 backtest trades were made vs 40 max.

The missing backtest trade was a valid trade (and was actually activated and traded Live with Interactive).

I have been running further Backtests, and if I put in say 10 Max Positions for the backtest, the backtest will only produce a maximum of 9 trades (even for a day like 11 June when 40 trades would have been the expected level).

Checked against different days in the backtest period, and varying Max Positions set. eg Max Positions set at 6, get no more than 5 trades in the backtest results. Then set at 5 Max positions, will only get 4 trades on backtest, on the exact same day that in earlier test had 5 trades.

In each case, the max positions showing in the backtest result, is 1 less than the Max Positions set for the backtest.

I am using the CBT MOC template, and have not adjusted or gone near, any of the smarts incorporated within that template – just added my set up conditions.

Has anyone else seen this occur i.e. backtests showing fewer trades than Max Position rule (for days when the Max Positions would be hit, that is)?

Thanks

[Author]

Posts