Home › Forums › Trading System Mentor Course Community › AmiBroker Coding and AFL › ** Eliminating Selection Bias **
- This topic is empty.
-
AuthorPosts
-
May 9, 2018 at 4:12 am #101809Nick RadgeKeymaster
Part 1
A very big thank you to Julian who pursued the selection bias issues and found a solution via the custom backtester (CBT). He’s done this at his own expense and has kindly allowed me to share with the group.
We have looked at the code on both MOC and multi day systems and have so far concluded it’s doing what it’s supposed to do.
That said, the onus is on you to check for yourself.
This post will be a brief outline on the code and associated aspects. Over the coming week we will rewrite the appropriate course notes.
To use the code a few changes need to be made elsewhere:
The Buy statement requires the limit price condition to be excluded:Signals will be ranked before sending through the CBT. The following seems to work reasonably well. Insert this snippet above your setup conditions.
Amend the PositionScore to:
Now add the CBT code to the bottom of your existing code:
Code:_SECTION_BEGIN(“Custom Backtester”);
DailyEntryOrderLimit = MaxPos;
Ticker = Name();
StaticVarSet(Ticker + “LimitEntryPrice”, Ref(BuyLim,-1));SetCustomBacktestProc(“”);
if (Status(“action”) == actionPortfolio)
{
bo = GetBacktesterObject(); // Get backtester object
bo.PreProcess(); // Do pre-processing (always required)
MaxOrdersToPlace = 0;
TotalOrdersPlaced = 0;for (i = 0; i < BarCount; i++) // Loop through all bars { MaxOrdersToPlace = MaxPos - bo.GetOpenPosQty(); TotalSignalCount = 0; SigCount = 0; // Count all entry signals for this bar for (sig = bo.GetFirstSignal(i); sig; sig = bo.GetNextSignal(i)) { if (sig.IsEntry() && sig.IsLong()) TotalSignalCount++; } // Process signals for (sig = bo.GetFirstSignal(i); sig; sig = bo.GetNextSignal(i)) { if (sig.IsEntry() && sig.IsLong()) { SigCount++; LimitPrice = StaticVarGet(sig.Symbol + "LimitEntryPrice"); LowPrice = Foreign(sig.Symbol, "Low"); // Restrict number of orders per day if (SigCount > MaxOrdersToPlace OR // Lower in the list than MaxPos less Open positions
SigCount > DailyEntryOrderLimit) // Exceeds max orders per day parameter setting
sig.Price = -1; // Therefore exclude this signal
else
TotalOrdersPlaced++; // Count orders placedif (LimitPrice[i] <= LowPrice[i]) // If limit less than Low... sig.Price = -1; // ...Doesn't meet entry trigger criteria } } // End of for loop over signals at this bar bo.ProcessTradeSignals(i); // Process trades at bar (always required) } // End of for loop over bars bo.PostProcess(); // Do post-processing (always required) st = bo.GetPerformanceStats(0); bo.AddCustomMetric("Total Entry Orders Placed", TotalOrdersPlaced, Null, Null, 0); bo.AddCustomMetric("Fill Rate", NumToStr(((st.getvalue("AllQty") / TotalOrdersPlaced) * 100), 1.0) + "%", Null, Null, 1); } _SECTION_END();
The exploration code is:
Code:_SECTION_BEGIN(“Explore”);if (Status(“action”) == actionExplore)
{Filter = Buy;
Filter = LESetUp;AddTextColumn(WriteIf(LESetup,””+BuyLim,””),”Lmt Price”,1.3,29,55);
AddColumn(Rank,”Ranking Measure”,1.2);SetSortColumns(-4);
AddRankColumn();
}_SECTION_END();
May 9, 2018 at 4:17 am #108686Nick RadgeKeymasterPart 2
When you run an exploration you will get the signals ranked from 1 onward.
If your system takes 20 positions, then you only ever place the top 20, etc etc.
If you hold positions overnight, then you only take new ones that will fill open positions slots.
When you run a backtest only those trades in the top ranking will be sent through to the backtester.
We’re working on DDE code – again, thanks to Julian for taking this route.
At this stage I am running the exploration, saving the required signals to its own watchlist, then using the existing DDE code on that watchlist. Once done, delete the watchlist content and repeat the next day.
We’ll write this up in more depth within the course in a few weeks.
May 9, 2018 at 4:46 am #108687JulianCohenParticipantWhen I first started to test this I found ranking by ROC worked well for me, but in keeping with the spirit of collaboration, I would greatly appreciate anyone finding a better ranking method to pass it on
Better might just be better for your system, but at least we all get a chance to try it.
May 9, 2018 at 4:50 am #108701AnonymousInactivewow – nice work Julian! Thank you for sharing.
May 9, 2018 at 5:17 am #108688RobGilesMemberReally appreciate it Julian, thank you. Once I get my head around it and test it out I’ll let you know ho it goes.
May 9, 2018 at 7:26 am #108703DanielBaeumlerMemberGreat effort! Thanks, Julian.
Just to make I understand the approach. On part 2, Nick says: ‘If your system takes 20 positions, then you only ever place the top 20, etc.’
Today, I may get 30 or 50 signals but end up with just 10 fills (for max 20 positions). With the new ranking I would only place 20 orders and may end up with just 5 fills. If so, there would be no need for the API to control the number of fills anymore. Is this correct?I tried the new code and gu
May 9, 2018 at 8:19 am #108704Nick RadgeKeymasterQuote:there would be no need for the API to control the number of fills anymore. Is this correct?If you’re holding positions overnight then the API can be switched off after order placement.
If you’re running a MOC system, then the API needs to remain connected to facilitate the MOC orders.
May 9, 2018 at 8:44 am #108689AnonymousInactiveThank you Julian. BIG TY!
I look forward to trying this out.May 9, 2018 at 8:48 am #108706DanielBaeumlerMemberThanks Nick.
I integrated the code into my MOC system but ran into attached error message.
Any thought’s what the problem could be?
Cheers. DanielMay 9, 2018 at 9:07 am #108707Nick RadgeKeymasterDaniel – my error. When I posted the code the forum reformatted. I have amended that now, so please copy and paste code again from above.
May 9, 2018 at 9:57 am #108705ScottMcNabParticipantDaniel Baeumler wrote:Great effort! Thanks, Julian.Just to make I understand the approach. On part 2, Nick says: ‘If your system takes 20 positions, then you only ever place the top 20, etc.’
Today, I may get 30 or 50 signals but end up with just 10 fills (for max 20 positions). With the new ranking I would only place 20 orders and may end up with just 5 fills. If so, there would be no need for the API to control the number of fills anymore. Is this correct?So the system would have a lower trade frequency for the elimination of selection bias ? Intuitively (ie probably incorrectly) I would think that a different system design would excel in this environment than the ones we have been designing to date ? Did you find this Julian ?
ps..I will enjoy spending hours testing different variations but keen to hear your thoughts Julian
May 9, 2018 at 10:46 am #108690ZachSwannMemberNT == 0;
In the LE Line where does it get initialized ?Julian have you tried using Volatility i.e H/L
May 9, 2018 at 8:54 pm #108709Nick RadgeKeymasterQuote:So the system would have a lower trade frequency for the elimination of selection bias ? Intuitively (ie probably incorrectly) I would think that a different system design would excel in this environment than the ones we have been designing to date ? Did you find this Julian ?Yes. But now you can make it more targeted which in fact will increase trade frequency.
May 9, 2018 at 8:56 pm #108710Stephen JamesMemberIgnore NT Zach. Its just part of that systems entry conditions (old delisted stock exit)
May 9, 2018 at 9:25 pm #108708JulianCohenParticipantScott McNab wrote:So the system would have a lower trade frequency for the elimination of selection bias ? Intuitively (ie probably incorrectly) I would think that a different system design would excel in this environment than the ones we have been designing to date ? Did you find this Julian ?ps..I will enjoy spending hours testing different variations but keen to hear your thoughts Julian
try lowering the stretch…you’ll see the fill rate climb dramatically. That’s where I was managing to achieve the trade frequency
-
AuthorPosts
- You must be logged in to reply to this topic.