MRV System From Amibroker to RealTest

[GlenPeake]

Just trying to get my Mean Reversion System (this is my MR#2 that I post in my journal) up and running in RealTest.

To establish that I’ve got it coded/working in RT correctly I’m running a backtest in Amibroker and then trying to get the same results in RT.

In a nutshell, the RT test results are not matching that of Amibroker…. so I’ve probably messed up something within RT… or missed activating a setting possibly. e.g. I believe RT defaults to ‘Unadjusted price/volume”, as I thought it might have been something along those lines.

Amibroker Stats: CAR/MDD 40.89% / -25.34% —- Ending Capital $24.2 million
RealTest Stats: CAR/MDD 34.13% / -31.69 —- Ending Capital $10.8 million

For the guys that are on top of RT….Any tips on troubleshooting you systems in RT? I know it has various debug options but not sure how to proceed from here atm etc…

I feel a little blind without any Chart plotting to cross reference against etc…. e.g. (as far a I know,) RT won’t plot a BUYSETUP / SELLSETUP arrow on the chart (like what we have in our Amibroker AFL template etc), sure you can toggle through the Trade List Charts for a completed trade etc but when you want to just confirm your code is working as expected on the chart for a SETUP bar, it’s a little like flying blind atm. Any suggestions on what worked for you when coding up your System Scripts in RT and double checking etc?

Of the sample of trades that I cross check RT vs Amibroker, the entries and exits appear to be ‘in-sync’ (so far)…. I just appear to be seeing less trades in RT (approx 300 less trades in RT) and a slightly different list of trades when checking RT vs Amibroker etc.

A snapshot of ‘parts’ of my RT code below (any obvious issues that you can see?). Keeping in mind that this is for a multi day Mean Reversion system with no leverage and has a StaleExit if my SellSetup exit doesn’t get hit after X days etc I’m also using an ADX Index Filter on the SPX and my Universe is the Russell1000 (which I think is working OK…..again it’d be nice to ‘APPLY’ an Index Filter RIBBON to the bottom of the chart to double check etc).

(Here is a trimmed down version of my RT Script….I realise it might be a little tricky providing suggestions, without seeing the entire System script, as I’ve omitted some of the more ‘specific details’ of the system etc… but we’ll go with this atm, in case there is something obvious I’ve missed).
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Import:
DataSource: Norgate
IncludeList: .Russell 1000 Current & Past // include all Russ1000 in the test
IncludeList: $SPX // import $SPX symbol if want to use for Index Filter
Constituency: $RUI // Index symbol to check for constituency
StartDate: 1/1/2000
EndDate: Latest
SaveAs: DataRUI1000.rtd

TestSettings:
DataFile: DataRUI1000.rtd // load above ‘file’ to test on
StartDate: 4/1/2005 // earliest date in file
EndDate: Latest // latest date
AccountSize: 100000

Parameters:
//

---

// Position Details
NumPos: from 5 to 20 step 5 def 5

// Price Filter
MinSP: from 1 to 500 step 5 def 20
MaxSP: from 1 to 1000 step 10 def 250

I have various other parameters (similar to what we use in Amibroker for the filters etc) in here that I later refer to under the DATA: section etc

Data:
//

---

// Price Filter
PriceFilter: C >= MinSP and C <= MaxSP
//

---

I have various other data options here, which I ‘think’ are working ok… but would be good to check ‘somehow’ etc

//

---

// Entry Conditions
Conditions: PriceFilter and RSIStockFilter and ADXIndexFilter and InRussell1000 and MA1StockFilter and BarEntryFilter

Template: MeanReversion
MaxPositions: NumPos
MaxSetups: NumPos – S.Positions // This simulates the CBT code in Amibroker – only place how many orders you want filled // could probably just use numpos because positions only last 1 bar
SetupScore: 100 + RadgeRanking // Ranking as per template
Allocation: S.Equity // compounding equity – Use s.startequity for non compounding // marsten reccommends non compouning. Stats like MDD will show % of start equity so it’s easy to see
Quantity: S.Alloc / NumPos / FillPrice // 5% allocation
Commission: Max(1, 0.005 * Shares ) // IB commission

Strategy: MR2_long
Using: MeanReversion
Side: Long
EntrySetup: Conditions // entry as per conditions variable calculated in data section
EntryLimit: BuyLim
// ExitRule: BarsHeld = 0 // exit on entry bar
ExitRule: SellSetup or BarsHeld = StaleExit
// ExitTime: ThisClose // exit at close
ExitTime: NextOpen

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

If I compare an order list RT vs Amibroker on a random day, the StockList and BuyLimit price match up and are the same, which would suggest that my RANK and BUYLIMIT stretch entry seem to be working as expected.

However if I go to the very first trades for the start of the backtest, there are mismatches from the very beginning e.g. AAPL is the first trade in Amibroker however RT doesn’t have it…..

Snapshot of a 2 year backtest comparison highlighting the stats differences:

I’ll keep plugging away to see if I’ve screwed something up elsewhere.

[TrentRothall]

Hey Glen,

Looks ok to me i think from what i can tell. RT Defaults to as traded price at entry time. So for example TSLA a few months ago would’ve been > 2000 (or whtever it was) so you shouldn’t have to touch anything there.

fyi you don’t need to put stuff in the parameters section it is mainly for optimizations etc. The figures can go straight into data.

PriceFilter: C >= 20 and C <= 250 See suggestions below 1. to check constituency use InRUI, i see you have InRussell1000 2.I would suggest running a shorter backtest to make it easier (less trades may make it easier to spot) 3. In the Debug panel type APPL (as Per you trade example.) then the date and press ‘Data’ this will show which conditions are true or not. You can also use Test OutPut > Log to show why trades were taken or why they might’ve been skipped.

4. if you want flick me a email or text. [email protected] – 0402824154

[GlenPeake]

Thanks Trent… I’ll double check a few things.

Even the shorter backrest i.e. 2019-2020 posted above is out of sync with Amibroker…. so a little strange atm.

I’ll remove the parameters part and just try using the raw figures in data to see if something is getting messed with somewhere.

One thing I thought might be a possibility is the Ranking might be a little out of sync i.e. in Amibroker we had to adjust the number of digits after the decimal from 1.2 to a larger number i.e. 1.8… so I was thinking that maybe RT might be doing something similar with the Rank.. which might explain the mismatch with some of the trades etc. So I’m not sure what RT defaults to in terms of the ranking digits etc.

I’ll try some more testing over the weekend and see what I get.

Thanks Trent

[GlenPeake]

UPDATE:

I setup a SCAN in my RT Code to look at the RANK value output.

I filtered on my BUY Conditions and then added a Column for the RANK output.

e.g.
I needed to add a SCANSETTINGS and a SCAN section.

ScanSettings:
DataFile: DataRUI1000.rtd // load above ‘file’ to test on
EndDate: 31/12/2004
NumDays: 1

Scan:
Filter: PriceFilter and RSIStockFilter and ADXIndexFilter and InRussell1000 and MA1StockFilter and BarEntryFilter
Ranking: 100 + RadgeRanking

The output displayed a ranking with ONLY 2 decimal points……. so I suspect this could be causing the mismatch in trades between Amibroker and RT….

I posted the question in the RT forum on how to increase the ranking from 2 to 8 decimal points.

RealTest Scan, showing the RANK appearing as just 2 decimal points.

Hopefully this is the root cause for the mismatch in trades between Amibroker and RT.

[TrentRothall]

Did this work Glen? I saw your post in the forum

[GlenPeake]

Trent Rothall wrote:

Did this work Glen? I saw your post in the forum

Hi Trent,

Nah, no luck atm.

I modified the rank output to 8 decimal points using the following syntax:

Scan:
Ranking: {#8} MyRanking

And then ran a SCAN, which showed the rank to 8 decimal points etc.

Also made the same adjustment in the Template

Template:
SetupScore: {#8} MyRanking

One thing I noticed is that there was a difference in the RANK value output, so a bit more debugging required.

e.g.

In your systems, when using an 8 decimal point value in the output, are you able to see a difference between AB vs RT or are they the same?

I also removed the Parameters etc… and went with just ‘RAW’ values…. which produced the same backtests results.

[ScottMcNab]

Hi Glen,
Having Pad and Align on or off changes my back test slightly and I assume this is due to changes in the ranking scores (but may also be due to changes in indicators). I am wondering if this is a possible cause of the discrepancy. Is there any documentation or discussion in the forums describing how RT deals with this issue ? Perhaps toggle it on and off in AB and see if they agree.

[GlenPeake]

Thanks for the suggestion Scott.

I believe in RT it is off by default…you need to add the padding command into your code etc.. so I’m running RT backtests without padding.

In Amibroker, with/without padding the backtest results are pretty similar i.e. 1-2% difference….

I’ll try coding up my other MRV systems in RT and see if the statistics are in the right ballpark or not.

[GlenPeake]

I tested one of other MRV systems and although the RT stats were closer to Amibroker, they were still a little off…

Amibroker Stats: CAR/MDD 40.44% / -21.52% —- Ending Capital $20.9 million
RealTest Stats: CAR/MDD 34.84% / -20.77 —- Ending Capital $11.8 million

@ Trent: Thanks….This is handy: Test OutPut > Log to show why trades were taken or why they might’ve been skipped.

I thought I may have incorrectly configured my index filter, but used the SCAN option in RT to collect the days that RT had determined that my Index Filter was OFF and then compared to the Amibroker output…. all in SYNC there, so all good.

Below is how I used the SCAN option:

////////////////////////////////////////////
Parameters:

// ADX Index Regime Filter
ADXIndexPeriod1: From 1 to 100 step 1 Def 5
ADXIndexLevel: From 1 to 50 step 5 Def 25

Data:

// ADX Index Regime Filter
ADXIndex1: adx(ADXIndexPeriod1)
ADXIndexSPX: Extern($$SPX, ADXIndex1)

ADXIndexFilter: ADXIndexSPX > ADXIndexLevel

Scan:
//Filter: Symbol=$$SPX
Filter: Symbol=$$SPX and Not(ADXIndexFilter)
ADXIndexOnOff: ADXIndexFilter
ClosePrice: Close

//////////////////////

I’ll keep plugging away with the debugging.

[TrentRothall]

Interesting, still a big gap hey…

you can also use the debug formula editor to output certain calculations for a symbol, i’ve used it a few times. I agree that AB is better for visualizing a trading system on charts.

[GlenPeake]

As a sanity check, I used Trent’s RT MOC script example he posted in another thread and then modded the Amibroker MOC AFL template so the systems were coded the same in AM vs RT.

I ran a backtest in each and compared…. results were pretty similar i.e. in the right ballpark (what you would expect etc).

Trades were matching i.e. same trades on the same days etc.

e.g. Amibroker vs RT

Not sure at this stage for the mismatch in the multi day MRV in RealTest atm…. I’ll keep troubleshooting.

[JulianCohen]

Glen did you send your MRV script to Marsten to check over? It might be worth it. He’s quite happy to check it from my experience…especially if there’s a chance it might show a bug

[TrentRothall]

have you ran a exploration in AB and compared it to a RT scan or orders list?

it probably only takes a few different trades to skew the numbers over a larger test, especially for multi day holds from the flow on effect of not being able to take trades. might even be a good test for robustness? If both look good then happy days! Live trading is more random than the 8th digit in a ranking formula imo.

[GlenPeake]

Thanks for the suggestions guys.

I’ll start from scratch and try backtesting the AFL MR Template “CBT template_mean reversion_with looping.afl” in Amibroker and then coding it up in RT and running a few comparisons and go from there.

[GlenPeake]

UPDATE:

Made some progress on the differing backtest results.

In a nutshell, I needed to remove excess signals from RT…. similar to the EXREM option in Amibroker.

More details on the RT forum:
https://realtest.discoursehosting.net/t/remove-excess-signals/475

More testing required, but the results in RT are now looking much closer and in the same ballpark to Amibroker….

ATM, I have added the following code in RT to workaround the excess signals….

//////////////////////////////////
Benchmark: All_Signals
Side: Long
Pyramid: True
EntrySetup: Conditions
Quantity: 1
ExitRule: SellSetup

Strategy: MR3_long
Using: MeanReversion
Side: Long
//EntrySetup: Conditions // entry as per conditions variable calculated in data section
EntrySetup: Conditions and Extern(@All_Signals, Shares==1)
/////////////////////////////////

[Author]

Posts