Julian’s Journal

[JulianCohen]

Trent Rothall wrote:

Does the spread between Min & Max MCS tighten very much once you have reduced the selection bias that much Julian?

I guess that is the desired result??

I’m just doing tonight’s entries. Then I’ll run the MCS and let you know. Gimmee three hours

Technically the desired result was to get the selection bias better than 80%. I’m hoping it doesn’t affect the MCS variance too much.

[TrentRothall]

Julian Cohen wrote:

I’m hoping it doesn’t affect the MCS variance too much.

Shouldn’t the variance be tighter now though because you are catching a higher % of trades?

[ScottMcNab]

I found some similar results with VIX Julian if you want to play with that too

[JulianCohen]

Trent Rothall wrote:

Julian Cohen wrote:

I’m hoping it doesn’t affect the MCS variance too much.

Shouldn’t the variance be tighter now though because you are catching a higher % of trades?

Blooody hope so!

[JulianCohen]

Scott McNab wrote:

I found some similar results with VIX Julian if you want to play with that too

I have played with the VIX but not for a while. How did you apply it Scott?

[ScottMcNab]

I tried it both as a filter for changing the size of the stretch for the buylimit and also as an extra condition in the buysetup… I was trying to test it to change positionsize too (ie switch to 40 positions at times of high volatility and drop to 20 in low volatility) but the coding was beyond me

[JulianCohen]

Julian Cohen wrote:

Trent Rothall wrote:

Julian Cohen wrote:

I’m hoping it doesn’t affect the MCS variance too much.

Shouldn’t the variance be tighter now though because you are catching a higher % of trades?

Blooody hope so!

original: CAGR was 25%-36%
Now 16% – 23%

MDD Original was 18%-11%

Now 15%-8%

[TrentRothall]

How does that sit with you? Is it better to reduce SB or just accept tthat you will miss a lot of signals but still have a higher CAR. That is the main issue with selection bias in my eyes in those results.

[JulianCohen]

Trent Rothall wrote:

How does that sit with you? Is it better to reduce SB or just accept tthat you will miss a lot of signals but still have a higher CAR. That is the main issue with selection bias in my eyes in those results.

Well I would welcome Nick’s input here but my take is this. The backtest that produces a 30% odd result is affected by selection bias. In real life I would probably not get those returns on a long term basis…I say long term as so far I have got those returns, but I have to put that down to luck.

The reduced selection bias system has poorer returns in the backtest, but I believe that I am much mod=re likely to see those returns in real life over a longer period.

Nick…is that reading of the situation correct?

[TrentRothall]

I am no expert obviously, but i tend to agree. The only thing is the minimum MCS in the first run was higher than the Top of the second..

[JulianCohen]

Trent Rothall wrote:

I am no expert obviously, but i tend to agree. The only thing is the minimum MCS in the first run was higher than the Top of the second..

I know (gutted)…but if it is an intrinsically flawed MCS due to the selection bias then it doesn’t matter how great it looks. After all in theory, theory and practice are much the same. In practice, they are not.
I have hundreds of these by the way

[ScottMcNab]

Systems or anecdotes ?
Or both ?

[JulianCohen]

Scott McNab wrote:

Systems or anecdotes ?
Or both ?

Ha ha…both!

[ScottMcNab]

Trent Rothall wrote:

I am no expert obviously, but i tend to agree. The only thing is the minimum MCS in the first run was higher than the Top of the second..

Interesting….I have thought the same thing Trent. One really rough rule of thumb I have found is that when I tested those days where possible buys far exceeded available positions, the live results have consistently (in all of the half dozen times…hah) been about 1% worse than the backtest result for that day….so if were 12 instances in the year when this occurred, then I subtract 12% from the CAR…if it occurred twice a month then thats a deduction of 24% !!

last night was a good example….lost 1.5%…turns out when I tested it a few minutes ago there were 70 possible buy signals …when ran backtest over the day 1000 times (with positionscore set to random) the possible outcomes ranged between -0.11% up to -1.6%…with an average about -0.6%

not sure if that helps Trent but I have used that rough rule to help me (grudgingly) face the reality that I don’t really want to confront (ie my beloved system is really a dog)

[TrentRothall]

Scott McNab wrote:

Interesting….I have thought the same thing Trent. One really rough rule of thumb I have found is that when I tested those days where possible buys far exceeded available positions, the live results have consistently (in all of the half dozen times…hah) been about 1% worse than the backtest result for that day….so if were 12 instances in the year when this occurred, then I subtract 12% from the CAR…if it occurred twice a month then thats a deduction of 24% !!

last night was a good example….lost 1.5%…turns out when I tested it a few minutes ago there were 70 possible buy signals …when ran backtest over the day 1000 times (with positionscore set to random) the possible outcomes ranged between -0.11% up to -1.6%…with an average about -0.6%

not sure if that helps Trent but I have used that rough rule to help me (grudgingly) face the reality that I don’t really want to confront (ie my beloved system is really a dog)

Cheers Scott, that is interesting with those results.

Even if it is only half or a quarter as bad as your observations then that’s a decent wack!

I know the feeling!

[Author]

Posts