Reducing Entry Signals

[LeeDanello]

Scott, the resolution of the image makes it difficult to see the detail. I post my images to postimage and then share the image by copying the image link (share then direct link) and pasting it using the image link from this forum.
It’s free. Example below.

]

[ScottMcNab]

Here goes

https://s22.postimg.org/471ac3owh/buysignals2015.jpg

[LeeDanello]

Better. Wow 3 days where the system goes to the extreme. They must have been huge down days. Obviously not statistically significant.

[Anonymous]

Trades taken as a % of possible trades:

This has been discussed already I am just trying to figure out if my logic is flawed.

I ran tests on my MRV to see if I have a potential selection bias issue.
(MOC I know I have an issue. Reasons why have been made quite clear.)

I ran an exploration on “Buys” which gave me all available buy trades, summed them up per day and then looked at how many days the number of buy trades was more than my available max position (I use 20) and found 5.74% – so border line on the recommended 5%, but as it has been performing I am giving it some slack, neither here nor there.

I then ran a backtest using max 20 positions and got the number of trades taken. The number taken as a % of total was around 65% which is off the 80% mark.
Is this an issue based on the below logic?

Here is my logic, have I missed something:

Assuming 20 postions:
Day 1 – 10 available trades and 10 triggered
Day 2 – 10 available trades and 10 triggered – there are no exits so I am at max
Day 3 – 10 available trades – none taken

If I were to use my “Buy” exploration I would get 30 available trades, but I have only taken 20 = 66.67%, however I have captured 100% of the trades and my backtest would = actual based on this logic, but if I use 66.67% as a guide I may think that I have a selection bias issue, which in this example there isn’t?

Does this make sense and am I applying the principles correctly?

[JulianCohen]

Miguel I don’t think you are applying the logic correctly.

Here is my take on the situation. Firstly I don’t think it matters how many buy entries you have on any given day. What is important is the total number of entries that it was possible to have filled on any given day; the actual number of trades.

To check this run one backtest at 20 positions over say 5 years. Note the number of trades that were executed. Then change the allowed positions in the backtest to 200 and the position size to the comparative amount.

For example I am running 20 positions at 10% so I run the second test at 400 positions allowed to take at 0.5% (I am being cautious to make sure I cover every single trade, 200 at 1% will probably suffice).

This will give you the total amount of trades that it is possible for your system to take if every signal that it was possible to fill were filled on. If (test1/test2) < 80% then you probably have a system that is in danger of too much selection bias and you might want to rethink the entries. I hope my thinking is correct or it’s back to the drawing board for me:)

[LeeDanello]

You’re right Julian. I would run the test over 20 years

[Anonymous]

Thanks Julian, what I have done is run it through an exploration instead of, as you suggested changing the backtest run, either way it will give the same answer. I am just questioning the logic of dividing the number of trades taken by the total number that could have been taken as the backtest will take that into account as in the simple example I have given.
After writing that though I can think of a few more permutations that will make that logic incorrect, the same goes for how many of the days there have been more buys than positions available.

Example:
Day 1 – 10 potential buys – 10 taken
Day 2 – 5 potential buys – 5 taken
Day 3 – 10 potential buys – only 5 taken (max 20 positions) but now I have a selection bias on 5

How many of the days have the potential buys been higher than available. Answer = 0%, if using a simple formula of counting the number of potential buys on a day and comparing that to the max position allowed. An additional formula will need to be taken into account to get a cumulative balance and take exits into account.

There are lots more permutations to consider, I agree with the overall selection bias “issue”, I just don’t think the simple formula of doing it this way takes the various scenarios into account, or alternatively simple is best and I shouldn’t be complicating it with different permutations. I am just not sure how “bad” the selection bias problem is by applying the simple formula, but it does create additional scenarios. On a MOC system, there is definitely an issue, but with longer than one day holds, it creates uncertainty for me.

I am questioning how much luck had to do with your system’s profits because of the ratio of trades taken to potential (I have the same thing), if you dig deeper you may find that is because of the way it is calculated and therefore little luck involved because the backtest is taking that into account, to a degree.
Obviously I also think there is a limit to how much this logic could be used, 80% is worth aiming for, just don’t place to much emphasis on luck. (There is a limit of course)
All of that being said I am trying to tighten up my system and aim for the higher “trade capture %”. It has a better feel to it in terms of matching backtest to actual.

[JulianCohen]

Hi Miguel

Yes I see what you mean. However in a non MOC system as per your example, although the daily selection bias is not there, the cumulative one still as as the system could have taken the other trades on day three if it was allowed to, and without the benefit of intra day data it is impossible to know which trades it would have taken. Therefore as you say, that is the selection bias, not knowing which trades it should have taken first.

My understanding is that if we had correct intra day data to work from, then selection bias wouldn’t be an issue.

[Anonymous]

This was bugging me yesterday and “naturally!” occupied my mind. I thought of too many permutations where my logic could be proved to be BS. The luck part of my system was that in the last 6-7 months it has captured over 80% of trades even though the history shows it at ±65%. If that changes what will the results look like. I am not sure if I want to find out.

I am now going to be phasing in the new system that will use the same rules, but narrows the universe by using a slightly different filter. I will hopefully be done with the stress testing in the next week or so and then change over.

Thanks for the input.

[ScottMcNab]

Miguel Teixeira wrote:

Thanks Julian, what I have done is run it through an exploration instead of, as you suggested changing the backtest run, either way it will give the same answer. I am just questioning the logic of dividing the number of trades taken by the total number that could have been taken as the backtest will take that into account as in the simple example I have given.
After writing that though I can think of a few more permutations that will make that logic incorrect, the same goes for how many of the days there have been more buys than positions available.

Example:
Day 1 – 10 potential buys – 10 taken
Day 2 – 5 potential buys – 5 taken
Day 3 – 10 potential buys – only 5 taken (max 20 positions) but now I have a selection bias on 5

How many of the days have the potential buys been higher than available. Answer = 0%, if using a simple formula of counting the number of potential buys on a day and comparing that to the max position allowed. An additional formula will need to be taken into account to get a cumulative balance and take exits into account.

Anyone out there trading MRV or swing systems have a “work around” they use for the issue of holding trades for multiple days/weeks and how this affects calculation of selection bias?

Systems using positionscore have some obvious advantages

[SaidBitar]

honestly it is not as simple as MOC systems where you start everyday clean, the issue whatever logic you will use it will never be accurate. If you changed the maximum open positions from 20 to 100 still you will not get idea of the total trades since some trades you will be in and out within 2 days others it will take more than a week.
Maybe a workaround is to change it to MOC and test the selection bias and assume it is the same with x days holdings. Sounds stupid

Position score is dangerous to use even though i was thinking about this recently that i should give priority for the stocks that have their open price closer to the limit order (percentage wise) over the others since normally they are the ones that are filled first but again this can be used just to get idea on the difference in the performance you can not use it when making the system design.

[TrentRothall]

As Said said it’s difficult. I have a N-day exit in mine and just change it to 1 day to try to capture most signals, rather than accumulating open positions over a week or so. That give you a idea any way.

I find it’s good to set your starting capital to a stupid amount like 10,000,000,000 then 500 or so max positions with 10k per position. Run the system with normal rules then you can get an idea on how every signal stacks up. win %, P/L, etc. Not just the ones a regular backtest or MCS will capture

Nothing perfect in the end, just need some confidence in it

[LEONARDZIR]

When testing my mr system I had less signals than I thought I would have. The reason is that after I accumulated a number of positions which had not reached my profit target they were then excluded as potential buys when I ran my daily exploration even though many of these positions would be buy setups. So to check selection bias I ran my system with the usual 200 positions at 0.5% and seemed to have reasonable selection bias numbers.

[LEONARDZIR]

When testing my mr system I had less signals than I thought I would have. The reason is that after I accumulated a number of positions which had not reached my profit target they were then excluded as potential buys when I ran my daily exploration even though many of these positions would be buy setups. So to check selection bias I ran my system with the usual 200 positions at 0.5% and seemed to have reasonable selection bias numbers.

[SaidBitar]

did not know where to post it but looks like here is good
So i came up with equation that will force Amibroker to trade similar to real life situation especially on the days where there are much more trades than the maximum allowed. this will not reduce the number of the trades but will try to select the trades that are most probable to get in real life.

Code:

PositionScore = IIf(O < Ref(BuyLim,-1), 200, 100 - abs((BuyPrice-O)/O) * 100);

normally we are using random which makes lots of sense but what this is saying in case the open is lower than the Buy limit price give the stock value of 200 (rank) and in case the open was higher measure in percentage the distance between the open and the limit price and subtract the percentage from 100

and finally position score will start selecting starting from the highest value i.e. 200 so we are making sure to capture all the ones that are with open gap bellow the limit entry and then will select the ones that opened closer to the limit entry price.

it is not exact thing but it can give idea on how the system should perform in real life.

I need to check this more in depth but sounds reasonable to me.

[Author]

Posts