IsIndexConstituent Code

[Stephen James]

In the past we have just used the IsIndexConstituent code in backtesting versions of systems to identify that a buy signal takes place at the time that the stock was a member of a particular watchlist or universe. We had not used it in operating versions but recently came across an example of why it may be prudent to also add it to operating versions of systems.

This occurred due to recent quarterly rebalance of ASX indices which took place at close of trade on March 18. Example as follows using the All Ordinaries as an example of our trading universe:-

Without IsIndexConstituent code in the buy setup conditions:-
Stock XYZ generated a system buy signal in early March but was not a member of the XAO so the trade was not picked up.
On March 19 it is included in the XAO so an open trade now appears in the exploration with the entry in early March (no big deal as we know why it is there).
However, a backtest using IsIndexConstituent code after the rebalance date shows an open trade with an entry on 23 March as the system had produced another entry signal.

Add IsIndexConstituent code:-
No entry in early March.
A correct entry now shows on 23 March after the stock was added to the XAO.

The IsIndexConstituent code should be included therefore in the setup conditions of operating code.

[TrentRothall]

Craig, do you think we should add the IsIndexConstituent to all operating versions including API code? I figure it is probably not as important for shorter term systems but the above could still happen.

[Stephen James]

Yes, it would be a good idea just to make sure.

[JulianCohen]

Is it a good idea to use the Historical Database in order to ensure that the stock was a constituent of the Index instead of relying on Premium Data’s watchlists?

I have found I get a different result if I run the Historical Database IsIndexConstituent code for S&P 500 AND choose S&P 500 Historical Watchlist than I do it I just use S&P 500 Historical Watchlist

[SaidBitar]

I think you are mixing stuff

the Historical database has all the symbols that were traded on the S&P500 at any stage in time.
IsIndexConstituent will help you to know exactly at any specific point in the past if the stock was or wasn’t in the S&P500.

example:
SSP it is in the Historical database for S&P500 but it was only from 20/5/2005 until 30/6/2008
so you still need to use IsIndexConstituent

[JulianCohen]

Thanks a lot Said!

[LEONARDZIR]

I do have a question about using the operating code. When I use the Op code on one of my systems and run explore I get different results if the watchlist in the filter is SP500 rather than SP500 historical constituents. Why is that?
Len

[Stephen James]

Len,

The historical watchlist contains stocks that were once in the S&P but no longer, some of which may fit the exploration conditions. So for operating code (day to day system management), include the IsIndexConstituent code for your chosen universe and run on current universe watchlist, not historical. This will then account for any future changes to the index.

[LEONARDZIR]

Craig
Thank you
Len

[StephaneFima]

Craig,

There is something I do not understand quite well: why not using adatabase including historical constituent AND using IsIndexConstituent ?

Thanks

[Stephen James]

Stephane

The watchlists ‘Current and Past’ for the NDU data provide a list of symbols that were or are a member of a given index at any point in time.
The code NorgateIndexConstituentTimeSeries or the old IsIndexConstituent identifies that the symbol was in the index at the time of a signal (buy or setup, depending how coded).

Is that what you mean?

[StephaneFima]

In fact, I do not understand why we have to use NorgateIndexConstituentTimeSeries to check whether a stock was in an index or not if a “current and past” watchlist already provides this.

[SaidBitar]

the reason is that you don’t know when the stock joined the index and when it left it.
so with this NorgateIndexConstituentTimeSeries it will check if at any specific date the stock was trading as part of the required index or not and this is the way to eliminate the survivorship bias

run the following code on the S&P500 current and past and you will see what i am talking about.

Code:

#include_once “FormulasNorgate DataNorgate Data Functions.afl”
Filter = 1;
AddColumn(NorgateIndexConstituentTimeSeries(“$SPX”),”In S&P500″);

if you check AAL until 23rd of March 2015 was not in the S&P500

So the only way to know this is by using NorgateIndexConstituentTimeSeries otherwise you may have traded AAL in 2014 as part of the S&P 500 while in reality this would have never happened

[Stephen James]

The watchlist only provides a list of the symbols that have been or are in an index. It alone does not tell us when a symbol was in the index.

The NorgateIndexConstituentTimeSeries code returns 1(true) or 0(false) for each bar (1 when the symbol is a member of the index, 0 when not).

Therefore it can tell us if the symbol was in the index at the time of a signal. That is a distinct difference to just a list of symbols.

[StephaneFima]

Thank you Craig, this is very clear now!

[Author]

Posts