Can ChatGPT Create a Trading System?

With the AI boom in full swing and seemingly every piece of software now including some sort of AI assistant, a question that has been asked consistently on trading forums for the last few years is “Can ChatGPT help me create a trading system?” It’s a simple question, but as always, the answer is not so simple. 

ChatGPT is a Large Language Model (LLM), a type of chatbot ‘AI’ capable of eerily human conversation and answering pretty much any (legal) question you ask of it. But herein lies the problem for us. ChatGPT and other LLMs exist to converse with, not to solve logic problems. While it can draw on a vast array of information to knit new ideas, the way it combines this information can be flawed. 

One easy example of this is asking ChatGPT to count something. 

While it started out well by correctly identifying 11 letters in the word “accommodate”, when I asked it a more specific question, the answer was incorrect. It also failed to correctly answer a more obvious version of the question, how many U’s are in the word “accommodate”. So, what we can see is that while it is clearly capable of confidently answering a question in a way that looks correct, the contents of those answers should be scrutinized.

Keeping this in mind, let’s get it to design a trading system. The first thing I will do is teach it some RealTest functions. I do this by simply providing the full list of RealTest functions and their explanations and asking it to create for me a trading strategy.

Here is the output provided: 

In all honesty, I was surprised at how well it was able to do this. There are some obvious syntax issues, but we can work with it. The strategy it has described here is a momentum breakout based on a moving average crossover. This is a well understood indicator, so it’s a good base to build on. It’s also included a Relative Strength Index (RSI) indicator, using this to determine whether a stock is overbought or oversold as an entry and exit.  

I coded this up in RealTest, fixing the syntax so it actually runs. I also altered the position sizing to 10 positions at 10% (instead of a flat $10k) and made sure commissions were accounted for. I then asked it which market or index to trade it on, and it suggested “any market or index that exhibits trending behavior and has suitable liquidity for trading”. I therefore selected the S&P 500. 

Backtesting the strategy for the first time from 2010 onward, the results were as follows: 

While the Maximum Drawdown is unreasonable for the rate of return achieved, we can build on this. For the sake of this experiment, I asked ChatGPT how to do this. Its response was to alter every variable, lowering both EMAs, the RSI period, and the oversold level, while raising the overbought level. It also added some dynamic position sizing based on ATR. The results were as follows:

It’s a decent improvement on the MaxDD, but the Rate of Return still isn’t cutting it. When asked to improve the system again, ChatGPT gave me the same instructions for the original system. I asked again, and it suggested to: 

• Continuously backtest and optimize your trading strategy using historical data 
• Stay informed about market news, economic events, and geopolitical developments. 
• Implement strict risk management protocols to protect capital and limit downside risk. 

Decent suggestions, but it seems to have run out of ideas.

So, what we can see here is that ChatGPT has obviously picked up information about a trading system, but without practical knowledge, it cannot improve on those ideas. And of course it can’t; that’s not what it’s built for. Furthermore, using its conversational abilities, ChatGPT was able to explain the purpose of each of the strategy elements but was not able to use that explanation to develop the strategy any further. 

The other obvious thing to mention here is that ChatGPT has no access to a backtest engine, so is only offering hypothetical suggestions. However, even if it did have access to software like RealTest, it has no means of evaluating the results, because it has no concept of what improvement is in this situation. 

Taking a brief look into the backtest results of the strategy, it’s clear that the exit rule given is too tight, often exiting and immediately re-entering the next day. Simply changing the or condition in the exit rule to an and brought the RoR up to 24%, but the MaxDD worsened to -52%. The next step to improve on this could be looking at something like a regime filter, since the biggest loss across the period occurred during 2020.

If you would like to learn more about designing and optimising trading strategies, consider our Beginner’s Guide to Building Trading Systems.