Oct. 10, 2022

Here is a post I made (as is) about a walkforward on a QuantConnect thread. It should be to your benefit and, therefore, worthwhile.

The post presents one particular pitfall of automated trading, and that is no matter how promising a trading strategy might be, it could still go wrong and not perform as expected.

I would like to use a version of the "In & Out" strategy (frozen in time since October 2021) as a one-year walk-forward example.

The price data after October 2021 will be totally unknown to the strategy. It is assumed the strategy will continue to do what it was designed for and simply execute its code. After all, it was designed to do just that.

The strategy received a lot of praise for its outstanding performance: 83.198% CAGR over the period from 2008 to October 2021, and this, with an 86% win rate. Exceptional. Just based on this remarkable CAGR performance and portfolio metrics, people might have started trading it live at the time. Hopefully not.

Payoff Matrix Equation

The above payoff matrix equation covers the period up to τ (October 2021) to which is then added its walk-forward part up to t (meaning current date).

The first part of the payoff matrix covers the first n trades up to time τ. The period after τ has all the other trades (N - n) that occurred after October 2021). N is the total number of trades over the entire trading interval up to t.

The overall expectation was that the market, going forward, would behave as it did over its past 13 years. And therefore, the trading procedures would continue to do what they did, and this is: continue to prosper.

We should expect the market average rm to stay about the same over the period. The same goes for the strategy's alpha (its excess return). Nonetheless, the market average during the walk forward could still be different. It is only if (t - τ) would be large enough (some 20+ years) that we could expect rm to tend to its long-term average expectation. Over the short term (one year), it might not be the case, as this case shows.

Based on the above equation, you might not be able to evaluate the walk-forward period without actually living it. Any estimation for the period (t - τ) might be just an extrapolation, some kind of wishful forward disposition with absolutely no guarantees that it could even happen.

You have to live that period ((t - τ)) doing live trades just to see what would have happened. And if you did, there would be only one solution, and that is the above equation, where all the variables would be identified with their before and after values. The only outcome would be this walk forward since trading strategies, by their very structure (programs), only consider one portfolio path.

To demonstrate the equation, I used the "In & Out" strategy (October 2021 version). Here is its QuantConnect URL address and code: Amazing returns Strategy as of October 2021 (It could take up to15 seconds to display, so wait for it a little).

The cited simulation shows its results up to October 2021, and below, you will see what happened next. The strategy did not know and could not know what was coming its way. It would have to make do following its preset trading procedures.

We can observe a total breakdown of the strategy with an -85% drawdown. By definition, no one is intended to trade during a walk forward; it is just another phase in the simulation and testing process. But, for those who opted to trade that strategy live without first adding downside protection, it must have been misery.

The strategy still made money, but it was down -85% from its peak after October 2021. That was somewhere around a \$455 million loss for a strategy that started on a \$100,000 initial stake. If you added some 30% leverage as could be used in a 130/30 strategy, the drawdown would have been -92%. Almost a total collapse.

So, What Went Wrong?

The initial assumptions made, the trading rules, and what was classified as trade-triggering signals did not hold up going forward. Yet, the trading environment from 2008 to October 2021 had a generally upmarket for most of the period.

It was as if most of the trading signals were wrong, as I already stated in another thread on another version of the "In & Out" strategy. But, in an upmarket, being wrong did not matter so much. The tide can lift all boats, as some say.

I think what was mostly missing were protective measures for when the market might be heading south or when the logic behind the signals would stop working. We should design trading strategies for what is coming next and not necessarily for what was. A simulation is just to show how your strategy would have behaved, showing you what might have happened if such and such. The important part is what happens after.

Another point of interest would be that the code modifications made after October 2021 were to improve the program based on the knowledge of the added data points. And that, in reality, becomes incremental curve fitting, but curve fitting nonetheless. The "oh, let me arrange that" after the fact is not a viable scenario in trading.

This is not an attack on this trading strategy. What it is, however, is an observation, an example of what happened before τ and what followed. It provides a perfect example of a walk forward.

There are at least 5 versions of the "In & Out" strategy. In one of them, last year, I recommended adding downside protection and further improving trading signals before use. I estimated at the time that 80% of the trading signals were wrongly timed. It would appear it was not bad advice.