On Back-Testing II

October 7, 2011

My back-testing methods usually start the same. I locate a script for one reason or other, try it on a few stocks, then look at the script to explain what I saw on the generated charts and analyze the trading decision making.

In this case, I started yesterday with the: “One Minute Bollinger Band System” chartscript listed on the old Wealth-Lab 4 website. This script is presented as Technique 16 in the book: “Trade Like a Hedge Fund”. (Note: The Wealth-Lab 4 website has been shut down).

You do not need to test that many stocks to see if a chartscript is worthwhile. The total profit generated by the five charts below amounts to $ 2,968 after having taken 553 positions of which 551 were closed ($11,000 in commissions). A total of $500,000 was invested over the 5.8 years test. Roughly, the total profit, on average, was about the equivalent of $100 per stock per year per $100,000 invested.

The original charts as generated by the Wealth-Lab simulator follow:

( click to enlarge))

I am sure the author had back-tested his script over a lot of stocks and must have found it a good performer back then to put it in his book. The original script was released in June 2004 on Wealth-Lab. So this script could be looked at as just another example of a script breaking down when faced with its future.

All the price data being presented now (past 6 years) has never been seen by this script; it is all new data. Whatever type of optimization that worked prior to 2004 will not necessarily be there today. For this script, what would usually be a back-test becomes a walk forward-test. And the big question is: is there anything in this script that can be salvaged, or is there anything that can be added to make it a worthwhile script? From the 5 charts above and looking at the raw performance numbers, one would tend to say no.

One thing is sure, and that is if someone had adopted this script back then, he would most certainly have suffered from a severe case of under-performance going forward.

Script Improvements

You have a script which could be characterized as almost worthless. It is as if it was just gambling, making the bet that after exceeding the low Bollinger Bands by more than 3%, it was reason enough to buy for a 1% potential reversal. And if after 10 periods it had not taken its 1% profit, it would take the loss whatever it might have been. That you can not win with such a trading philosophy should be evident from the above charts. It is akin to gambling and to make the matter worst, gambling for peanuts. I agree, 5 charts are not much to make a point, at least, the outcome was positive.

When I make improvements to a script, I think in terms of procedures that can span the whole trading interval. I do not think in terms of indicators. I visualize what the procedure would do depending on the price movement; even if I don't see the price data. Will it increase the number of profitable trades, will it increase the profit spread or better both?

Below is a list of 18 charts, all on AAPL as I was adding trading procedures. It started with the AAPL chart above to which I would progressively add procedures that I thought would benefit performance. The correlation coefficient between the number of trades to the generated profits is 0.94. This does say that as I increased the number of trades, so did I increased the generated profits. All the charts are in chronological order (numbered by Wealth-Lab), a way to see the evolution and the impact of adding trading procedures.

Trade Acceleration

( click to enlarge))

As I increased the trading volume, knowing that I had generated a profitable spread, (see it as a trading edge), I could scale the trading upwards and reach new performance plateaus. I could push even further, all I needed was to put more pressure on the governing equations. I think that with the procedure implementation as presented in the above 18 charts, I am starting to make the point that trading over a share accumulative process one can improve overall portfolio performance.

To make the point even more concrete, all I should need to do is run the original 5 charts at the beginning using the last version of the improved chartscript. That script can still be improved upon, but that is not the point at the moment. So here are the same stocks as presented above using the improved script.

Generalized Trade Acceleration

( click to enlarge))

For the above 5 charts, the performance went from $2,968 to $83,968,060 over the same investment period all due to the added trading procedures. Total trades went from 553 to 2737, almost a 400% rise. Nothing has changed for the price data series. The only thing that was modified was the management of the inventory over the whole investment period. Now imagine that I could push the total trades to say 10,000 or 15,000, what would be the results. Oh yes, done that in my article: Trade Acceleration.

What might have been considered a “fluke” improvement on AAPL turns out to be applicable to other stocks as well. And then, the improvements cannot be considered as a random phenomenon, but the direct result of the trading procedures themselves.

There was no trading edge in the original script, there still isn't, yet, by transforming its trading philosophy, I was able to improve performance way beyond what the original author expected.

This back-testing procedure is also applicable to your own testing methods. Whatever script you design, it needs to be based on a particular instrument, just as a starting point. You make improvements that can span beyond that one stock on which you are testing. And from there, you expand to other symbols, trying to generalize your trading edge by implementing your methodology in your trading procedures.

I know that to some, even with all these demonstrations that there is something under the hood for this trading methodology, it is not enough. Maybe, they consider that I should give my work away (it is already given away). In all my papers, all the equations to perform what you see in the above charts is provided. Therefore, I think, that anyone using the principles and trading formulas in the Alpha Power methodology can achieve the same results or better. I've often said that there is not only one solution, there are whole families of such equations that can govern portfolio performance. What I provided is the mathematical framework, with its set of equations as building blocks for the design of your own trading system. And if you dig deep enough you might find that there is some wisdom way down there at the bottom, but there is also simplicity.

In one of my papers: “The Trading Game”, I concluded that the best methodology to be used trading is the one used by Mr. Buffett. The reason is very simple, all my equations are a representation of what he does and has done over the years. I only slightly improved on his design.

Good trading to all.