September, 08, 2011
Not everyone needs to do back-testing, and therefore, to some, it is not a worthwhile endeavor. There are thousands of ways to play the stock market game, and a lot of them do not require back-testing at all.
For instance, I am sure Mr. Buffett does not do any kind of back-testing, nor does his staff have any time for it.
Mr. Buffett's investing methodology is based on his great understanding of the fundamentals of the markets (he reads some 200 annual reports a year), his wide experience, and the fact that he views the market in a long-term uptrend that has lasted for decades. He is ready to bet and has bet that this secular trend will continue for some time to come.
For other reasons, there is no need to back-test for momentum and discretionary traders following their hard-earned experience on how to play and win.
Back-testing is for those wishing to automate high trade volume over shorter trading intervals. For those wishing to delegate the decision-making process to a program that will execute without emotions its required task, and for those wishing to find a long-term tradable and profitable “edge”. All a back-test can say is if, statistically, your chosen trading rules would have, in all probability, produced a profit in the past.
The shorter the trading interval, the higher the random nature of price movements, and the harder it is to find trading methods that consistently make money out of the chaos and that can outperform someone like Mr. Buffett over the long run. It is not enough to find a profitable “edge”; it must also beat, as a bare minimum, the Buy & Hold strategy. Outperforming Mr. Buffett long-term is the real ball game.
The Challenges
The challenge is considerable. You need to find a trading methodology that can thrive in a harsh, almost random-like trading environment with unforeseen and unforeseeable events like earthquakes, tsunamis, or nuclear melt-downs, not to mention all the cheating that can go on in the markets at every level: frauds, book cooking, manipulations, mismanagement, and outright incompetence. Add product obsolescence, technological innovation, and failed biotech trials to the mix. It becomes almost normal to view the future as a series of uncontrollable random events. And with globalization, all markets get interconnected, so indirectly, one would have to predict future prices from a global point of view.
No back-test will be able to predict unforeseeable events. “Black swans” are statistical aberrations (outliers) and, as such, cannot be predicted. However, you can play the market in such a way as to profit from “black swans” went they do occur. That is a totally different problem.
It is not because you have detected a statistically “profitable” scenario over past data from a specific group of stocks that this back-tested “edge” will necessarily be maintained in the future. Far from it, the future will have its own unique signature. This is probably why most trading systems break down when going forward.
I do think, however, that the main reason for trading system breakdowns is due to simplistic and incomplete trading procedure designs (like designing a dip-buyer strategy with no stops). They break down because their trading strategies were flawed from the very beginning, even on positively back-tested data. There is no one size fits all and there is no single strategy that will outperform no matter what is thrown at it. As a minimum, the selected stocks should have a tendency to move in the same direction as the underlying trading philosophy; it makes no sense to buy or accumulate a triple inverse ETF in an up-trending market.
Automated Trading Rules
A back-test is simply a set of automated trading rules and procedures performed on past market data. Its purpose should be mostly to provide a kind of proof of concept. Doing this or that, would these procedures have a positive outcome? If not, why pursue that avenue of research? And if yes, then what were the reasons for the out-performance, will they still prevail in any future scenario, can they be enhanced or scalable, are they still valid over other groups of stocks, over other trading intervals? All legitimate questions that need answers.
There are many kinds of traders who do not need to do back-testing based on their respective trading methodologies (ex: discretionary traders). However, even though back-testing is not for everyone, it does not mean that back-testing has no value; it is just another tool, and like any tool, it can be abused and misused. It can also be a great tool guiding one's search in the right direction for automated trading profits. Should your back-test show negative outcomes for various groups of stocks with different trading intervals, wouldn't it be wise not to implement those trading rules on real-life data? Surprisingly, I like it when I find very bad trading rules as I immediately explore their opposites.
Your Unique Perspective
In a way, your back-test, based on your understanding of how prices move and how you should react (trade) based on these price fluctuations, could be considered as your own game within the game. You are carving out procedures out of the thousands available or creating your own set of rules based on your vision of this trading universe and, as such, become a unique, one-of-a-kind player, trader, or investor. In the final analysis, you are the one pulling the trigger, be it by hand or by program, based on your vision and understanding of your trading environment where you have to set when to trade, determine the risk you are willing to take, and quantify within your portfolio constraints the size of your next trade knowing that this last trade will not be the only one.
Back-testing over past market data provides no profits at all; not a penny to be made. However, from back-testing, one can design trading rules that will prevail in the future, even if the future may come around only once. And maybe there lies the problem: you can do thousands of back-tests on thousands of datasets of varying length using thousands of different trading rules, but in the end, you will have to choose one, the one that you will play your future on, even just for a time, once.
A Special Note:
The old Wealth-Lab 4 website Chartscript Center has over 1,800 scripts and represents a considerable bank of knowledge on trading methods and concepts, all available free. Trading scripts span 10 years, and all scripts prior to 2005 will not have seen any current data; thereby, a special kind of walk-forward, and no script has been modified since mid-2008 except for the last 9. A lot can be learned from all those trading scripts. (Note: that website is no longer available 12/04/2018)
Created on ... September 06, 2011 © Guy R. Fleury. All rights reserved.