January 10, 2013
In early June 2011, I had this great idea: take a known and publicly available trading strategy and offer a challenge to improve its performance level using whatever enhancements one could bring to the task. I figured it would be a simple way to showcase my own trading philosophy; if it was any good, it would easily show in the test results.
Issuing such a challenge to the Wealth-Lab community was risky; I might find that my own modifications might not be enough or might not do better than other participants. But I had more than sufficient confidence in my methods to know beforehand that whatever the outcome, I could win the competition and with ease. So, technically, it might not have been such a fair competition, me against them? Nonetheless, it would be a great opportunity to learn more about how others viewed the problem and what course of action they would have taken to achieve their own objectives.
I preset the challenge to be on equal grounds, and I think fair to anyone who might want to participate by providing for the simulation 43 preselected stocks to be tested over the previous 6 years (a data set which would enable comparing strategy results between participants and with my other previous simulations using other trading strategies).
Everyone having the same data set meant that everyone had the exact same price series to deal with. The only way to improve performance would be to have a better trading strategy. The way the test was structured made it equal to everyone, and anyone could win. It would be a leveled playing field. The test was not to win on a single stock, even if one could, but to win at the portfolio level over the 6-year testing period (1,500 trading days over 43 stocks). A sufficient statistical sample to show the merits of any trading strategy.
I wanted it to be a live experiment, chronicle the development cycle, and discuss how each participant found ways to improve his/her performance. Everyone would start with the same free original trading script, add their own favorite trading routines and procedures, and modify the original code to do whatever they wanted without ever having to show their code, thereby keeping their own programs secret.
The scripts could be run remotely on the Wealth-Lab 4 simulator (free to use) using Wealth-Lab's price data. So the testing ground was set: same original script, same simulator program, same price data series, same trading interval, and the same number of stocks. The only thing that could be made different would be the modifications to the trading strategy that each participant could bring to the challenge.
The Selected Script
The selected script was the Livermore Market Key; its code is still available for free on the old Wealth-Lab 4 site. (note: the Wealth-Lab 4 website has been shut down. 13/04/2018). It's a complicated script that translates Jesse Livermore's trading methodology as described in his last trading book a few years before his suicide in the 1940s. You will find many books on the Livermore legend, but none of those books dared provide a systematic test of his trading methodology; you would see some examples that worked out, but no rigorous test.
I chronicled the experiment as it evolved in the Wealth-Lab forum under the Alpha Power thread (using the handle: Roland); search for the first date: 6/1/2011, where the challenge begins. (The test was also referenced on my site, look for the Livermore tests under the simulation menu).
Livermore's methods were the result of what he considered his vision of how the market worked in his time, and it took him years, not to say decades, to develop and improve on his methodology. He did not have a computer on his desk to really test his methods; what his book described was his ongoing search for a trading method that fitted what he was seeing in the markets and, at times, served him well. Livermore was a discretionary trader. His methods were not specifically designed for automated trading. They were, at most, guiding lights for a brilliant man.
If the Livermore trading methods were any good, it would show under test; and from there, I expected to find ways to improve further on the methodology on the same basis as anyone else wishing to participate in the challenge.
The Original Livermore Script
Obviously, the first step was to use the Livermore strategy as is (meaning no modifications) to see how it would have performed over the selected data set. With the reputation and the quasi-legendary trading expertise, Livermore's trading methods had to show, from the start, outstanding performance results.
Well, they did not. Far from it, the script barely broke even. I would say making peanuts over the 6-year test. A disappointment considering all the hype associated with Livermore's trading methods and with all those still promoting this “how could you miss” strategy to untold riches... Next time anyone proposes to you the Livermore trading methods as the basis of their trading methodology, make sure that it has been extensively modified or thoroughly tested since the original version won't do you any good. Its only redeeming quality was that it stayed afloat, but barely, which is more than many trading strategies I've tested.
I really wanted the original trading script to live up to its reputation. Improving on a good script is a lot easier. Usually, you are not considered a nice guy if you attack a “legend” while he's down. However, I can only say them as I see them. And the test results did make the point. Anyone wishing to redo the original Livermore script test would have obtained the same answers I did. The test, after all, was on the same set of stocks over the same trading interval using the same price data controlled by the same simulation program.
The Real Challenge Starts
After publishing the performance results using the original Livermore Market Key script it was now time to start coding my own modifications starting at the same time as anyone else could. I expected that within a week or two, I would achieve sufficiently interesting results to report back. It would also provide sufficient time for other Wealth-Lab users to participate in the challenge with their own solutions, generate some discussions, and express their points of view.
The Livermore Market Key original script is complex. It tries to define major and minor trends with primary and secondary retracements. It has a trend-following philosophy and will try to buy on a dip in a major uptrend. Even if it succeeds at times in doing this, it is not enough to compensate for all the losses; the hit rate is a mere 24%. It takes too many early stops on its losing positions and thereby eats up most of its potential profits. Surprisingly, it is also quite terrible at shorting (one of Livermore's preferred tactics).
After having posted the results of the original Livermore script, I started doing my own modifications. Within a few hours of adding various trading procedures that followed my own trading philosophy, I already had achieved outstanding results to the point that I did not think anybody could achieve higher performance, having never seen any published Wealth-Lab developer's programs reach such levels. So I published my own test results in the Wealth-Lab forum and was technically forced to call an end to the challenge.
I did not expect to achieve those results so fast and so easily. Sure, I knew my trading philosophy inside out and how to code procedures I found interesting. However, the script was still quite complex; it should have taken me more time to analyze its structure, find its strengths and weaknesses, and then find ways to improve on the trading methodology. By ending the challenge so soon after being issued, I was losing the opportunity to have others show how they would have tackled the optimization problem presented by such a challenge. I think it would have been interesting to see the different approaches used by others to solve the problem.
After showing my response to the Livermore challenge, I thought some people might still show interest in the trading methods used to achieve those results. But that wasn't the case. I could not understand why. Was it a “not invented here” syndrome? Or was it that no one, in their right mind, could accept such high-performance levels, especially with the metrics presented? I was using the same program they did, the same dataset. The only reason for the improved performance was a better trading strategy. I haven't figured it out yet.
But this did not change what had been accomplished. I, for one, was not surprised by the results achieved. They were in line with my expectations, I was simply passing to the next testing level of my trading procedures as can be shown in the progression of the previous simulations done over the previous few months. Furthermore, I knew I could do even better, having put only a few hours on the Livermore script.
My Livermore response showed, as with other methods I used in prior simulations, that changing trading procedures could make a major difference in trading strategy output. The transformation was considerable, from a non-productive strategy, adding an accumulative stance with improved trading methods had pushed performance to such levels that most considered them impossible, unbelievable, and therefore, total hogwash.
But it was still the simple execution of a trading program, a set of instructions to be carried out by the simulation program. Nothing mystical, just the execution of a program. True, some of the trading procedures were quite clever, but it was the trading philosophy itself that was the major source of the over-performance.
In the challenge response, there was not one stock out of the 43 that did not show improvements. It was all 43 of them which showed that the improvements were sufficiently general in nature to apply to all. A simple comparison with the original script results made this quite evident. How could a trading method show performance improvements over all the selected stocks over the 6-year testing period?
That I provided a mathematical background that explained what was happening was not sufficient. That I provided all the charts generated by the system, time-stamped with the Wealth-Lab logo, to corroborate all the entries in the performance table, did not seem to matter either. That some 19,000 trades were executed over the 6-year test period over the 43 stocks did not seem to count. The test had a 78% hit rate, but that, too, did not matter.
I supplied all the relevant data to make the test meaningful and statistically significant. Still, nothing doing. As if I was presenting manipulated results even if the program was run on the Wealth-Lab machine using their testing program and their price data. What was wrong with the approach, or more precisely, what was wrong with my presentation? How come no one seemed to understand what I thought was so simple? Would not anyone's trading methods be improved by incorporating some of the principles I applied to my own trading strategies?
The general concept must have seemed too weird - trading over a stock accumulation process - that it must have gone against the grain. Or was it that neither the fundamentalists nor the technical traders could accept a process that was not entirely one-sided? But still, the alpha power method produced more than one or the other method standing alone. My method played short-, mid-, and long-term while at the same time playing a Buy & Hold program in the background. After all, my program was only saying: trade over a stock accumulation process.
The Livermore challenge dates June 2011, it had made its point. It might not have been what some were looking for, but still, the trading methods used had value and showed that anyone could take some trading strategy and push it to higher performance levels, and for my part, I would say with ease (a couple of hours of coding and debugging had been sufficient).
After having canceled the challenge, I opted to continue to make modifications to the script. I had a few more unused hours that I could spend on improving the trading and accumulative procedures within the constraints of having available cash in the account to accept new trades. Had the challenge lasted longer, it would have been what I would have done anyway.
… to be continued... soon
Added: January 19, 2013
For the continuation of this article, see: Part II
Created... January 10, 2013, © Guy R. Fleury. All rights reserved.