March 19, 2013

Designing a Trading Machine.

I participate in a LinkedIn forum on automated trading strategies, here are some my observations over the past few days, starting with March 11th.
This forum is about automated trading strategies, and yet a lot of talk is on discretionary trading methods which by definition are not automated. In fact, if your trading method is not programmable, it is discretionary; and thereby can not be systematically back tested; otherwise going full circle it would be amenable to code.
Mind you, I have nothing against discretionary trading methods: experience, know how and skills have their place. Furthermore, discretionary trading can also win at this game. My own emphasis, however, is on designing automated trading strategies.
In order to develop trading machines, meaning automated strategies, a lot of skills and know how is also required; not just on the computing side – programming skills –  but in many other disciplines as well: gaming theory, signal analysis, stochastic processes, predictive algorithms, economics, …, accounting, financial analysis, etc..., not to mention technical analysis, which is prevalent in these forums, and fundamental analysis. One thing is sure, there is a lot of math involved.
Long term, can some automated trading strategies not just survive market vagaries but greatly outperform even successful discretionary methods?
LinkedIn March 11th 2013
Backtesting a stock trading strategy has for first objective: Σ(H.*ΔP)  > 0. A trading strategy's payoff matrix designed to make a profit, what ever the trading strategy H, and what ever the n selected securities ΔP over the specified trading horizon (T). 
The payoff matrix and objective of a Buy & Hold portfolio of n stocks would be expressed as:  Σ(H(0).*ΔP)  > 0, where H(0) is simply the initial quantity bought in each of the n securities at time t = 0, and held to time (T).  A trading strategy (A), to be profitable, again what ever it is, could be expressed in the same manner:  Σ(H(A).*ΔP)  > 0.
If you design another trading strategy (B) over the same time interval and using the same data set, then this strategy would be expressed as:  Σ(H(B).*ΔP)  > 0. To have strategy (B) outperform alternate strategy (A) over the same time interval, over the same stock selection, requires only that H(B) > H(A). This results in:  Σ(H(B).*ΔP)  > Σ(H(A).*ΔP)  > 0.
Should you design your very own trading strategy (C), based on what ever trading method you prefer, and by this I really mean what ever the trading method you would want to implement, this strategy (C) has to live up to expectations: its first one being: Σ(H(C).*ΔP)  > Σ(H(0).*ΔP)  > 0, meaning that it must at least outperform the Buy & Hold.
If such is the case, then your strategy H(C), as a monolithic block with all its inventory movements in all the n stocks in your portfolio over the entire trading interval (T) becomes your main concern. Since your inventory will change only due to your trading decisions, profits or losses will consequently be entirely your responsibility. Your know how, your trading skills, the way you change your stock inventory levels over time will be the major reason for the alpha generated. 
How many trading strategies H(D) could have the same or better performance than yours? That is easy to answer: Σ(H(D).*ΔP)  > Σ(H(C).*ΔP)  > Σ(H(0).*ΔP)  > 0, millions if not billions. The holly grail would be the ultimate trading strategy H(H*), the one in one googol power one googol. So stop looking for that one, and stop mentioning it; it is not even attainable in a googol life times. However, more mundane results like H(D?), or H(E?) are certainly attainable by anyone working on the main problem: the distribution of the trading decision process over the entire stock inventory over the total length of the trading interval and in accordance with your long term objectives.
Still LinkedIn March 11th 2013
The next step would be to study the implications of designing one's trading strategy H(C) in such a way that:  Σ(H(C).*ΔP)  > Σ(H(0).*ΔP)  > 0, meaning that it must at least outperform the Buy & Hold. The trading strategy H(C) can be any size, it represents all the inventory changes over the long term trading interval.  Naturally having H(C) = 0, would say no market exposure and therefore no profits being generated, and would result in: Σ(H(C).*ΔP)  = 0. Not participating in the market brings no rewards, but it also does not generate any losses.
The payoff matrix has for output the total profit generated by all the inventory movements over the selected stocks over the entire time horizon. A 2x1 payoff matrix is sufficient to represent the case where only one trade is executed (Q.*ΔP). It's profit or loss will depend on the value of: Q.*ΔP = Q[P(out) – P(in)]. Quite simple when you look at it this way. The payoff matrix is a simplification and can summarize thousands of trades over an extended period of time in a single expression: Σ(H.*ΔP). Take the 30 DOW stocks over the past 20 years: H.*ΔP(DOW), this matrix would have for size: 5,000 rows by 30 stocks, some 150,000 data points showing the evolution of profits and losses due to prices changes and changes in the inventory holding levels for the 30 DOW stocks over the last 20 years.
Now that you can view a whole portfolio over a long term horizon, we can start looking at the problem for what it is: a multidimensional multiperiod optimization problem. Not just a period to period Markowitz type of problem, but a 2,000 or 5,000 periods at a time problem. For sure I won't be able to change ΔP(DOW), what ever I may attempt in my trading strategy. It would be the same if trading the S&P500 or S&P100 stocks. All I would have are bigger matrices. Over a 20 year trading interval ΔP(S&P500) would have some 2,500,000 price variation data points and the ΔP(S&P100) matrix would have 500,000 price variation entries. Not what could be called small matrices to deal with. But, nonetheless, showing the real nature and extent of the problem.
The payoff matrix might be a simple expression: Σ(H.*ΔP), however it is starting to be a huge portfolio analysis tool that can span the time horizon of your choice. What ever the size, meaning your stock selection over your investment period of interest, the output provides what counts most: the total profit generated by your very own trading strategy: Σ(H(C).*ΔP). And that is where your efforts should be concentrated: designing your trading strategy H(C) to span your entire stock selection ΔP in such a way that: Σ(H(C).*ΔP) > 0 as a bare minimum. Note that there is no maximum provided; this will all depend on your own payoff matrix.
LinkedIn March 14th 2013
Once you have accepted the payoff matrix: Σ(H.*ΔP) as a portfolio analysis tool, an historical map of all your trades with for outcome the total generated profit or loss, it might be time to look at its limitations: what it says and what it doesn't. 
First, your trading strategy matrix H(C) what ever it is, is amenable to this matrix format. That you make 1 or 100,000 trades over the next year or the next 10, that you trade in 1 or a 1,000 stocks, that you play at the millisecond, day or short-mid-long term level, the payoff matrix can handle it. Your holding matrix H(C) is simply the chronicled stock inventory changes (the quantity on hand) over the entire history of your portfolio. The inventory level will fluctuate or vary according to your trading methodology. An example of the components in a payoff matrix can be seen in the following figure:
Excel Matrices S1a
What is seen in the holding matrix H is the ongoing inventory level as shares are bought and sold. Shorts are represented as negative inventory and covering is treated as a buy. These inventory levels changes are the results of your buying and selling operations, and can be expressed as:  H(C) = B(C) .- S(C); where B(C) is the add to inventory matrix, and S(C) the subtract from inventory. All matrices being the same size. Changes to the inventory will depend on your surrogate decision matrix: D(C), which means: changes to inventory levels will be the result of what ever the decision process that triggered a trade.
Your trading strategy matrix H(C) is starting to take form and meaning; and it can be viewed in its entirety as the outcome of all the trading decisions made over the entire history of your portfolio. Any thing that can affect your trading strategy (decision process) will have repercussions over your payoff matrix: Σ(H(C).*ΔP). You can design now for things that might happen in 10 or 15 years down the line.
The main idea here is to design a decision matrix that can affect the entirety of the holding matrix H(C), even if it is 5,000 trading days (20 years) by 10, 100 or 1,000 stocks. A real multidimensional, multiperiod optimization problem where the future is still to unfold.
LinkedIn March 15th 2013
You're looking for your stock portfolio to outperform and realize early that the game is not a one trade game: take a profit and run kind of thing. You need more. The market you want to trade is so complex that you could view it from many different angles and still justify your unique vision of the game. 
You want it to be a short term game, no problem it will accommodate. You prefer a swing trade type of game, again the market can answer your wishes. You're a long term investor looking at the big picture, there is a place for you too, and on your terms. Technically, it is up to you to design the game you want to play based on your belief system and temperament. I think the only caveat would be to be consistent within the type of game you want to play. Please understand, not consistent in trading or win/loss ratios, but being consistent methodology wise.
Can you win based on your playbook preferences? I say yes, as long as you delimit your trading environment, set up your trading rules to be consistent within this environment, and make provisions for the unexpected. A long term investor worrying about a nickle wiggle spread should be considered schizophrenic to say the least. 
The market does not care what you think. It's there to spread the risk of business ventures. It's mission is not to respond to your program, it's mission is just to be there, provide a market place, give you the opportunity to exchange cash for shares if you want to take the risk somebody else does not want to take anymore, or shares for cash when you are no longer willing to hold that risk. The time and the price of this exchange is in your hands. I describe this exchange function using a payoff matrix: Σ(H.*ΔP). 
LinkedIn March 16th 2013
What's your payoff matrix? What are the characteristics of your present trading method? What are the numbers? You must have traded long enough to know these numbers by now: average win per trade, average number of trades per period, etc...
The average long term market return over the past century has been around 9 to 10% compounded. To achieve this long term level of performance, all that might be required is a lot of time and a dart to initiate things. But when you look at it closely, and see what is implied, you could almost achieve this performance level blindfolded, or even go as far as letting your machine do the job. 
To achieve 10% per year on your investment, requires an average net move of $0.02 per day on a $50 stock equivalent. Imagine the concept here: two cents to be made in a single day trading a $50 stock. Even if you take a week to make your $0.10 net, you are still on track to reach your annual goal. How many 10 cents move in a week, or even a day can you make? All your machine has to do is: get in, get out, collect $0.10, thank you, next. And then the question becomes: how many next?
Oh, you want 30% plus per year, no problem! You're looking for a net move of $0.06 per day or $0.30 per week on your $50 stock. The variance per day on a $50 stock is about $1.50, on a weekly basis I would guesstimate in the vicinity of $2.50 (variance is a random variable here). Over 800 stocks will move by more than $1.00 in a single day. How many $0.50 moves are there in the market in a single day? There is no lack of opportunities out there.
I know, it is boring to accumulate nickels and dimes, then go for quarters! It is up to each one of us to design better mouse traps, better trading strategies; that they be boring or not is irrelevant. Your job is to figure out ways for your machine to accomplish these mundane tasks, and then watch your machine do its intended job properly. In the end, it's your trading strategy H(C), and your mission is to fill up your own payoff matrix: Σ(H(C).*ΔP?). 
LinkedIn March 16th 2013
Here is a little study I did some years ago in an attempt to explain Berkshire Hathaway's over-performance using my modeling equations. Mr. Buffett is very good at what he does, and I wanted to see if his investment methodology was an easy fit, or more appropriately that my “stuff” fit his methods. Be assured, Mr. Buffett has figured this out years ago. So consider this as some he does, and some he does not.
Mr. Buffett is a long term investor, and as such he is often said to have a Buy & Hold investment strategy. And since he is more savvy than most, I set his stock picking abilities to produce 50% better than an average index 10% secular long term market trend. The expression for this would be: W(t) = W(0)(1+1.5r)t; where an initial wealth W(0), would grow at a compounding rate of return r including dividend reinvestment. I found this figure acceptable and reasonable. Some would argue that I should set this number higher, since everyone knows that Mr. Buffett's rate of return is about 22% CAGR and not 15% as estimated here; but then, doing so I would be hard pressed to explain some of his other investment methods.
Mr. Buffett reinvests, not only dividends received, but also part of the profit generated by his holdings. This “administrative” procedure of reinvesting dividends and profits is still not enough to totally explain Mr. Buffett's performance over the years. If we say administrative reinvestment policy then where would it fit in the equation, certainly not on the price function side; it is not because you reinvest part of your profits that it will modify the price series itself.
Berkshire Study 20 years 
The only other alternative is to say that those profits “buy” something, in occurrence some additional shares of some stock or other; this would not change the price series, but it would change inventory levels. This leads to the reinvestment factor (g) in the above figure. Mr. Buffett likes to maintain cash reserves to be prepared for his next “elephant” target which is why only part of the incoming profits are reinvested in order to let reserves accumulate. So his investment equation takes more the form:  W(t) = W(0)(1+ 1.5(r + d) + g)t; where (g) and (d) are part of the reinvestment policy rate. Using numbers, I would guess something in the order of:  W(t) = W(0)(1+ 1.5(0.065 + 0.025) + 0.06)46. But this is still not enough, Mr Buffett did better than that. 
He does a little trading on the side, meaning that he does get into a position to later liquidate that same position with a profit or loss. His trades are not always winning trades. But again, this does not affect stock prices, it can at most marginally increase or decrease his cash reserves which  are part of his re-investable funds.  The trading contribution rate might be on the high side in the case of Berkshire, however, I have no way of verifying this without having access to the company's historical books. So I'll leave this as a reasonable guesstimate.
Using the trading contribution rate one would have a better explanation to Mr. Buffett's overall performance level. W(t) = 10M(1 + 1.5(0.065 +0.025) + 0.06 + 0.052)46, generating about 257B, kind of a reasonable estimate. 
My own conclusions to what Mr. Buffett does, and others operating on similar basis, is that their reinvestment policies are dominating factors in their respective success stories. And there is a lesson to be learned in these portfolio management policies, one of which being that these policies are programmable.
LinkedIn March 18th 2013
As a follow-up to my prior post. If Mr. Buffett's investment strategies and reinvestment policies have merit, then it should show up in a simple Berkshire / S&P500 comparison, here it is:
Berkshire Feb 3 2013
The above chart does make its case. The regression line spread between the two curves is increasing linearly over time and that can only be expressed as alpha generation, furthermore, it is being maintained over the entire investment period. 
Mr. Buffett has more munitions in his trading arsenal. One over-looked is probably his bet sizing methodology. As he acquired more companies, or taken bigger stakes in others, these investments would generate more and more operating profits on their own, plus more dividends, more paper profits, all of which could be funneled back to his increasing war chest creating and enabling a feedback loop. More profits implied greater reserves, implied bigger and bigger bets, implied more and more profits to be reinvested to make even bigger bets. Simple logic. He is earning profits on his accumulating profits instead of letting them go to waste, like leaving them idle or in the bank earning low returns.
Not making bigger bets as his portfolio evolved would have been a huge mistake. You buy a 40B company, buy 40 companies of 1B or accumulate cash reserves operating at low interest rates. Which would provide the less work, for yourself and for your staff, and which provides the best return to your shareholders? 
The most reasonable solution would be to make bigger bets in order to maintain everything manageable. And it is this bet sizing that will push performance levels even higher:
Berkshire 20 years
The above chart shows another component to the equation presented in the first link, and that is betting size. Mr. Buffett, as his portfolio grew, increased his bet size and is now “hunting” for “elephant size bets”. It has not always been the case. It was a gradual process that spanned  some 50 years after all. 
I estimate that the way Mr. Buffett proceeded should be a lesson for others wishing to attain his performance level. Mr. Buffett manages a 250B empire with a staff of 25, it is a remarkable achievement. I would be proud imitating some of his trading/investment methodology.
So, there you have it; a basic Buy & Hold scenario to which is added a profit and dividend re-investment policy, over which trading is allowed to take advantage of market situations to add  more profits and where bet size is gradually increased according to achieved portfolio levels. 
Should you want to outperform, then those investment principles should also be part of your trading philosophy. And should you want to do even better, then increase your own short term trading component, that is where real over-performance resides. You trade to achieve a higher long term objective and that is to buy more and more of the best long term performing companies. I usually resume this in an enhanced payoff matrix: Σ(H+.*ΔP).
The way to outperform Mr. Buffett is to do more than he does. One area would be to use our machines to do more trading. Not just any trading, but controlled short-mid-long term trading over the entire investment period. The less time it takes to produce a  ΔP > 0, or preferably ΣΔP > T  (greater than a threshold), the better. The point becomes: executing as many trades having a ΣΔP > T as possible. A trading script can handle hundreds of stocks at a time. The biggest constraint I see is the lack of available capital because there is no lack of opportunities.
It all boils down to common sense: managing a portfolio for the long term under uncertainty with limited resources. I can not make 5B IBM bets. I'm really to small for that, but it should not stop me from accumulating IBM shares while trading over this accumulation process and thereby generate a much higher return. 
LinkedIn March 19th 2013
Now that a mathematical background and philosophy has been set. It's time to design a trading machine. The objective being to achieve as much profits as possible under portfolio constraints (like available capital and uncertainty) over the game's entire trading horizon resulting in one's enhanced payoff matrix : Σ(H+.*ΔP). 
No two players will have the same payoff matrix; there are too many variables, too many unknowns and too many unknowables. The job here is not to produce the ultimate payoff matrix, but to greatly outperform market averages over the long term, and by long term, I do mean 20 years or more.
The payoff matrix, even though it does provide the total history of all trade executed over the entire time horizon, it does not show time, only realized and unrealized profits or losses. It chronicles all the portfolio swings up and down on a dollar basis. You add up a row and you obtain the portfolio profit or loss for that day or period of interest. You add a column and you get the net cumulative result of all the trades in a single stock.
You see me always taking about long term scenarios. And yet propose short-mid-long term trading strategies. Adding short term trading strategies can really push annualized performance higher. The following chart shows what happens when your annualized return is achieved in a short time interval than a year.
Shorter Interval
The annualized rate of return will increase as you shorten the trading interval; while your potential annualized percentage loss will narrow. It will not change the amount you won or lost but can help put things in perspective.
Shorter Interval 5 pct
In the above chart, observe the knee jerk reaction over the period from period week 3 to week 1. From this one should conclude that the chase for shorter trade intervals for positive trades could be more than worthwhile; while the converse might be detrimental to your portfolio.
LinkedIn March 19th 2013
Since short term trading can add to a long term view of a portfolio, the next step should be to find a kind of short to mid term edge. The shorter the trading interval, the smaller the sum of the expected price variations (ΣΔP) between opening and closing prices should tend to be. 
And, if such is the case, note that depending on portfolio size, edge expectancy and number of these potential edges, all of them will become major considerations in time. Nonetheless, shorter trading intervals can be a great source of annualized profits as the previous charts showed.
Then back to square one: defining what is an edge. That definition should be relatively easy, it has already been stated: ΣΔP > 0 > T (positive and greater than a threshold); you do play for money after all. This is simplicity at its best, and as most have found out in their own trading; it is not because you have a definition for an edge that you have won that edge. You also need to extract this edge consistently. And the first question is: what can be said of your edge?
The equation for a price series was stated as: P(t) = P(0) +  ΣΔP, then ΣΔP = P(t) – P(0). You could draw a line between any two prices and your minimum requirement would be to have a positive slope. But then, you could draw multiple lines all over any chart using all types of indicators or functions, support/resistance lines, pivots, Murreymath, sushi or Fibonacci lines, rainbow moving averages, stochastic, relative strength lines or maybe some 200 or more line types. The number of lines might not really matter. I have scripts where none are used and still the scripts made profits.
You've tried other people's trading methods like Turtles scripts, Livermore trading methods, random entries, Ichimoku or what ever, but none fit your personality. Maybe most of all: none in your view showed significant long term profits. My belief is that you are the only one to design what best fits you. Somebody else can help you design what you have in mind, even tell you where not to go and avoid certain none productive directions; but if you do not understand and believe in the underlying trading strategy you want to implement then how can you, or your computer, press the buy or sell button when it's time?
The point I would like to make is that you can draw a lot of lines on a chart, have your program respond to any of them and still it would not give you an edge for the simple reason, that what ever the lines you trace on your chart, a price series is bound to cross some of them more often than you wish. And maybe most of all, no price series is bound to follow or obey your lines.
All you want is: nΣΔP > 0 > T , not just on a single trade as stated above, but on thousands upon thousands of trades. What can you extract from all the data? You have over 200 years of stock market prices, thousands of stocks. Remember that lots of them went bankrupt, and on the optimistic side, some have gone to stratospheric heights. It is up to you to design a trading strategy that can survive what ever the market wants to throw on your plate. 
So how will you handle the quasi-randomness of price movements? That is your ultimate question. I have my own answers, but I certainly would like to hear on what yours are based.
LinkedIn March 20th 2013
Accepting quasi-randomness for short term price movements, then how is one to extract short term profits, how is one going to extract a favorable ΣΔP > 0 > T (a profit greater than a specified threshold)?
The first caveat would be: it is not because you have identified past favorable ΣΔP that the one you want to trade next has a positive outcome. And since you are chasing a price differential; you have, or your trading script has, to determine which direction you want to play it: up or down. 
Here is an interesting study I did some 10 years ago. The surface map below shows how some 450 trades behaved between two cutoff points: an ultimate stop loss was set at $-2.00 while the profit target was limited to $+5.00 which explains the limits to this graph. It's a study of maximum favorable and adverse excursions (MFE, MAE), it was for personal use and had no need to be cute. I just wanted to know the distribution of MFEs and MAEs for the trading strategy used. 
The focus is on the general shape of the graph. Starting from the lower corner at 0.00 you have where all trades originated, naturally without a profit, and the backdrop where they were cut off. The trading strategy had a positive bias which explains most of hump in the graph.
Surface Map
Some observations
The maximum obtainable percentage return value rides on the ridge of the hill. This ridge moves to the back as you go uphill to the maximum point. This immediately implies that you have to accept a certain amount of draw-down if you want to maximize your return. The right amount of draw-down depends entirely on your own methodology and risk tolerance. The last few slices of draw-down do not seem to improve overall performance. So there is no need to risk more for no added return.
In this particular case, draw-down worst than $-1.00 accounted for less than 5% of all trades. If the draw-down exceeded $-1.00 there was an 80% chance that the final result would be at a loss when the trade would be closed. It is hard to dig yourself out of the hole. The more draw-down you get, the more likely the trade will result in a loss. The valley in the forefront of the chart shows the direction of your profits if you do not put in a definitive limit to your stops. This makes a good case for fixing an absolute stop loss and in this case it did not need to be 3 points away.
However, adding more slices of run-up tended to improve performance. Just like for the draw-down, if the run-up exceeded $+1.00, there was an 80% chance the trade would end up with a profit. This can be translated to: should your trade be profitable, try to push for a little more; the probability is that you might get it more often than not.
Your trading method should provide about the same results (chart wise) and your profit distribution map should be about the same shape as this one. But you should probably find the same thing I did, almost all trades suffer, to some extent, some draw-downs.
The whole point of such a map is to study the impact of draw-down on a trading method and help find the best stop loss to achieve the best possible results and help move you profit target higher: two ways to improve your profits.
The chart may be a restatement of the obvious. The more of a run-up you get the better the chances of retaining part of it. Limiting the losses becomes a critical factor unless one wants to ride down the valley of draw-down into the abyss of depleted accounts. But I thing I am repeating myself.

Created... March 19th, 2013 © Guy R. Fleury. All rights reserved. 

Alpha Project