Intermarket Trading Strategies
Optimization
Trading system optimization is a controversial subject among analysts and traders. But what is optimization? When designing a technical trading system, the analyst must use an indicator or formula to produce trading signals.
All indicators, however, contain one or more variable parameters. A parameter can be set to one of several possible numeric values, which can greatly alter the behavior and performance of the indicator. The dilemma is what value to assign to each parameter. That is where optimization occurs. Optimization is the process of choosing the best parameter values for the indicator.
This is accomplished by having the software "try" a large number of different combinations of parameters or combination of indicators, until a set is found that gives the best result. The result could be net profits, percentage of profitable trades, minimum drawdown and so forth. An example of a trading system parameter is the length of a moving average used. It is very important to identify the ranges into which the values for parameters and criteria fall.
A thorough development of a trading system must involve the testing of thousands of parameter combinations to identify those that persist in producing profits, even though they are not necessarily the optimum. Thus, it is often better to pick up those parameters in the best ten percent of the parameters instead of the best based on stable neighbouring parameters.
Today's software for trading systems can take you down a dangerous path. A big profit, however, on an optimized test does not necessarily mean you have a good system. The danger with optimization is that what worked well in the past will almost certainly not perform as well in the future. Optimized variables can reflect irregularities in the data far more than any underlying logic of the market or the trading strategy.
Usually, such curve-fitting fails as soon as the parameters hit the real world with its random shocks, because the more control you take of the data by specifying indicators and their optimal parameters, the less substance is left in the data to provide predictive capacity. Statisticians call this a loss of degrees of freedom; traders call it curve-fitting.
Some analysts claim that if the data sample is large enough, then the curve-fitting captures most of the price dynamics and the future buy and sell signals will be just as good as the curve-fitted ones. These claims are generally accepted as being true because there is a lack of knowledge, but the truth is that they are based on nothing more than hope.
So how can we design trading models and avoid the illusion of optimization?
The usual way to validate an optimized trading system is to test the system on data it has never been tested on before -- that is, test the system on some data set aside and not used when developing the model. This holdout data set is referred to as out-of-sample data. If the system performance holds up well on the out-of-sample data, we can be confident the system will perform as well in the future as it has in the past.
A more rigorous method of back-testing is that of walk forward testing. The concept is similar to out-of-sample testing but instead of optimizing on seven years of data and using the last three years for testing, a system is optimized on five years and then tested the next year. Once this test is completed, the whole time window is moved forward one year to include the year just tested and the system is then re-optimized. It is then tested again the next year. This process is repeated again and again, "walking forward" through the data series until all the data is exhausted. At the end, all tabulated one-year out-of-sample results are merged to create one large out-of-sample results segment and the system performance is evaluated based on the combined out-of-sample results.
The main advantage of using this method is that since market dynamics change slowly over time the optimum values based on the most recent five-year segment will be accurate in forecasting one year into the future before the model falls apart.
The time periods used in this example for optimization and walk forward testing are not set in stone. The selection of the optimization period depends on the available data and the time required to model the market dynamics. The walk forward period can be between 12-20 % of the optimization period.
Although computationally intensive, this is an excellent way to study and test a trading system. Even though optimization is occurring, in a sense all the trades are tested on what is essentially out-of-sample test data. In addition, the results will closely simulate the process that occurs during real trading, in which traders frequently re-optimize their systems to bring them up to date with fundamental or technical changes in the traded market or its intermarket relationships. Even though nothing can guarantee future results, this approach appears to be as close as we can practically come to estimating trading relationships during constantly changing market conditions.
Unfortunately, totally automated walk forward testing is not available in most popular off-the-shelf trading software and implementing it manually can be time consuming. There is, however, a Tradestation add-in that can be purchased separately. Walk forward testing is also available in Deepinsight Professional and TradersStudio.
A major drawback of out-of-sample testing is that to exhaustively determine the optimal indicator sets as well as their respective periods would require an inordinately large amount of data. If one uses this approach consistently, every piece of data must be thrown away after a single use in the testing phase. In addition to the cost of subscribing to data providers, some futures have only recently begun trading. If the amount of data you possess is limited, you may want to consider the following additional methods of reliability testing.
Cluster spotting and profit mapping: When optimizing a system, traders often save the parameter set which produces the highest level of profit. Once the system with the highest level of profit is identified, you should inspect the results of surrounding parameter values nearby to be sure it is not merely the result of a freak coincidence in the data. Optimal results which are surrounded by similar levels of profit are much more desirable. A better approach is to export the test results produced from each simulation to an Excel spreadsheet and plot a three-dimensional graph, with one parameter scaled along the X-axis, the second parameter scaled along the Y-axis and the profit as height along the Z-axis. Casually inspecting this graph, one can find the region where variations in both parameters have the smallest impact on profitability. Parameter sets in this range may be assumed to be more reliable than parameter sets which produce spikes of profits. This is because low parameter sensitivity in the past implies continued low sensitivity in the future, even though the parameter values producing maximum profits are outside this parameter range.
The process described above can be applied to a wide variety of trading systems as most of them are either already described by two parameters or can be altered to fit a two-parameter model by optimizing each rule or indicator separately.
Monte Carlo simulation: This technique is mainly used to obtain a more realistic figure for the drawdown statistic than the figure obtained by simulation testing and is performed by rearranging the trades randomly (hence the name).
To rearrange the trades, the Monte Carlo simulation uses a random number generator to select a random number between zero and one. This random number is then used to select a trade from the specified listing. The Monte Carlo simulation can reveal whether the account would have experienced a deeper drawdown and what the actual expected drawdown could be in real trading. If the Monte Carlo drawdown is significantly different to that obtained from the simulation model this can also indicate that the system has been overoptimized.
A limitation of this method is that the random number generator assumes that trade returns are independent of each other. It is therefore a good idea to check for autocorrelation of returns before running a Monte Carlo simulation.
Low number of inputs: By increasing the number of inputs to extremely high levels, you could design a system that fits almost perfectly around a price series. This system, however, will almost certainly fail when confronted with real time data that it has not seen before. It is therefore a good idea to keep the number of parameters in the model relatively low, because the larger the number of variables in the system, the easier it is to create an overoptimized system that will underperform in real time.
Diversification with intermarket indicators: Last but not least, the key to robust system development is to select non-correlated inputs that have predictive ability. Using a superfluous number of moving averages and oscillators that are derived from the same price series won't create a profitable trading strategy. In fact, it will most likely fail to work in real time. Rather than using standard, price derived indicators as your only input, try to ascertain what other markets are related to the market being traded and use the intermarket relationship as a filter or replacement of a redundant classic indicator. One interesting advantage of intermarket analysis is that it relies on at least two different data sources or price series. Most single market analysis tools are derived from the same price data. A price shock or bad data would affect all the indicators or tools based on those data. By using two different data sources, you can help to insulate yourself against such an event. The use of intermarket relationships in developing trading systems is explained extensively in the following chapters.
By using some of the above techniques, I am convinced that anyone can, at the very least, avoid trading with systems of absolutely no value. Considering the amount of money at stake in these highly leveraged and volatile markets, it's hard to believe anyone can justify trading a system without first performing some type of validation and reliability testing.
Markets: The choice of securities used in back-testing is also important. For example, if a broad market system is tested only on oil stocks, it may fail to do well in different sectors. As a general rule, if a strategy is targeted towards a specific sector, limit the stocks to that sector, but in all other cases, maintain a large universe for testing purposes.
Software inputs: Back-testing customization is extremely important. Many back-testing applications have input for commission amounts, round (or fractional) lot sizes, tick sizes, margin requirements, interest rates, slippage assumptions, position-sizing rules, same-bar exit rules, (trailing) stop settings and much more. To get the most accurate back-testing results, it is important to tune these settings to mimic your trading style and the broker that will be used when the system goes live.
Paper trading: The market is dynamic and strategies that performed well in the past fail to do well in the present because ofchanging market conditions. Be sure to paper trade a system to ensure that the strategy still applies in practice.
Data quality: Price data often contain errors, which can cause serious problems, Special care should be exercised when handling missing values as these are usually replaced by zeroes. Sometimes free internet data sources, such as Yahoo Finance, do not adjust stock prices for splits. The resulting gap will not only affect the system's profitability but will also bias the system to select the appropriate parameters to position itself on the right side of the market when the price jump (or split) occurred.
While easily spotted manually, data errors tend to corrupt a mechanical system ifa price series is used for testing without careful data preprocessing and error checking. There is an option in Equis' Downloader to test the data for over 100 different error conditions. The most useful is testing for a large change in close.