The Value of Correlation During Backtesting And Forward Testing
There is one common mistake, made by traders, who want to try their ideas in a live market. These traders rely only on backtesting results, trying to find out whether their system would bring profit or not. Indeed, backtesting can help traders, providing them with important data, but there are many cases when it misleads you. Besides, there are other tools for evaluation: forward performance testing and out-of-sample testing. These methods confirm effectiveness of the system and show its capabilities before you use real money. If you want to be sure that your system is viable, than you should determine correlation between results of out-of-sample testing, backtesting, and forward performance testing.
This method shows good results for forex scalping experts and general purpose expert advisors. This method is also suitable for training of expert advisers based on forex neural networks.
Introduction to Backtesting
Backtesting is a process of optimization of trading system on the basis of historical data in order to check the behavior of the system during the specified time period. A lot of modern trading platforms support backtesting. We successfully use metatrader with build-in tester/optimizer for this purposes. Thanks to this method traders can easily test effectiveness of their trading strategies without any risk to lose funds in a trading account. You can use backtesting to test not only simple systems (performance of moving average crossover on historical data) but also more complex strategies, which have many triggers and inputs.
If you can measure an idea then you can use backtesting to test it. Sometimes investors and traders employ experienced programmers in order to make a testable form of their ideas. A programmer uses proprietary language of the trading platform and creates so-called “tweakable” systems, which allow traders to change or input some variables manually. In case of moving average crossover system you can input lengths of the two moving averages. Then you can use backtesting in order to test moving averages lengths on the historical data. It will help you to find the lengths which would have performed the best.
Work on Optimization
Most trading platforms like a metatrader also give you an opportunity to optimize a system. You simply enter a range for the specified input, and then the best input will be found automatically by a computer. You also can use optimization with multiple variables when two or more variables are combined in order to find the levels, which would have led to the best results. For instance, you can enter the inputs, which you would like to use, in the program, and they will be optimized to their ideal weights given the tested historical data.
Often backtesting shows amazing results when an unprofitable system suddenly becomes very profitable and successful forex trading system: you only need to do some optimizations. Of course, the trading system is profitable on the past data, and if you tweaked your trading system in such way, it would perform badly during real-time trade. The system looks good, but because of such excessive optimization it brings profit only on paper.
Curve fitting is when you use analytics of optimization in order to create the maximal number of winning trades, which were the most profitable; here you take the historical data, which were used during tests. Backtesting results may look very impressive, but your system still will be unreliable, because curve fitting results are true only for that particular time period and data.
When you want to evaluate your trading system, you can use optimization and backtesting, which give you a lot of benefits. However, these tools are only a part of the whole process. Next you should test your system on a new historical data, which has never been used for tests before.
In-Sample Data against Out-of-Sample Data
When you perform tests of your system on historical data, it is always useful to “save” some time period of historical data, which will be used for other testing purposes. Generally, there are two types of data: in-sample data and out-of-sample data. In-sample data are the data, which were initially used for tests and optimization; out-of-sample data are the data, which have been saved. Saving data for further tests is very important for the process of evaluation. Therefore, you provide yourself with data that have not been used before in the optimization model. So, out-of-sample data will not influence your system and you will be able to find out real performance of your trading system. Simply said, you will see the possible behavior of your system during real-life trading.
Before starting any optimization or backtesting you should save some amount of the historical data, which will be used for out-of-sample testing. For example, you can split the historical data into three parts and set one part aside for out-of-sample testing. Then you should use only the remaining two parts of the historical data (in-sample data) for the initial testing and optimization. As you can see on the Picture 1, one-third of the historical data on the time line is set aside for out-of-sample testing, and the other two parts are used for the initial testing as the in-sample data.
The order of data placement on the line isn’t quite typical: as a rule, the out-of-sample data stands not in the beginning of testing, but right before the forward performance.
As soon as you’ve tested your system on the in-sample data, you can continue the process and apply the system to the out-of-sample data. It is done in order to give you an opportunity to compare the results of system performance on in-sample and out-of-sample data.
Correlation is the process of finding likenesses between the trends and the performances of in-sample and out-of-sample data. Almost all trading platforms allow you to compare reports about performance of your system, created during tests. If the correlation between the data sets is high, then it is quite possible that your strategy will be successful and will show good results during forward performance tests and real-time trade.
If correlation between two data sets is insignificant, like on the left chart, there is high probability that the system won’t work well during real-time trade because of over-optimization. In case of strong correlation (the right chart) you can proceed and perform additional test referred to as forward performance test.
Introduction to Forward Performance Testing
Forward performance testing is another tool for testing performance of your system. This set of out-of-sample data is often called paper trading, because all operations are executed “virtually”. During forward performance testing a possible situation in a live market is simulated, and the system works according to its parameters and logic as if it traded live. It means that any profit, loss, entries, and exits are recorded, but no real operations are performed. Remember: it is very important to follow the logic of the system as precisely as possible, or you won’t be able to evaluate forward performance testing results. You should honestly specify all trade entries and exits without any exceptions and without any explains. Only in this case you will be able to evaluate the system properly.
Many brokers have an option of opening a virtual trading account, which you can use for practice: open trades, calculate profit and loss, etc. This account allows you to test you system without threat of losing real money.
In case of good correlation between all data sets your system is ready for using it for a real trade.
Conclusion
Backtesting is a very useful and powerful tool, which you can find in many trading platforms. Separation of the historical data into several data sets gives you efficient and useful tools for estimation of your trading system. After optimization of your system you should check its viability, using data that haven’t been used before. Before starting a live market trade you should perform out-of-sample and forward performance testing in order to be sure that your system is profitable. Good correlation between all data sets increases chances of success of your system during actual trading.
BJF Trading group inc.
http://iticsoftware.com
in drop-down menu at setting up an order. It does not mean you cannot choose the desirable lot. You should only edit the value according to the size you want to have, or to set defaults in options (Service -> Options -> Trade).
