[TUTORIAL] Complete Backtesting and Analysis Setup (100% Free)

[Tansen]

Original By ClarkFX
Original By ClarkFX
Original By ClarkFX

All workings by ClarkFX, not me, permission granted by him to repost here

"In this thread, I will be teaching readers one of my methods for performing accurate backtests of automated trading systems as well as post-test analysis. This method is great for someone who does not have much start up capital as it is completely free, yet still very powerful and comprehensive.

My aim is to help readers set up a complete system development infrastructure. By the end of the tutorial, one will be able to very easily develop and test trading systems with a high efficiency.

The setup process is pretty straightforward:
1) Download tick data
2) Format the data so it can be used in our testing
3) Streamline steps 1 & 2
4) Further formatting so we have the data required to test various styles of algorithms
5) The actual backtesting
6) Analysis

It primarily focuses around MetaTrader 4 but also incorporates other software designed around MT4 that are easily accessible for any individual (as well as a small piece of Python code I wrote a couple days ago).

This thread will cover:
Setup and Tools Required
Getting Tick Data
Formatting Tick Data for MT4
Importing Data
Converting to Various Time Frames
Example Backtest
Backtest Result Analysis Technique
Portfolio Backtest and Analysis


In the following posts I will start discussing each in detail. I recommend readers to read this thread in order as it is step-by-step.

Note: I am not endorsed by any of the companies or products I am discussing. In no way do I intend to sell anything. I'm in a giving mood today.

If you'd like to collect your own tick data from any broker via MetaTrader 4, visit this thread: Collect your own tick data via EA"

 

[Tansen]

Setup and Tool Requirement

Before we do anything, let's start with gathering all the tools and getting the preliminary setup covered.

The resources we will need is:
Python 3
Dedicated MT4 Client
StrategyQuant's Tick Data Downloader
StrategyQuant's EA Analyzer
D2MT4.py Script
Period_Converter_Opt MT4 Indicator


I will talk about what each one does when we get there. For now, let's start getting these and set up.

 

[Tansen]

Setup and Tools Required (Continued)

Python 3
Python 3 is a programming language, it must be installed on your computer in order for the D2MT4.py script to work.

Download link:
Download Python

Note: For 32-Bit users download "Python 3.3.0 Windows x86 MSI Installer". For 64-Bit users download "Python 3.3.0 Windows X86-64 MSI Installer".

The installation process is quite straightforward; just follow the installation wizard, you should have a Python directory after in the directory: C:\Python33.


MT4 Client
For our setup we're going to be downloading a separate MT4 client in order to keep ourselves organize and avoid confusing our trading with our testing. Another reason is we want to be disconnected from any servers when we are performing our tests (more on this later). In this thread, I will be using MetaQuotes' MT4 with no broker licenses.

Download link:
http://files.metaquotes.net/metaquot...4/mt4setup.exe

After installation is complete, I like to prepare the client in order to avoid future issues:

1) Open up your MetaTrader 4 folder.
2) Find "terminal.exe"
3) Right click on it and select "Properties".
4) Under the "Compatibility" tab, make sure "Run this program as an administrator" is checked. Then press "OK".

StrategyQuant's Tick Data Downloader
StrategyQuant's free software which allows users to download tick data from Dukascopy automatically. We'll be using this to download our raw tick data. For more information: StrategyQuant Tick Data Downloader

Download link:
http://www.strategyquant.com/licenses/d?code=tdd

Note: DO NOT install this in your Program Files or Program Files(x86). The software has issues saving data in there. I would recommend saving it in My Documents or Downloads directory.


StrategyQuant's EA Analyzer
Another free software from StrategyQuant which allows further analysis tools for MT4 Strategy Reports. For more information: StrategyQuant EA Analyzer

Download link:
http://www.strategyquant.com/licenses/d?code=eaa

Note: DO NOT install this in your Program Files or Program Files(x86). The software has issues saving data in there. I would recommend saving it in My Documents or Downloads directory.


D2MT4.py & Period_Converter_Opt
These two tools will assist us in formatting data. D2MT4.py was a piece of code I created for this tutorial and Period_Converter_Opt was an indicator released by "wfy05" on the MQL4 Codebase.

I have attached these two in this thread.

1) Place "D2MT4.py" in the "tickdata" folder of your StrategyQuant's Tick Data Downloader path. Ex) "C:\Users\Admin\Downloads\SQ_TickDataDownloader\tic kdata"

2) Place the "Period_Converter_Opt.mq4" and "Period_Converter_Opt.ex4" files in the indicators folder of your MetaTrader 4 you just downloaded. Ex) "C:\Program Files (x86)\MetaTrader 4\experts\indicators"

Tick Data Tools.zip

 

[Tansen]

Getting Tick Data

First off, we need data to perform our testing on. There are many free sources online, but most have large gaps and missing quotes. Some are only End-Of-Day which is useless for intraday strategies.

Fortunately, Dukascopy has kept a large archive of tick and bar data available for free online. Unfortunately, their bar/OHLC data is garbage. Some bars are off by over 300+ pips on some pairs. BUT their tick data is surprisingly very accurate and detailed. Why the huge difference between the two; I have no idea.

What we want is tick data. Dukascopy's online interface is ridiculously slow. At the same time, one must download tick data one day at a time, that would be amazing slow.

Luckily for us, StrategyQuant offers the Tick Data Downloader software which allows us to download tick data automatically. All one needs to do is to check the pairs they want and the starting and ending dates. "Start Download" will begin the export process. Tick Data Downloader will automatically start downloading the time series one symbol at a time. Depending on how much data you are downloading it can range from 20 seconds to 10+ days straight. Just make sure to keep your computer on and uninterrupted when the downloading is happening. The tick data is stored in the "tickdata" folder inside the installation directory. It is also where we saved our D2MT4.py script.

Again, if you want more information: StrategyQuant Tick Data Downloader

"tickdata" folder

If you open the tickdata folder as you start downloading data, you can see that each symbol creates a new folder, within the folder, additional folders are created for each year, then each month (0 to 11), then the day. Finally you will see "xxh_ticks.bi5" files. XX ranges from 00 to 23; you guessed it, those represent each hour of the day. Tick Data Downloader first exports these ".bi5" files first. Then it "Regenerates" these files and creates a ".csv" file. Ex) "EURUSD.csv"

The format of the CSVs are:
YYYY.MM.DD HH:MM:SS.mmm, bid, ask, bidcount, askcount

Keep this in mind for the next post, where we start formatting these files so they are ready to be imported.

 

[Tansen]

Formatting Tick Data For MT4

Let's first talk about how to import data into MT4.

Many people and resources have talked about .fxt and .hst files and converting tick data into these to be imported into MT4. In my opinion, that's just too much work and takes way too long.

Luckily there is an easier method - History Center. Let's open up our MT4 client that we just downloaded. When started up it prompts you to create an account as well as login. Just cancel out of those windows. We are working offline for our purposes for a reason. When you are connected to a data feed, the History Center is filled up with the data from the server. So when you import your own data, there will be times where your data will actually get replaced. This is something we must avoid. Which is why we are working offline, on a dedicated MT4 client.

MT4 History Center

By pressing F2 or going to Tools -> History Center, we get the History Center. You can see the symbols offered. By double clicking on the symbols you can see the different timeframes. You'll notice that MT4 does not support tick data, but only OHLC bars. This is one of the disadvantages of MT4 strategy testing (no tick data support, no variable spread, no commission calculations, or slippage).

What we will do is actually format our Dukascopy tick data into MT4 accepted 1 minute OHLC data. Yes, it will lose some precision, but the way I see it is if you are using MT4 to develop strategies, you shouldn't be looking at HFT strategies to begin with. 1 Minute bars will be more than accurate for 95% of strategies and testing needs.

Okay, let's start!
Once we have our data fully exported via Tick Data Downloader from Dukascopy, you will see the CSV files named after the symbol inside your "tickdata" folder in the installation directory of Tick Data Downloader. What we need to do now is read this tick data and create a 1M bar file.

As an example I have downloaded data for EURUSD. The file "regenerated" by TDD is named "EURUSD.csv". This is where the D2MT4.py comes into play. Let's run it. Command prompt will open up it will ask for the filename and we will simply type "EURUSD" (without the quotations) and then press Enter. You will see that a new CSV is created "EURUSD_1m.csv". Wait until the Command Prompt closes to know that the new file is completed.

 

[Tansen]

Importing Data

Now let's import this file into MT4. Open up MT4, again cancel the Account Creation and Login prompt upon starting the client. Press F2 or Tools -> History Center. Double click EURUSD and then make sure you have selected "1 Minute (M1)" and click "Import" and the lower right of your screen.

The settings are just the default since the Python script has ensured that it is exactly the right format that we need it to be in. Press Okay, once the settings are set correctly and you see a table of your 1 minute bars.

Congrats! You just imported your first data file.

Next post I will cover how to turn this into the other time frames so you are able to test your strategies on any time frame.

 

[Tansen]

Converting To Various Time Frames

1) Set "Max bars in history" and "Max bars in chart".

Tools -> Options -> Charts Tab
Let's set both to 10 000 000 just to be on the safe side.

2) Open EURUSD chart, make sure it's 1 minute time frame and then apply the Period_Converter_Opt. Leave all settings alone, except PeriodMultiplier. If you want a 1H chart, set PeriodMultiplier to 60, if you want 15 minutes set it to 15, Daily is set to 1440, etc. Double check OutputCSVFile is set to 2.

3) Open up your MT4 installation folder. Go to experts -> files and you'll find your CSV files you just generated. Now all you have to do is import this data the same way you did with the 1m chart.

This completes the data importation process. I know it's long and tedious, but it's all worth it, and this is probably one of the faster ways with MT4.

Next few posts will show you how to backtest, and how we take advantage of the Strategy Report generated by MT4's Strategy Tester.

 

[Tansen]

Example Backtest

Let's do an example backtest with the MACD Sample EA provided by MT4 upon installation. It doesn't matter if this EA is profitable, it's just to show how to perform a backtest.

Open up your Strategy Tester window. View -> Strategy Tester

Select the "MACD Sample" for Expert Advisor. "EURUSD, Euro vs US Dollar" for Symbol. "Every tick" for Model (most accurate). I am using the M15 time frame as an example for Period. Check "Use date" and choose your From and To date. You can choose to check visual mode if you want. And then make sure to uncheck Optimization. Clicking "Expert properties" will allow you to view and change the parameters of the EA, for this example I have left it as default. Once you're ready, just click Start and it will begin the backtesting process. Depending on your computer and the strategy, the test will vary in time.

After it is complete you are able to view the test's results via the Results, Graph, Report and Journal Tab. If you go to the "Report" tab you can right click on it and "Save as report". Let's do that now; we will use it later in future posts.

Under the Report tab, we can see that mismatched charts errors should 0 or least less very low. Modelling quality does not matter at all in our case whether it's 25% or 99%.

"The reason is because of inconsistent number of ticks. The terminal tries to model the ticks within a bar and the low quality comes from consistency. In our case, Quality != Accuracy." This was taken directly from a Skype chat I had with a colleague of mine. Do not be worried if your Modelling quality is sub-par.

Note: Do not worry about the performance of this EA, this is for demonstration only.

 

[Tansen]

Backtest Result Analysis Technique

The MT4 Strategy Report shows some common and basic analysis of the strategy as well as the trades. But in my opinion, this isn't enough. Let's open up EA Analyzer by StrategyQuant. Remember that Strategy Report you saved as a .html? Well let's import that file in EA Analyzer. Once it is loaded you can see much more information compared to the normal report.

I will let you play around with the program and find out what it offers.

 

[Tansen]

Portfolio Backtest and Analysis

Many traders would like to incorporate multiple trading strategies at once. One of the drawbacks is MT4 cannot do an entire portfolio backtest. This is where EA Analyzer can help us again. The process is simple. We run a backtest of each trading strategy on the time frames and symbols desired and we save the strategy reports for each one. EA Analyzer gives us the ability to merge multiple reports as one and do an overall analysis of all the trades in combination.

In this example, I ran a backtest of MACD Sample on the EURUSD 1 minute timeframe and 15 minute timeframe.

We add all backtest reports into EA Analyzer and at the bottom we check the reports we want combined in our portfolio, and then we click "Create Portfolio" which will create a portfolio and the analysis.

This concludes the main tutorial. I hope everyone learned something new. I will post more information as I can come up with it. As well as clean up the previous posts for any spelling/grammar errors as I tend to make a lot of those. I'll also fill the posts with more detail later on, but this should be enough to get you started.

I look forward to feedback, comments and sharing ideas.

 

[Ramy3]

Re: [TUTORIAL] Complete Backtesting and Analysis Setup (100% Free) Read more: http:

Very nice, thks. Will be following and trying this out. Working on some EAs at the moment, and the strategy tester in MT4 is quite inaccurate. Looking for a way of getting the same results in a live environment that the EAs bactesting results give. So far it is hit and miss. Relatively new at automated systems. Learning curve is a bit steep for me (Time consuming).

Great work!

Cheers!

 

[jaywalker]

Hi

Many thanks for creating this thread.

I am working through the steps as outlined above, and seemed to have hit a snag with downloading of tick data.

I have downloaded tick data for the EURJPY pair, from 1/01/2009 until now. However, when I open the tickdata folder, I see only 2 folders: the new EURJPY folder (with the breakdown of hourly folders etc within it), and the D2MT4 folder. There is no EURJPY.csv folder. When I try to run the D2MT4 app, nothing happens (presumably because there is no .csv file.

Could this be because this is a new version (version 4) of TDD which does not produce a .csv folder/file?

Many thanks
Mike