Python Development Environment / Jack's Technology Stack


jack

Administrator
Staff member
Could you detail the advantages to using Python over MQL.
(sorry for the late reply, I didn't see your message for some reason.)

With anything, there's trade-offs.

When using Python, you gain access to every library available to python to use with your project. Want a web interface? Done.. want a GUI for custom controls? Done... want to use the most widely supported and accessible machine learning libraries and tools? Done... want to integrate 3rd party data into your trade decisions? Easily done!..

The drawback is, should your broker not already provide a Restful API or Python native API, you'll likely have to write some sort of bridge code yourself (such as the agents I describe to connect to MT5.) It's a bit clunky, and it takes a little effort, but it works.

Other benefits include writing code once that will be easily ported to new platforms. If you do something advanced in MT5 (such as using machine learning,) it's gonna be a real pain to port it outside of MT5 later on should you ever use a different broker who doesn't support MT5. For instance, many decent traders "grow up" to using a broker like Interactive Brokers, who doesn't offer MT4/5 at all (and before anyone tries to link to some 3rd party tool or bridge service.. they are horrendous for use with Interactive Brokers.)

Or say you want to get into futures with existing strategies you have running in MT5... that limits yourself to maybe 1 major broker and FCM out there and their offering, while fine, is limited. Having your code written in a language that's generic and only requiring you to connect the dots to a new broker's API means it's a lot easier to port to one of many decent futures brokers should you ever want to make this switch.

You can build in some fault tolerance... setup a heartbeat monitor on your primary instance and have some sort of risk and order manager kick in should your main machine go down... MT5 doesn't do anything like this.

Finally... another big advantage I noticed was running 'light'... often I can run dedicated python code on a micro server (without a graphical interface) in Amazon's cloud... you might run a total bill of $2-3 bucks a month to power multiple strategies... You simply need a beefier machine if you're gonna run MT5 and a graphical interface, typically costing 10x as much.

Between all the 3rd party libraries available for python, and all the nifty setups you can do, it really comes down to flexibility.
 

sven

Member
@jack I don't see any mention of 'backtesting' algos. Given you are using a hybrid MT/python setup how do you backtest algos?

I get there are limitations to historical testing and many don't hold a great deal of weight in it but its safe to say you need some comfort there is an edge(albeit transient for most)

I'm sure there are different needs for book based algos etc but as a general approach...
Do you backtest?
if so, how?

I do like forward testing but there is a place for backtests. curious.
ta
 

jack

Administrator
Staff member
@jack I don't see any mention of 'backtesting' algos. Given you are using a hybrid MT/python setup how do you backtest algos?

I get there are limitations to historical testing and many don't hold a great deal of weight in it but its safe to say you need some comfort there is an edge(albeit transient for most)

I'm sure there are different needs for book based algos etc but as a general approach...
Do you backtest?
if so, how?

I do like forward testing but there is a place for backtests. curious.
ta
That's a good question. I have a few approaches but the issue isn't quite backtesting on it's own, but what exactly you want to prove with said backtest.

For time series analysis (which is not my bread and butter but likely where most people start in the algo trading world,) you could use something like MT4/5, MultiCharts, or NiinjaTrader to test and prove trading ideas. However, the downside of time series analysis are rampant though:

  • Candlesticks are painted based off either ticks (time of sales, transaction records,) or in MT4's case, the bid price. Both are narrow slices of the total picture of what's happening in the market at a given point of time. You do not see where people have resting orders
  • Candlesticks / bar data is also VERY machine readable, and very accessible to market participants. Meaning if RSI + MACD with any given settings could yield a profit and prove alpha, you better believe some computer has already chewed through every combination and the results are known and/or already crushed out.
  • Candlesticks / bar data is also very low resolution.. it's just Open, High, Low, Close (and maybe Volume,) within a given period of time. 4 or 5 data points to summarize what happened over 60 seconds. Look at 1 minute of the SPY ETF's TOS (tick data, transaction records, time of sales, etc..) data to realize the volume of information being devolved into just 5 float numbers.

A lot of my strategies depend on things like knowing where the spread / L1 / NBBO prices are compared to where the security is trading actively (transactions,) which cannot be tested on platforms like MT4... or, for that matter, cannot be traded on MT4 since anything dealt to you over MT4 is on a made market and no one is going to cross the spread and hit your resting order like you'd find on a proper exchange. But I digress..

So... to overcome this.. I use a bit of a hybrid between using MT5 / NinjaTrader to 'replay' data via their strategy tester features (which load my code as indicators and pipe out the market replay data to my algo platform, which then makes trading decisions and communicates back, and some 3rd party tools like BuildAlpha for quick vetting of simple ideas. The market replay method is slow... very slow... so I'll sometimes prototype an idea in NT8 or MT5 directly to vet it should I want to run a test across more than a few months worth of data at a time, or I'll use BuildAlpha with custom imported data should the idea rely on just simple time series analysis.

I'd like to avoid writing a backtesting engine directly into my algo platform because this would be hella time consuming and I'd feel like I'm reinventing the wheel. I can see such functionality being helpful, but I can also use other platform to do this for me without the time commitment to coding it myself.

As a side note: I also trust forward testing a lot more than back testing, plus I'm actively watching the market, order books, transaction feeds, etc.. while the bots are working with small risk in live, or in sim. That way the feedback I get is FAR more useful than just straight P/L numbers.

Anyhoo.. hope that answers your question.
 

sven

Member
I'd like to avoid writing a backtesting engine directly into my algo platform because this would be hella time consuming and I'd feel like I'm reinventing the wheel.
Phew I thought you were going to say you built a full blown backtesting framework and I was going to have to leave the room ;)


I guess due to the markets I'm limited in I'm going to have to focus on timeseries and hope I can exploit some mediocre feature. Would not know where to begin on L1 and such.. one for a little further down my journey I suspect.

thanks again.
 

Top