Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
 

Author Thread: Jack's Automated System Journal -- V2  (Read 2375 times)

Offline Jack

  • Facilitator
  • ****
  • Posts: 910
    • View Profile
    • FXGears
Jack's Automated System Journal -- V2
« on: April 25, 2013, 06:35:58 pm »
Disclaimers:
  • I am not a seasoned automated systems developer. I'm a manual trader, and I don't pretend to be much else.
  • I'm also not a programmer by trade. I've had some experience scripting in my IT days, and I'll admit I first started playing with EAs and MQL4 indicators as a way to keep sharp in coding, but I'm far from considering myself 'good' in that department.
  • So read no further if you expect tips from a master coder who designs automated systems in his sleep. :P The main purpose of this journal is to detail my thoughts and findings as I apply my years of experience in manual trading to automated systems.
  • I won't be detailing strategies or providing EAs here. Heck, I don't even want to disclose the pairs they run on. I'll be happy to share my progress in development, the performance so far, methods I've used to test and refine, etc... but the strategies themselves will be held back (as well as details I feel could allow people to figure out said strategies.)
Overview:

This journal will consist of two main themes: My thoughts on the process and craft, as well as the performance and commentary of any strategy implemented live.

I'm not writing this journal to show off or prove anything. I just felt that since I'm starting a new theme and journal topic I could put it online so others might benefit from my experiences.

For manual trading, I keep physical journals handy to write down my thoughts. I don't leave for work without my journal, and it stays beside my desk at work and at home with a pen beside it for easy access.

I find keeping a journal to be a very important tool for traders, but it's the kind of tool you need to put effort into in order to get anything out of it.

The first part of this journal will bring people up to speed of where I'm at now. Then I'll be detailing things as I progress.

Lastly, all strategies that I've put into 'production' with live trading will be within the same account (to start) and have a myfxbook page to track it's progress. I'd like to use ForexFactory's Trade Explorer, but it doesn't allow me to hide historical/open trades while still showing the balance and equity. Til they add that feature, I'll have to stick to myfxbook.

You can follow the MyFXBook page here:
http://www.myfxbook.com/members/FXGearsDOTcom/jacks-algo-trading-journal/408319

And MyFXBook widgit here:


The idea here is you can see the progress for yourself that's verified by a third party. Showing the net P/L balances after open positions are accounted for will also mean I can't hide losses.

To include aspects of this journal and any code snippets / tools I've found useful. This will happen within a few weeks, since it's in the process of a redesign. Of course, all content will be available here as well.

Rules


Do Not's:
  • Do not ask me to reveal any strategies.
  • Do not ask me to sell anything.
  • Do not promote the use of martingale in this thread, nor discuses it. It won't ever be used and I don't feel like mucking this thread up with posts about how it's bad.
  • Do not discuss commercial EAs unless it's about their strategy or components of their strategy for use in custom rewritten code.
  • Do not post decompiled or cracked code.


Do's:
  • Do ask questions, give your opinion, suggest things... etc..
  • Do be positive and polite toward other traders posting here.
  • Do share code snippets, experiences, etc...
  • Do give credit where credit is due. If you didn't write the code yourself, just mention who did or where you found it.
  • Do have fun! :P


Now lets's get started! :D


[table of contents will be placed here in the future for easy navigation once this thread gets past a few pages.]

Offline Jack

  • Facilitator
  • ****
  • Posts: 910
    • View Profile
    • FXGears
Re: Jack's Automated System Journal -- V2
« Reply #1 on: April 25, 2013, 06:38:00 pm »
Bringing you up to speed.

Around two years ago I started playing with writing my own scripts for trading platforms. It was an excuse to keep sharp with scripting/coding, which I did a bit of back in my IT days.

It started with ThinkScript, a javascript-like language used by Thinkorswim's platform for chart indicators.

I use Thinkorswim for charting stocks and indexes since I have an account with them for equity options and there's no data fees for realtime charting / quotes. Their charting package actually blows away the charting tools at my work, so I started using it instead of my work's in house software. Plus, it's a lot cheaper than an eSignal subscription or the like.

I wanted some indicators they didn't have built in, and I wanted to highlight a few patterns, candlesticks, etc... so I sat down and went through the tutorial on ThinkScript, and turned out a few scripts and indicators that did what I want. It surprised me how easy it was (they made ThinkScript for traders, not coders, so it was simple but kinda limited in ways.)

That experience made me want to pick up MQL4 for MetaTrader. I started with following the official book by MetaTrader (http://book.mql4.com/) and taking notes. The syntax was simple to pick up but I found the script's structure to take a few passes before I could wrap my head around it.

I converted a lot of the scripts I wrote for thinkorswim into MQL4 for MetaTrader, and once I had a handle on that I wanted something more challenging.

This lead me to EAs, and the MQL4 code base by MetaQuotes.

At first, all I did was download anyone's freely available EA and backtest it with all sorts of custom settings in hope to find something profitable. But this was fruitless. It was interesting to me just how much code was out there that didn't work well or just wasn't in tune with the market. I knew I'd have to write the strategy part of it myself if I wanted to see any real progress.

I also started to notice other programmer's ways of compensating for a failing strategy: They'd start to curve fit their strategies so it works rather well over a period of time in the past, but once in production the EA would fail hard or not perform as well since the strategy was over tuned for a past market condition. Or worse yet, they'd dabble in betting systems a la martingale or the like. Sigh..

At least I was learning from others here..


This lead me to the following realizations:
  • While you can optimize an EA for a period of time, the optimization can't be overly strict or lead to curve fitting.
  • The EA/strategy has to have an edge first, and money management can only server to limit risk or increase that edge, not be the edge in itself (this wasn't so much a realization as it was affirming what's true in manual strategy development as it applies to the automated world.)
  • The EA has to be backtested for many years. Even if it performs poorly a few years back, it's important to understand why it didn't do so well and the risk of it not doing so well again in the future.
  • An edge in an automated system is harder to come up with than a manual system, for it runs all day long and doesn't have the same sense of the market that you have to not take a valid signal (like just before a news release for instance.)

Offline Jack

  • Facilitator
  • ****
  • Posts: 910
    • View Profile
    • FXGears
Re: Jack's Automated System Journal -- V2
« Reply #2 on: April 25, 2013, 06:39:49 pm »
But I had to start somewhere...

I also came to the conclusion that if I wanted to trust the results of my testing, that means striving for the absolute best testing environment possible, and even the best has to be scrutinized.

After reading around the net on the subject of backtesting, I found a lot of sites and forums suggesting that the default historical data available in MT4 was junk, especially if you're testing a strategy that works on lower timeframes or holds tight stops. It made sense once I thought about it, tick level historical data is very bandwidth intensive to distribute out, and takes a long time to download, so what MetaQuotes does in MT4's history center is provide down to 1M period data and 'tick volume' numbers for the period. If the tick volume period is high, then it randomly makes the price jump around within that period's range as if there were many ticks. It doesn't actually follow the real market ticks and price changes that happened during that time.

So off I went to find proper tick data for my testing, leading me to two sources:

TrueFX

and

Dukascopy

TrueFX has a section of their site where you can download CSVs of tick level data from Integral's generic feed. It only goes back to about 2009 (if I recall correctly) and it's broken up into one file per month, so this method is a bit of a headache, but it's best if your trading on a broker who uses their price feed.

People have been having issues with registering on the TrueFX site, so I'll take advantage of their lazy IT people and post the direct download links:
http://truefx.com/dev/data/

Again, you have to combine the files since they are broken up into months.

So first extract the files (they are large once extracted, expect to have about 2-3Gigs per symbol for 2-3 years of data.) Then, from Windows' command prompt we use the copy command to merge the extracted files:

Code: [Select]
copy USDJPY*.csv USDJPY.csv
This will work since it copies in numerical order (the dates in the file name) with the wildcard (*) in place.

You will end up with one big "USDJPY.csv" file, and if confident things went without a problem, you can delete the old zip and monthly csv files.

The Dukascopy option is a lot easier. Just sign up for a demo and once you're in JForex, there's a history center tab where you can download tick data to CSV. You'll have to set the date range desired, the period to "ticks", and such, but it will product a single file that's ready to be converted to MT4. While easier, this might not be best if you trade with brokers that have vastly different spreads than Dukas.

The next step is taking the monolithic CSV file and turning it into something MetaTrader can use. This part is a bit more complicated and would take up a few posts in itself.. so I'm going to skip writing about it and instead point you in the right direction:

Google "eareview convert-tick-data"

The first result is what you want. I don't want to link there directly since the site is full of affiliate links to commercial EAs.

Basically, you want to follow the steps on that site for the CSV2FXT script.

Lastly, on the subject of tick data, I'd suggest installed a separate instance of MT4 to a different folder for testing use. That way you can mess up all sorts of stuff without botching your regular installation setup. Plus, once setup, you'll be doing most of the backtesting offline (so metatrader doesn't add it's own history to your collected data.)

All this might be boring to some, but it's part of the process, and the difference between bad historical data and tick data could mean overlooking a good strategy or going forward with a bad strategy you think is good.

So moving on...

Offline Jack

  • Facilitator
  • ****
  • Posts: 910
    • View Profile
    • FXGears
Re: Jack's Automated System Journal -- V2
« Reply #3 on: April 25, 2013, 06:41:26 pm »
First steps of a long journey.

With a good testing environment in place, we can start really digging into the fun stuff: Strategy development.

As I said in the first post, I'm not a seasoned automated strategy developer. I have an idea about how the big boys do it (heavy statistical analysis and risk modeling) but I just don't have the resources to do things on their level (a floor full of analysts and researchers combined with a few mathematicians, programmers and a server farm would be awesome, but it's just not in my budget. :P) So basically looking for a possible edge first, then building rules around it to define and bring the edge out.

In looking through the code examples from MetaQuote's codebase, and digging all over the net for free EAs with source code to learn how they are structured and written.. I'm left thinking the other end of the spectrum (individuals, non-firms) are doing things in the wrong order. It's very common to see strategies written around an indicator or single as if adding rules to said indicator or signal will make a profitable system.

So I took a page from the 'big boys' and chose to stop messing around with signals and rules, hoping some magic combination would just hit me in the face as I test... and instead, start detailing things from the top down and analyse price relations first in looking for a theoretical edge (later using signals/patterns/rules and such to see if it's viable.)

I'm not sure if you're following me on this, but I'm trying to explain the difference in mindset when I approach the problem of finding a strategy. I figure if I'm going to grind away crunching data and testing (and believe me, there's a lot of this involved) then I probably shouldn't start where everyone else is starting since it doesn't appear to be working very well for them.

I also wanted to come up with strategies that fit my personality. I don't like holding risk overnight, so the big hurdle here would be coming up with something that won't cause me to lose sleep. As well, I don't mind trading often so long as it's worth the risk and cost (plus, more frequent trades means a larger sample size to analyse and more accurate expectancy.)

So I outlined a series of objectives for my ideal strategy, which would help me narrow down what sort of price relationships I analyse to search for an edge:

  • Trades frequently, few times per week if not a few times per day. Again, this relates to sample size in testing.)
  • Is based around a statistical edge that I uncovered on my own and won't reveal to protect said edge.
  • Drawdowns are ok so long as the upside potential is present. If a drawdown period spanning multiple days/weeks/months can take 10%+ out of an account, then recovery better be a multiple of that amount and happen quickly once the strategy is in favor with the market.
  • That is focused mainly on one pair, and not a strategy that's trying to perform well on all pairs.

Offline Jack

  • Facilitator
  • ****
  • Posts: 910
    • View Profile
    • FXGears
Re: Jack's Automated System Journal -- V2
« Reply #4 on: April 25, 2013, 06:43:43 pm »
Some preliminary technical requirements.

I also realized I'd need to outline some technical requirements so I don't spend time redoing work or waste my efforts on redundant things. Plus, I wanted an environment where I could express new ideas for strategies quickly without being bogged down writing tonnes of code for each strategy's order/risk/etc management.

So my base code has to:

  • Allow me to define pretty much any variable from a set file or external parameters making most adjustments simple and quick.
  • Keep track of account balance and risk.
  • Be able to only work with a % of the total balance should it be sharing an account with another EA.
  • Not confuse orders with another EA.
  • Have a way of handling MT4 errors and disconnections. Specifically, knowing when to resend an order request or knowing it still has to close out an order once connectivity comes back.
  • Manage multiple trades as one single 'position' should it be breaking up entries.
  • Set stops but also close out trades based on account risk rules (don't just let a trade go south til it hits a stop if some other risk rule was violated.)

Actually, this list goes on for some time... I've been adding to my own personal code base for a while with functions that do most of this, but there's always more that could be added.

I won't pretend that I wrote all that functionality from scratch, in most cases I stitched borrowed code blocks from various sources and combined them to do what I needed. I'll do my best to provide said code here as well, quoting specific examples that came in handy or neat tricks/mods I came up with, but the bulk of the code I found (again) on the MQL4 community code base, portions order management code from some free EAs by lifesdream(dot org), and account level risk and proportion control from "Blessing" by JTalon (please do not mistaken me here, Blessing was written to be a martingale grid trader, and I didn't want anything to do with it, but I found the fairly well written code it uses to manage account and trade level risk to be very suitable so I used parts of it directly while using other parts as inspiration.)

With a small collection of code blocks in place, I could now just focus expressing strategy ideas and testing them out.

Offline Jack

  • Facilitator
  • ****
  • Posts: 910
    • View Profile
    • FXGears
Re: Jack's Automated System Journal -- V2
« Reply #5 on: April 25, 2013, 07:10:56 pm »
Teasing myself...
 
Up to this point, all this was just an exercise and experiment for me. I had no expectations that I'd come up with anything significant.

I started plugging away, playing off some of the trendy EAs currently all the rage right now. Tick scalping, asian session scalping, etc...

I found out the following:

Tick scalping is very hard to backtest. Sure tick level data can be accurate, but it doesn't account for slippage and bad fills.. and tick scalpers are very prone to that. So going this route would mean I have to forward test to be sure, and even then it would have to be live before I could truly trust the results. Not a fan of that prospect.

If you aren't aware, tick scalpers like the commercial EA's NumberOne and wallstreet, follow price spikes with a trailing stop limit order to catch a whipsaw in the other direction. These spikes and whipsaws happen so fast, that often the fill prices are way off from where the order was triggered.. and since the expectancy of a tick scalper is a profit of just a few pips, even 1 pip of slippage can destroy the results of the EA. So the strategy doesn't exactly work well on most brokers... might be best used on some API connection right to an ECN.

-

Then I noticed a common price relation at nearly all times of day on one pair that stood out more than others. I put the idea into code and ran a few backtests...

My heart nearly stopped when I saw this:


(this is using myFXbook's backtest analysis tool, basically I uploaded MT4's Strategy Tester report HTML file to their 'strategies' tool and used it to look over the results.)

That's starting from $300 and only one year worth of data. It performed about the same for 2010, 2009, and most of 2012. Had I made this strategy report public (instead of private in myFXbook's settings,) it would take first place of all public strategies and would do so without tricking the system.

So here I am jumping around my living room as if I won the lottery... and then I realize, the spread setting was wrong, and I wasn't accounting for the total cost of trading.

And reality set in... it stung... I made the corrections and ran it again. Not nearly as impressive this time.. maxes out at a few hundred thousand. Still not bad, but not millions like before. :P It also has a very nasty drawdown, and on any conservative setting (dropping the drawdown to below 20%) the return doesn't get a chance to compound as well and limps along.

In the end, I had just teased myself. I had a positive expectancy system, but only slightly positive, with a final profit factor of between 1.11 and 1.21 depending on what year I was testing. In the real world's trading conditions, factoring in slippage and such, there was no way this strategy would turn $300 into such a high amount like it did in my initial test.

But it was a start... and the first strategy / idea that I came up with on my own to show such promise. So it was time to refine and optimize with more realistic trading conditions to see if this thing would survive running live.

Offline Jack

  • Facilitator
  • ****
  • Posts: 910
    • View Profile
    • FXGears
Re: Jack's Automated System Journal -- V2
« Reply #6 on: April 25, 2013, 07:18:12 pm »
Error Reporting and Good Coding
 
A little side track here, but about practical things:

When testing code, self written EAs, etc... error catching and reporting can be really useful.

One of the code blocks I found online that covered this well, not only reporting the error but telling you what the error means in english right in MT4's journal log was found tucked inside some free EAs that Boston Technologies put on their site:

First, it creates a proper dispatch table for error codes so it can return info on the error itself (holy shit, a switch statement, you have no idea how refreshing it was to read this after sifting through so many endless if/else sequences for the same job, which is a very 'expensive' operation in the coding world.)

Code: [Select]
string ErrorDescription(int error_code)
{
    string error_string;

    switch( error_code ) {
        case 0:
        case 1:   error_string="no error";                                                  break;
        case 2:   error_string="common error";                                              break;
        case 3:   error_string="invalid trade parameters";                                  break;
        case 4:   error_string="trade server is busy";                                      break;
        case 5:   error_string="old version of the client terminal";                        break;
        case 6:   error_string="no connection with trade server";                           break;
        case 7:   error_string="not enough rights";                                         break;
        case 8:   error_string="too frequent requests";                                     break;
        case 9:   error_string="malfunctional trade operation (never returned error)";      break;
        case 64:  error_string="account disabled";                                          break;
        case 65:  error_string="invalid account";                                           break;
        case 128: error_string="trade timeout";                                             break;
        case 129: error_string="invalid price";                                             break;
        case 130: error_string="invalid stops";                                             break;
        case 131: error_string="invalid trade volume";                                      break;
        case 132: error_string="market is closed";                                          break;
        case 133: error_string="trade is disabled";                                         break;
        case 134: error_string="not enough money";                                          break;
        case 135: error_string="price changed";                                             break;
        case 136: error_string="off quotes";                                                break;
        case 137: error_string="broker is busy (never returned error)";                     break;
        case 138: error_string="requote";                                                   break;
        case 139: error_string="order is locked";                                           break;
        case 140: error_string="long positions only allowed";                               break;
        case 141: error_string="too many requests";                                         break;
        case 145: error_string="modification denied because order too close to market";     break;
        case 146: error_string="trade context is busy";                                     break;
        case 147: error_string="expirations are denied by broker";                          break;
        case 148: error_string="amount of open and pending orders has reached the limit";   break;
        case 149: error_string="hedging is prohibited";                                     break;
        case 150: error_string="prohibited by FIFO rules";                                  break;
        case 4000: error_string="no error (never generated code)";                          break;
        case 4001: error_string="wrong function pointer";                                   break;
        case 4002: error_string="array index is out of range";                              break;
        case 4003: error_string="no memory for function call stack";                        break;
        case 4004: error_string="recursive stack overflow";                                 break;
        case 4005: error_string="not enough stack for parameter";                           break;
        case 4006: error_string="no memory for parameter string";                           break;
        case 4007: error_string="no memory for temp string";                                break;
        case 4008: error_string="not initialized string";                                   break;
        case 4009: error_string="not initialized string in array";                          break;
        case 4010: error_string="no memory for array' string";                             break;
        case 4011: error_string="too long string";                                          break;
        case 4012: error_string="remainder from zero divide";                               break;
        case 4013: error_string="zero divide";                                              break;
        case 4014: error_string="unknown command";                                          break;
        case 4015: error_string="wrong jump (never generated error)";                       break;
        case 4016: error_string="not initialized array";                                    break;
        case 4017: error_string="dll calls are not allowed";                                break;
        case 4018: error_string="cannot load library";                                      break;
        case 4019: error_string="cannot call function";                                     break;
        case 4020: error_string="expert function calls are not allowed";                    break;
        case 4021: error_string="not enough memory for temp string returned from function"; break;
        case 4022: error_string="system is busy (never generated error)";                   break;
        case 4050: error_string="invalid function parameters count";                        break;
        case 4051: error_string="invalid function parameter value";                         break;
        case 4052: error_string="string function internal error";                           break;
        case 4053: error_string="some array error";                                         break;
        case 4054: error_string="incorrect series array using";                             break;
        case 4055: error_string="custom indicator error";                                   break;
        case 4056: error_string="arrays are incompatible";                                  break;
        case 4057: error_string="global variables processing error";                        break;
        case 4058: error_string="global variable not found";                                break;
        case 4059: error_string="function is not allowed in testing mode";                  break;
        case 4060: error_string="function is not confirmed";                                break;
        case 4061: error_string="send mail error";                                          break;
        case 4062: error_string="string parameter expected";                                break;
        case 4063: error_string="integer parameter expected";                               break;
        case 4064: error_string="double parameter expected";                                break;
        case 4065: error_string="array as parameter expected";                              break;
        case 4066: error_string="requested history data in update state";                   break;
        case 4099: error_string="end of file";                                              break;
        case 4100: error_string="some file error";                                          break;
        case 4101: error_string="wrong file name";                                          break;
        case 4102: error_string="too many opened files";                                    break;
        case 4103: error_string="cannot open file";                                         break;
        case 4104: error_string="incompatible access to a file";                            break;
        case 4105: error_string="no order selected";                                        break;
        case 4106: error_string="unknown symbol";                                           break;
        case 4107: error_string="invalid price parameter for trade function";               break;
        case 4108: error_string="invalid ticket";                                           break;
        case 4109: error_string="trade is not allowed in the expert properties";            break;
        case 4110: error_string="longs are not allowed in the expert properties";           break;
        case 4111: error_string="shorts are not allowed in the expert properties";          break;
        case 4200: error_string="object is already exist";                                  break;
        case 4201: error_string="unknown object property";                                  break;
        case 4202: error_string="object is not exist";                                      break;
        case 4203: error_string="unknown object type";                                      break;
        case 4204: error_string="no object name";                                           break;
        case 4205: error_string="object coordinates error";                                 break;
        case 4206: error_string="no specified subwindow";                                   break;
        default:   error_string="unknown error";
    }

    return(error_string);
}

After that, they can work that into error catching as various tasks fail, here are some examples


Code: [Select]
Print("OpenTrade: error '"+ErrorDescription(GetLastError())+"' when setting SL/TP");

Code: [Select]
Print("Exit: error '"+ErrorDescription(GetLastError())+"' when exiting with "+DoubleToStr(volume,3)+" @"+DoubleToStr(prc,Digits));
So now the log files and MT4 Journal will actually be useful, instead of me getting some unknown error code and having to look it up myself..

Offline Jack

  • Facilitator
  • ****
  • Posts: 910
    • View Profile
    • FXGears
Re: Jack's Automated System Journal -- V2
« Reply #7 on: April 25, 2013, 07:31:36 pm »
Measuring System Performance

I suppose I'll get the dull subjects out of the way.. since I don't feel very motivated to write about them yet they are rather important in my process (and great tools to understand in general if you're any type of systems trader, automated or not.)

For performance comparison, I'll be looking at the following metrics:

Drawdown, both realized and unrealized losses - I don't mind a high drawdown so long as the profit potential is many multiples of it, which means I can reduce size or risk in some other way and get a more favorable outcome. That is to say, I'm not scared off by high drawdown so long as other aspects of the system meet the right ratios in relation to it.
Time to recovery while in drawdown. Few reasons for this, the market changes often so it would be harder to judge a system getting out of sync with the market itself when it has a long average period of time between peek and recovery. For instance, if I have to wait half a year for a system to make a new high, a lot can change in the underlying structure of the market in that time, and maybe I won't notice til a lot more damage was done.
Number of trades - If two variations of a system have the same metrics in other areas, I'd probably want the one with the greater number of trades. If you have a compelling reason why this is wrong, please speak up, I haven't researched that far in this direction, I just like having more trades in a sample to analyse (and thus have a higher expectation that my analysis is right.)
Profit Factor - Total gains divided by total losses. This little guy is key, and also very hard to keep above a decent level. The main reason I don't care about a win/loss ratio is profit factor. Ideally, I'd like to get a PF of 1.10+ as a baseline for any strategy employed, should it trade frequently that is... if infrequent (like once every few days) then it better be much higher since I'd have far too few trades in my backtesting sample size to justify a lower PF (risk wise, I need to expect to win a lot more per dollar lost should it only take a shot every so often, and thus probably have a bigger bet size.)

Lastly, if I can't simulate at least 1000 trades on any strategy over the last 2-3 years, it's hard for me to trust the system. This is sample size again, I know I'm harping on it, but this comes from my manual trading. Do you want to know how I scalp and discretionary trade with confidence with manual execution? Many thousands of trades to review, analyse, and show there's a positive expectancy...


-

For strategy refinement I completely abuse MT4's optimization functions...but one must be careful with these since they often lead to curve fitting. For this, I do regression testing with each adjustment to see how it might impact the strategy in other time periods.

Sometimes I get an idea that makes a given strategy work better during the recent price action we've been seeing in the market. However, if I run this adjustment against the last few years of data and it cripples the results the non-adjusted strategy would return, then the idea is tossed out.

Ideally, when I'm making these adjustments or testing variations of a strategy, I'm looking to either reduce drawdown, or increase profit potential. If the profit potential has increased, but so has the drawdown, then it's garbage..since I can do the same by just jacking up the lot size in the original strategy.

Offline Jack

  • Facilitator
  • ****
  • Posts: 910
    • View Profile
    • FXGears
Re: Jack's Automated System Journal -- V2
« Reply #8 on: April 25, 2013, 07:32:56 pm »
Measuring System Performance

I suppose I'll get the dull subjects out of the way.. since I don't feel very motivated to write about them yet they are rather important in my process (and great tools to understand in general if you're any type of systems trader, automated or not.)

For performance comparison, I'll be looking at the following metrics:

Drawdown, both realized and unrealized losses - I don't mind a high drawdown so long as the profit potential is many multiples of it, which means I can reduce size or risk in some other way and get a more favorable outcome. That is to say, I'm not scared off by high drawdown so long as other aspects of the system meet the right ratios in relation to it.
Time to recovery while in drawdown. Few reasons for this, the market changes often so it would be harder to judge a system getting out of sync with the market itself when it has a long average period of time between peek and recovery. For instance, if I have to wait half a year for a system to make a new high, a lot can change in the underlying structure of the market in that time, and maybe I won't notice til a lot more damage was done.
Number of trades - If two variations of a system have the same metrics in other areas, I'd probably want the one with the greater number of trades. If you have a compelling reason why this is wrong, please speak up, I haven't researched that far in this direction, I just like having more trades in a sample to analyse (and thus have a higher expectation that my analysis is right.)
Profit Factor - Total gains divided by total losses. This little guy is key, and also very hard to keep above a decent level. The main reason I don't care about a win/loss ratio is profit factor. Ideally, I'd like to get a PF of 1.10+ as a baseline for any strategy employed, should it trade frequently that is... if infrequent (like once every few days) then it better be much higher since I'd have far too few trades in my backtesting sample size to justify a lower PF (risk wise, I need to expect to win a lot more per dollar lost should it only take a shot every so often, and thus probably have a bigger bet size.)

Lastly, if I can't simulate at least 1000 trades on any strategy over the last 2-3 years, it's hard for me to trust the system. This is sample size again, I know I'm harping on it, but this comes from my manual trading. Do you want to know how I scalp and discretionary trade with confidence with manual execution? Many thousands of trades to review, analyse, and show there's a positive expectancy...


-

For strategy refinement I completely abuse MT4's optimization functions...but one must be careful with these since they often lead to curve fitting. For this, I do regression testing with each adjustment to see how it might impact the strategy in other time periods.

Sometimes I get an idea that makes a given strategy work better during the recent price action we've been seeing in the market. However, if I run this adjustment against the last few years of data and it cripples the results the non-adjusted strategy would return, then the idea is tossed out.

Ideally, when I'm making these adjustments or testing variations of a strategy, I'm looking to either reduce drawdown, or increase profit potential. If the profit potential has increased, but so has the drawdown, then it's garbage..since I can do the same by just jacking up the lot size in the original strategy.

Here's food for thought, a technology note if you will:

MT4 is old as dirt.. pushing past 10 years now.

It's strictly 32bit. It can't make use of more than 2Gb of RAM, single threaded, and it doesn't handle well on Windows Vista and beyond.

It also can't make use of a lot of the newer hardware assisted math functions found in some processors... nor have I seen anyone putting out a version that can take advantage of GPU computing to speed backtesting up.

Basically, it's dated.

So I found for the best possible experience (speed wise,) for MT4 backtesting, start by setting up a system with a higher clock rate, higher cache, dual core (not quad or greater) processor. You don't even need dual core, but it's not like you can find single core processors these days and at least the 2nd core means you can browse the net without it feeling sluggish as your MT4 works away on a backtest.

Run 32bit Windows XP... actually, I've been told by a very high up person at a brokerage (who knows his way around IT stuff) that the best OS for MT4 (stability and speed wise) was Windows 2000..

64bit Windows has to run MT4 in WOW mode, that's Windows-on-Windows... it's a compatibility layer and it slows the kinda processing MT4 does during backtesting down quite a bit.

Don't worry about RAM greater than 3-4Gb. More is just not needed (OS won't use it, MT4 won't use it.)


Just a few thoughts if you ever are building out a system specifically for MT4 backtesting. You won't be future proofing yourself going this way, but at least MT4 will run like a dream.

Offline Jack

  • Facilitator
  • ****
  • Posts: 910
    • View Profile
    • FXGears
Re: Jack's Automated System Journal -- V2
« Reply #9 on: April 25, 2013, 07:33:34 pm »
So I started running that first strategy I found from the earlier post (with the account balance chart rocketing to the moon) through all sorts of backtests.

I think I changed nearly every setting once, making my computer run overnight testing it (optimization mode) a few times.

I came upon two very interesting problems and related questions that I never expected to arise:

1) I had two different ways of managing trades post entry that were both very good. While they performed differently in the long run, one was clearly stronger during the times that the other wasn't doing so well. They had nearly the same metrics, profit factor, etc... one had nearly twice the amount of trades over 3 years but both had sufficient trades to consider...This to me made me think that I could split up my accounts capital into two pools, and have one method work on each. The result (in theory) would give me a blended return between the both of them.

But that also presented two new questions:

What weighting would I give each method?

And would the account be split so the wins and losses are evenly divided between the methods or should each method keep track of a 'logical' pool to work from independent of each other?

2) Over the thousands of sample trades that proved a positive expectancy, the expectancy is on the signal itself... so if I'm in a trade, but get a signal in the opposite direction, should I take that as well?

The expectancy wouldn't change on the signal level, so in theory I should take the opposite trade and treat it separately.. but that kinda goes against my feelings toward hedging. And how would this impact performance overall?

Also, if done this way, I may find that additional adjustments could be done to just one direction over the other, making my testing work doubled and all previous adjustments that I ruled out on the combined (long/short) methods would have to be redone for just one side.


-


In the case of my first strategy and the two trade management methods I detailed, I worked out the numbers and came up with a 60/40 split of capital to each method (based on possible drawdown of each method, but I'm still reviewing and looking for better possible ways to allocate cash between two methods.)

And I decided to allow the trading of signals in the opposite direction of an existing trade. My theory here is while the initial trade is on, and possibly in the red, the other signal might book some profits reducing the total hit to the account balance. I also worked out a decision tree for the strategy based on adding in the opposite direction signal trades, and it helped make a few outcomes positive while only making one outcome worse off (with a very low likelihood of occurring, having one signal go bad, trigger the 2nd, stop out the first, then revers and stop out the 2nd. This is the worst case of adding in the opposite trade, but even if it happens the total loss wouldn't be much of the account ~3%)

Offline Jack

  • Facilitator
  • ****
  • Posts: 910
    • View Profile
    • FXGears
Re: Jack's Automated System Journal -- V2
« Reply #10 on: April 25, 2013, 07:34:31 pm »
Baby Names

So it was time to forward test, and at this stage I wanted to come up with a nick name for each method so I could stop referring to them by their test sequence numbers.. lol

I picked two names that reflected the method's personality a bit:

Runner - Who holds trades much longer, gets an average price by grabbing a 2nd or sometimes 3rd position depending on how the market is working (it won't average into a shitty trade, but if things are calm and going according to plan it will grab a bit more and tighten risk.)

Dash - Who will dump trades rather quickly if they move against him, and instead want to be available should the signal reappear soon there after.

Mind you, or remind rather, these are both the same strategy, just different methods of handling the trade once the signal is given.

I also broke out the long and short sides to these methods to separate EAs. It makes the coding side easier since I wouldn't have to double up a bunch of logic code and instead just set a variable to one direction or another.

So now my MT4 setup has 5 total charts up, 4 EAs running on 4 charts:
  • Runner LONG
  • Runner SHORT
  • Dash LONG
  • Dash Short

And the 5th chart is for monitoring.. where I keep market metrics up and some of the calculations the EA's use to base trading decisions on. I also have a "panic" button setup on this chart which kills all open trades.


And so it begins...

Offline Jack

  • Facilitator
  • ****
  • Posts: 910
    • View Profile
    • FXGears
Re: Jack's Automated System Journal -- V2
« Reply #11 on: September 21, 2013, 06:17:44 pm »
Just wanted to give this thread a quick update:

I'm still working on automated FX systems.

I've shelved my main AUDNZD strat simply because we moved into fresh yearly (or 3 year period) lows on a hard downtrend, and there was a component in the strategy that bases some judgement on recent price levels, so without the point of reference, it was trading a bit blind.

I'll wait til we establish a new consolidation range on the weekly/monthly chart and flip the switch back on then.

I've also moved that strat onto it's own account some time ago and away from the live account I use for testing and development (the one linked to this thread.) I'll be experimenting with new strats on the algo journal account very soon.

--

Here's an example of some new work I've done recently using mean reversion and averaging:



(Yes, I know what it looks like, and like I said, mean reversion and averaging, so there is a lot of risk here you're not seeing via the equity trend line and the edge here is still based off probability and stats, not some betting style or blind averaging... so don't take this the wrong way.)

Obviously the drawdown % makes the 505% profit very unlikely once I optimize this better for risk.. but it's still not bad for 1 month worth of trading and no news filters.

The idea I'm working with (in general) is that I can't easily predict when a given strategy will not be in favour with the market, so instead I want to push the risk envelope as hard as I can while it is working to make the most possible. Hopefully, that profit is greater than the initial account balance should the market fall out of favour and the strategy blow out.

In the example above, I would have pulled down 5x the account value (and potentially more had I left it running,) so I'd consider it a step in the right direction. Still lots of work to be done though.. once it hits the live testing stage, I'll put it on the journal account linked in the first post. :)
 

Offline Ramy3

  • Sr. Member
  • ****
  • Posts: 427
  • May the pips be with you!
    • View Profile
    • Figure it out! :P
Re: Jack's Automated System Journal -- V2
« Reply #12 on: September 21, 2013, 06:24:38 pm »
“Me thinks” you might be finally opening up your thinking/mindset. Just be careful and still keep in mind the potential dangers... Steady returns are way better than fast profit with high risks. IMOO.

Best of luck mate.

Cheers!
« Last Edit: September 21, 2013, 07:06:21 pm by Ramy3 »
No right or wrong way to trade. Only results matter.

0 Members and 1 Guest are viewing this thread.