Can Neural Networks beat the Crypto Market of September [Part 4]

10 min readNov 11, 2018

Check out my latest app — TradingGYM. It’s a trading simulator that helps you practice trading with a much faster feedback/learning loop and minimal lookahead bias. Once you open the app, you will see a chart with a random asset at a random point in time. Make trades and fast-forward time to see how it played out.

This is Part 4 in multi part series:

Part 1: Basic strategies, introduction, setup and testing vs June-July market.
Part 2: Advanced strategies and where to find them, testing vs June-July market.
Part 3: Basic and Advanced strategies testing vs August market.
Part 4: Neural Network strategies description and backtests against September market.
Part 5: Neural Network strategies backtests against October market.
Part 6: Did Neural Network strategies predict November 14th price drop?
Part 7: Crypto Trading 2018 in Review: 17 Advanced + 15 Neural Net strategies tested
Part 8: Intro to Statistical Arbitrage in Crypto — Pairs Trading
[NEW] Part 9: Crypto Trading 2019 Half Year Review: 17 Advanced + 15 Neural Net strategies tested

It’s time to bring out the big guns. In this part, I have gathered the baddest, the meanest, the most advanced NN strategies that could be found publicly on the Github market. AI future human overlords vs a single line on the chart, surely such advanced technology should be enough to know where that line will draw 1 day from now, right? Question is when, not if, we all get our Lambos.

Ok let’s get serious. I have with me 15 strategies that are waiting to be backtested against September market (October coming up soon, once I have the data). I tried to dig into each one of them and understand what really is going on. And this was much harder for NN strategies (compared to normal), because they contain more math, more customization, more magic and more complex short/long conditions.

I will test against the same coins as in previous parts. But, in addition, I will use higher candle sizes — 360m and 480m. Previously high candle sizes have shown better results than smaller, so this time I decided to go even higher.

WARNING 1: For candle sizes 240m and 360m I will use smaller subset of coins due to strategy history requirements. NN strats require to be trained with historical data and on higher candle sizes that equals to months of history data, going into last year, which some coins just don’t have.

WARNING 2: You might get different results when you backtest. With NN, there is always some random factor involved, that’s just the way NN works. Some strats are more susceptible than others. In strategy descriptions, I have marked the ones that showed notable variability (>5%) in profits when run multiple times.

WARNING 3: All strategies are open source and free, which means they come without any guarantees and should be used carefully with good understanding what is going on under the hood. Use at your own risk!

Strategies used

When you really dig into the code, 2 things become painfully obvious:

most of the strategies are copy/paste of one another
the amount of unused code (lines that are never executed, variables that are never used) is quite high, in some strategies well over 50%

All strategies share some common traits:

They use some kind of NN lib. Most (13/15) use convnetjs. The other 2 use LSTM type lib — synaptic or neataptic.
Most train network simply with price. A few use indicators.
Some strats use input normalization, some don`t.
Final short/long decision comes to “is the predicted price over some % threshold compared to current price”. Sometimes additional filters with indicators like RSI/BB are used.

Since most strats are copy/pasted with some modifications/improvements, I have tried my best to decypher the inheritance chain and detect which strats are root and which are evolution.

Without any further ado, let’s get to it. Here are the strategies with small commentary about what they do and how they make decisions when to buy.

neuralnet_SirTificate

Source: https://github.com/SirTificate/gekko-neuralnet

Uses convnetjs for predictions + SMMA as input, has stoploss.

Very clean code, compared to most other strats.

Makes short/long decision based on predicted vs threshold, check() logic pretty much boils down to:

let meanp = math.mean(prediction, currentPrice);
let meanAlpha = ((meanp - currentPrice) / currentPrice) * 100;
if (meanAlpha > this.settings.threshold_buy) => go long

neuralnet_zschro

Source: https://github.com/zschro/gekko-neuralnet

90% same code as neuralnet_SirTificate. Main difference — configurable Trainer methods — Adagrad/Adadelta/Nesterov/etc.

zuki_nn

Source: https://github.com/gekkowarez/gekko-neuralnet

Very similar (90%) to neuralnet_SirTificate and neuralnet_zschro. Main differences:

Instead of SMMA, uses candle high/low/open/close + volume array for predictions, like this:

this.priceBuffer.push([
    candle.low / this.scale,
    candle.high / this.scale,
    candle.close / this.scale,
    candle.open / this.scale,
    candle.volume / 1000,
]);

In learn() function, does another forward+accuracy+backward loop which is useless IMO because trainer.train() should already do that, that’s the definition of train().

mounirs-ga-version-1

Source: https://github.com/xFFFFF/Gekko-Strategies/tree/master/mounirs-ga-version-1

Uses convnetjs NN lib like the most strats in this part. Learns by keeping history of price and constantly re-training neural trainer by re-feeding all candles again and again (not sure if re-using same price is the way it should be done). Lots of unused code that never actually get’s executed. Has StochKD indicator defined, but looks like it’s not actually used.

check() logic boils down to:

prediction = predict(item);
mean = Price[Price.length — 1];
oldmean = prediction;
meanp = math.mean(prediction, oldmean);
sig0 = meanp < mean;
if (sig0 === false) => go long

which seems confusing at the first glance, but when you dig in:

meanp is calculated as math.mean() from prediction and oldmean which are the same variable basically, so at the end meanp = prediction
mean is not mean but actually just current price
sig0 is false if prediction (meanp) < current price (mean)
at the end, goes long if sig0 === false

Very twisted logic (double negative), but translates to:

if (prediction > price) => go long

mounirs-ga-version-2

Source: https://github.com/xFFFFF/Gekko-Strategies/tree/master/mounirs-ga-version-2

Very similar, but has even more unused code than mounirs-ga-version-1, but the check() logic has nice addition — threshold (hardcoded at magical 1.7%). Logic can be reduced to:

percentvar = ((meanp - mean) / mean) * 100;sig0 = global.meanp < global.mean && meanp != 0;if (sig0 === false && percentvar > 1.7) => go long

mounirs_esto

Source: https://github.com/xFFFFF/Gekko-Strategies/tree/master/mounirs_esto

Evolution from both previous strats. Main addition is RSI prediction. check() logic in a nutshell:

sig0 = meanp < mean && meanp != 0;
sig0RSI = meanpRSI < lastRSI && meanpRSI != 0;
if (
    (sig0 === false && percentvar > 2.0) ||
    (sig0RSI === true && this.indicators.rsi.result < 15)
) => go long

n8

Source: https://github.com/xFFFFF/Gekko-Strategies/tree/master/n8

Seems like evolution from mounirs-ga-version-2. Main addition is var VarList = new gauss.Collection() which is used to calculate lowvalue which is used as threshold incheck():

var percentvar = ((meanp — mean) / mean) * 100;
VarList.push(percentvar);var VectorVar = VarList.toVector();
var Density30 = VectorVar.density(30);
var low = per(Density30[0], 10);sig0 = meanp < mean && meanp != 0;if (sig0 === false && percentvar > low) => go long

So, instead of using static threshold, it calculates it dynamically with gauss lib by using density (percentile subset of values occurring within a data set).

n8_v2

Source: https://github.com/xFFFFF/Gekko-Strategies/tree/master/n8_v2

Has even more advanced dynamic threshold detection. This is the most difficult check() method to decrypt in this series. First thing — it limits VectorVar to 1000 length:

if (VarList.length > 1000) {
    VarList.shift();
}

Next thing, instead of percentvar > low check, it uses percentvar > low — lowup

lowup is calculated as average from Lowdiff, which in turn is array of low — percentvar

That is the simple version. If you really want to dig in and understand, I suggest you draw the variables on chart to see how they interact.

n8_v2_BB_RSI_SL

Source: https://github.com/xFFFFF/Gekko-Strategies/tree/master/n8_v2_BB_RSI_SL

As the name suggests, extends n8_v2 with BB, RSI and StopLoss. Also, uses HA (Heiken-Ashi) candles for predictions, instead of standard candles. So, the value which is fed into network is:

xclose = (candle.open + candle.close + candle.high + candle.low) / 4

Both RSI and BB are not used in predictions, but only as additional filters for short/long decisions. Actually, RSI is not used directly but to calculate StochRSI.

As a result, we get most complex check() condition I have seen so far:

if (
    sig0 === false &&
    percentvar > low — lowup &&
    hasbought === false &&
    this.HACandle.xclose > previousprice
) || (
    hasbought === false &&
    this.indicators.rsi.result < this.settings.thresholds.low &&
    StochRSIsaysBUY
) ||
    Price <= BB.lower
) => go long

NNv2

Source: https://github.com/zschro/gekko-neuralnet

One of the cleanest codes of all NN strats. Has prediction part (convnetjs etc) separated as indicator, which is cool. Use SMA slow/fast to detect bull/bear trend. Uses zTrailingStop with threshold.

At the end, short/long decision reduces to:

if (predictedPercentChange > this.threshold_buy) => go long

Another cool thing — at the end(), nicely logs profits separately for bull/bear.

Interesting fact — for convnetjs.Trainer uses adadelta method unlike most other strats here, which use sgd.

NN_ADX_RSI

Source: https://github.com/xFFFFF/Gekko-Strategies/tree/master/NN_ADX_RSI

Another heavily copy/pasted code, but also with heavy modifications. This seems to be a mix between previously seen convnetjs strategies and NEO strategy, which used ROC to determine BEAR, BULL and IDLE BULL trends.

Uses convnetjs with HCL candles (high + close + open / 3) as input.

Uses soup of indicators, few of which aren’t actually used anywhere in the code:

Uses RSI to determine overbought/oversold.
Uses StochRSI, which is manually calculated by formula, instead of using indicator.
Uses EMA which is not actually used anywhere in the code.
Uses ADX / ROC.

neataptic

Source: https://github.com/jmatty1983/gekkoNeatapticIndicator

Very different from all the other strats in this part, because it uses neataptic lib (not maintained anymore BTW), which implements a very special type of NN — LSTM or Long short-term memory. There is a lot of interesting information about this specific type of NN, but the part that we should be most excited about is that a lot of people suggest that this network is especially well suited for timeseries predictions, trading included. Neataptic works by using RSI and SMA train the network and make predictions.

check() decision can be reduced to:

checkLong(candle, prediction, percentThresh) {
  return (
    prediction > candle.close &&
    this.convertToPercent(candle.close, prediction) > percentThresh
  );
},

NOTE: This strat showed strong variability in results due to NN randomness.

LSTM_MACD_RSI_V3

Source: https://github.com/markchen8717/Gekko_ANN_Strategies/tree/master/Strategies

Another LSTM outlier, but unlike neataptic strat, which uses LSTM lib with the same name, this one use synaptic lib, which seems to be more mature and still maintained. Code is very clean and easy to read. Also, NN training part makes the most sense for me in this strat— this one uses nice combination of indicators, instead of throwing random price numbers at network — MACD, StochRSI, candle close/high/low/open and trade count.

At the end, makes short/long decision simply by checking if prediction > this.candle.close. This strategy could benefit from adding a threshold, like most other strats have in this category.

Luke_NN

Source: https://github.com/WeiWeiOuO/Gekko-Strategies

Seems somewhat unique strat at first glance, because it uses ScragzNN indicator, but when you dig in you see standard convnetjs/SGDTrainer, learning from price only and check() logic is hardcoded +/-1 threshold vs predicted change:

if (
    this.indicators.neuralnet.result.meanAlpha > 1
) => go long

ManuNet

Source: https://github.com/Alocaly/GekkoStrategies/

Use configurable 2xRSI, 2xSMA and volume to train the network. But, instead of trying to predict price, this strat tries to classify next price action into 2 buckets, which represent big enough (threshold) price actions either up or down:

var diffVariation = (futureCandleClose — candleClose) / candleClose;
if (diffVariation > this.buyZonePercent) return 0;
if (diffVariation < -this.sellZonePercent) return 2;

In the end, check() action is to go long/short if network predicts one of the buckets with 90% chance:

if (prediction.w[0] > 0.9) return “long”;
if (prediction.w[2] > 0.9) return “short”;

NOTE: This strat showed strong variability in results due to NN randomness.

Results 480m candles

Few strats show very good and stable results here. LSTM_MACD_RSI_V3 seems to be most aggresive with highest win but also few losses. Zuki_NN seems pretty stable. neuralnet_SirTificate with good average profits. Luke_NN also worth mentioning.

Results 360m candles

LSTM_MACD_RSI_V3 looks very unstable but also has a chance to hit it big time. Luke_NN one of the best in 360m. Mounirs-ga-version-1 doing good. Neuralnet_SirTificate seems the best in this. Zuki_NN still strong candidate, most stable in this run I’d say, no losses.

Results 240m candles

Best candle size for this month. All three n8 strats dominating here, extremely stable results. LSTM_MACD_RSI_V3 falling apart on this candle size. Neuralnet_SirTificate and zuki_nn both doing good but not as well as previous.

Results 120m candles

Much worse results than 240m candles. Basically all strats are heavy negative, except for mounirs_esto and n8_v2_BB_RSI_SL, which are close to 0.

Results 60m candles

The downfall continues. Can’t point to a single strategy that is going good here.

Results 30m / 10m candles

No comments. 120m was last usable timeframe, after that it’s all down.

Summary

Overall, the trend persists from previous parts— bigger candles bring better results. NN strats did much better than I expected and better than Basic/Advanced strats from previous parts. At least for now, in this month.

Coming up next:

Backtest against October data, that should bring better understanding how much luck was involved in these results.
Showing profit numbers is good, but that’s just the end result. I want to dig deeper and show you where that comes from. In next part I will dig out the predicted price and display that on chart. Hopefully that will uncover some interesting statistics.