Cross-Backtesting Pitfalls

Somethings that tends to repeat itself in the backtrader community is that a user explains the will to replicate the backtesting results attained in, for example, TradingView , quite popular these days, or some other backtesting platform.

Without really knowing the language used in TradingView, named Pinescript, and having null exposure to the internal of the backtesting engine, there is still a way to let users know, that coding across platform has to be taken with pinch of salt.

Indicators: Not always faithful to the sources

When a new indicator is being implemented for backtrader, either directly for the distribution or as a snippet for the website, a lot of emphasis is taken in respecting the original definition. The RSI is a good example.

• Welles Wilder designed the RSI using the Modified Moving Average (aka Smoothed Moving Average, see Wikipedia - Modified Moving Average )

• In spite of which, a number of platforms give the users something called RSI but using a classic Exponential Moving Average and not what the book says.

• Given that both averages are exponential, the differences are not huge, but it is NOT what Welles Wilder defined. It may still be useful, it may even be better, but it is NOT the RSI. And the documentation (when available) fails to mentions that.

The default configuration for the RSI in backtrader is to use the MMA to be faithful to the sources, but which moving average to use is a parameter which can be changed via subclassing or during run-time instantiation to use the EMA or even a Simple Moving Average.

An example: The Donchian Channels

The Wikipedia definition: Wikipedia - Donchian Channel ). It is just text and it makes no mention of using channel breakouts as a trading signal.

Another two definitions:

• StockCharts - School - Price Channels ).

• IncredibleCharts - Donchian Channels )

These two references explicitly state, that the data for the calculation of the channel does not include the current bar, because if it did ... breakouts would not be reflected. Here is a sample chart from StockCharts

Going now to TradingView. First the link

• TradingView - Wiki - Donchian Channels )

And a chart from that page.

Even Investopedia uses a chart from TradingView that shows no breakout. Here: Investopedia - Donchian Channels - https://www.investopedia.com/terms/d/donchianchannels.asp

As some people would put it ... Blistering Barnacles!!!!. Because there are no breakouts to be seen in the TradingView charts. This means that the implementation of the indicator is using the current price bar for the calculation of the channels.

The Donchian Channels in backtrader

There is no DonchianChannels implementation in the standard backtrader distribution, but it can be quickly crafted. A parameter will be the deciding factor as to whether the current bar will be used for the channel calculation or not.

class DonchianChannels(bt.Indicator):
    '''
    Params Note:
      - ``lookback`` (default: -1)

        If `-1`, the bars to consider will start 1 bar in the past and the
        current high/low may break through the channel.

        If `0`, the current prices will be considered for the Donchian
        Channel. This means that the price will **NEVER** break through the
        upper/lower channel bands.
    '''

    alias = ('DCH', 'DonchianChannel',)

    lines = ('dcm', 'dch', 'dcl',)  # dc middle, dc high, dc low
    params = dict(
        period=20,
        lookback=-1,  # consider current bar or not
    )

    plotinfo = dict(subplot=False)  # plot along with data
    plotlines = dict(
        dcm=dict(ls='--'),  # dashed line
        dch=dict(_samecolor=True),  # use same color as prev line (dcm)
        dcl=dict(_samecolor=True),  # use same color as prev line (dch)
    )

    def __init__(self):
        hi, lo = self.data.high, self.data.low
        if self.p.lookback:  # move backwards as needed
            hi, lo = hi(self.p.lookback), lo(self.p.lookback)

        self.l.dch = bt.ind.Highest(hi, period=self.p.period)
        self.l.dcl = bt.ind.Lowest(lo, period=self.p.period)
        self.l.dcm = (self.l.dch + self.l.dcl) / 2.0  # avg of the above

Using it with lookback=-1 a sample chart looks like this (zoomed in)

One can clearly see the breakouts, whereas there are no breakouts in the lookback=0 version.

Coding Implications

The programmer first goes to the commercial platform and implements a strategy using The Donchian Channels. Because the chart does not show breakouts, a comparison of the current price value has to be done against the previous channel value. Something as

if price0 > channel_high_1:
    sell()
elif price0 < channel_low_1:
    buy()

The current price, i.e.: price0 is being compared to the high/low channel values of 1 period ago (hence the _1 suffix)

Being a cautious programmer and unaware that the implementation of the Donchian Channels in backtrader defaults to having breakout, the code is ported and looks like this

    def __init__(self):
        self.donchian = DonchianChannels()

    def next(self):
        if self.data[0] > self.donchian.dch[-1]:
            self.sell()
        elif self.data[0] < self.donchian.dcl[-1]:
            self.buy()

Which is wrong!!! Because the breakout happens at the same moment of the comparison. The correct code:

    def __init__(self):
        self.donchian = DonchianChannels()

    def next(self):
        if self.data[0] > self.donchian.dch[0]:
            self.sell()
        elif self.data[0] < self.donchian.dcl[0]:
            self.buy()

Although this is just a small example, it shows how backtesting results may differ because an indicator has been coded with a 1 bar difference. It may not seem much, but it will for sure make a difference when the wrong trade is started.