Skip to content

Automating backtrader BackTesting

So far all backtrader examples and working samples have started from scratch creating a main Python module which loads datas, strategies, observers and prepares cash and commission schemes.

One of the goals of algorithmic trading is the automation of trading and given that bactrader is a backtesting platform intented to check trading algorithms (hence is an algotrading platform), automating the use of backtrader was an obvious goal.

Note

Aug 22, 2015

Analyzer support in bt-run.py included

The development version of backtrader now contains the bt-run.py script which automates most tasks and will be installed along backtrader as part of a regular package.

bt-run.py allows the end user to:

  • Say which datas have to be loaded

  • Set the format to load the datas

  • Specify the date range for the datas

  • Disable standard observers

  • Load one or more observers (example: DrawDown) from the built-in ones or from a python module

  • Set the cash and commission scheme parameters for the broker (commission, margin, mult)

  • Enable plotting, controlling the amount of charts and style to present the data

And finally:

  • Load a strategy (a built-in one or from a Python module)

  • Pass parameters to the loaded strategy

See below for the Usage* of the script.

Applying a User Defined Strategy

Let’s consider the following strategy which:

  • Simply loads a SimpleMovingAverage (default period 15)

  • Prints outs

  • Is in a fily with the name mymod.py

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)


import backtrader as bt
import backtrader.indicators as btind


class MyTest(bt.Strategy):
    params = (('period', 15),)

    def log(self, txt, dt=None):
        ''' Logging function fot this strategy'''
        dt = dt or self.data.datetime[0]
        if isinstance(dt, float):
            dt = bt.num2date(dt)
        print('%s, %s' % (dt.isoformat(), txt))

    def __init__(self):
        sma = btind.SMA(period=self.p.period)

    def next(self):
        ltxt = '%d, %.2f, %.2f, %.2f, %.2f, %.2f, %.2f'

        self.log(ltxt %
                 (len(self),
                  self.data.open[0], self.data.high[0],
                  self.data.low[0], self.data.close[0],
                  self.data.volume[0], self.data.openinterest[0]))

Executing the strategy with the usual testing sample is easy: easy:

./bt-run.py --csvformat btcsv \
            --data ../samples/data/sample/2006-day-001.txt \
            --strategy ./mymod.py

The chart output

image

The console output:

2006-01-20T23:59:59+00:00, 15, 3593.16, 3612.37, 3550.80, 3550.80, 0.00, 0.00
2006-01-23T23:59:59+00:00, 16, 3550.24, 3550.24, 3515.07, 3544.31, 0.00, 0.00
2006-01-24T23:59:59+00:00, 17, 3544.78, 3553.16, 3526.37, 3532.68, 0.00, 0.00
2006-01-25T23:59:59+00:00, 18, 3532.72, 3578.00, 3532.72, 3578.00, 0.00, 0.00
...
...
2006-12-22T23:59:59+00:00, 252, 4109.86, 4109.86, 4072.62, 4073.50, 0.00, 0.00
2006-12-27T23:59:59+00:00, 253, 4079.70, 4134.86, 4079.70, 4134.86, 0.00, 0.00
2006-12-28T23:59:59+00:00, 254, 4137.44, 4142.06, 4125.14, 4130.66, 0.00, 0.00
2006-12-29T23:59:59+00:00, 255, 4130.12, 4142.01, 4119.94, 4119.94, 0.00, 0.00

Same strategy but:

  • Setting the parameter period to 50

The command line:

./bt-run.py --csvformat btcsv \
            --data ../samples/data/sample/2006-day-001.txt \
            --strategy ./mymod.py \
            period 50

The chart output.

image

Using a built-in Strategy

backtrader will slowly be including sample (textbook) strategies. Along with the bt-run.py script a standard Simple Moving Average CrossOver strategy is included. The name:

  • SMA_CrossOver

  • Parameters

    • fast (default 10) period of the fast moving average

    • slow (default 30) period of the slow moving average

The strategy buys if the fast moving average crosses up the fast and sells (only if it has bought before) upon the fast moving average crossing down the slow moving average.

The code

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)


import backtrader as bt
import backtrader.indicators as btind


class SMA_CrossOver(bt.Strategy):

    params = (('fast', 10), ('slow', 30))

    def __init__(self):

        sma_fast = btind.SMA(period=self.p.fast)
        sma_slow = btind.SMA(period=self.p.slow)

        self.buysig = btind.CrossOver(sma_fast, sma_slow)

    def next(self):
        if self.position.size:
            if self.buysig < 0:
                self.sell()

        elif self.buysig > 0:
            self.buy()

Standard execution:

./bt-run.py --csvformat btcsv \
            --data ../samples/data/sample/2006-day-001.txt \
            --strategy :SMA_CrossOver

Notice the ‘:’. The standard notation (see below) to load a strategy is:

  • module:stragegy

With the following rules:

  • If module is there and strategy is specified, then that strategy will be used

  • If module is there but no strategy is specified, the 1st strategy found in the module will be returned

  • If no module is specified, “strategy” is assumed to refer to a strategy in the backtrader package

The latter being our case.

The output.

image

One last example adding commission schemes, cash and changing the parameters:

./bt-run.py --csvformat btcsv \
            --data ../samples/data/sample/2006-day-001.txt \
            --cash 20000 \
            --commission 2.0 \
            --mult 10 \
            --margin 2000 \
            --strategy :SMA_CrossOver \
            fast 5 slow 20

The output.

image

We have backtested the strategy:

  • Changing the moving average periods

  • Setting a new starting cash

  • Putting a commission scheme in place for a futures-like instrument

    See the continuous variations in cash with each bar, as cash is adjusted for the futures-like instrument daily changes

Adding Analyzers

Note

Added Analyzer example

bt-run.py also supports adding Analyzers with the same syntax used for the strategies to choose between internal/external analyzers.

Example with a SharpeRatio analysis for the years 2005-2006:

./bt-run.py --csvformat btcsv \
            --data ../samples/data/sample/2005-2006-day-001.txt \
            --strategy :SMA_CrossOver \
            --analyzer :SharpeRatio

The output:

====================
== Analyzers
====================
##  sharperatio
--  sharperatio : 11.6473326097

Good strategy!!! (Pure luck for the example actually which also bears no commissions)

The chart (which simply shows the Analyzer is not in the plot, because Analyzers cannot be plotted, they aren’t lines objects)

image

Usage of the script

Directly from the script:

$ ./bt-run.py --help
usage: bt-run.py [-h] --data DATA
                 [--csvformat {yahoocsv_unreversed,vchart,sierracsv,yahoocsv,vchartcsv,btcsv}]
                 [--fromdate FROMDATE] [--todate TODATE] --strategy STRATEGY
                 [--nostdstats] [--observer OBSERVERS] [--analyzer ANALYZERS]
                 [--cash CASH] [--commission COMMISSION] [--margin MARGIN]
                 [--mult MULT] [--noplot] [--plotstyle {bar,line,candle}]
                 [--plotfigs PLOTFIGS]
                 ...

Backtrader Run Script

positional arguments:
  args                  args to pass to the loaded strategy

optional arguments:
  -h, --help            show this help message and exit

Data options:
  --data DATA, -d DATA  Data files to be added to the system
  --csvformat {yahoocsv_unreversed,vchart,sierracsv,yahoocsv,vchartcsv,btcsv}, -c {yahoocsv_unreversed,vchart,sierracsv,yahoocsv,vchartcsv,btcsv}
                        CSV Format
  --fromdate FROMDATE, -f FROMDATE
                        Starting date in YYYY-MM-DD[THH:MM:SS] format
  --todate TODATE, -t TODATE
                        Ending date in YYYY-MM-DD[THH:MM:SS] format

Strategy options:
  --strategy STRATEGY, -st STRATEGY
                        Module and strategy to load with format
                        module_path:strategy_name. module_path:strategy_name
                        will load strategy_name from the given module_path
                        module_path will load the module and return the first
                        available strategy in the module :strategy_name will
                        load the given strategy from the set of built-in
                        strategies

Observers and statistics:
  --nostdstats          Disable the standard statistics observers
  --observer OBSERVERS, -ob OBSERVERS
                        This option can be specified multiple times Module and
                        observer to load with format
                        module_path:observer_name. module_path:observer_name
                        will load observer_name from the given module_path
                        module_path will load the module and return all
                        available observers in the module :observer_name will
                        load the given strategy from the set of built-in
                        strategies

Analyzers:
  --analyzer ANALYZERS, -an ANALYZERS
                        This option can be specified multiple times Module and
                        analyzer to load with format
                        module_path:analzyer_name. module_path:analyzer_name
                        will load observer_name from the given module_path
                        module_path will load the module and return all
                        available analyzers in the module :anaylzer_name will
                        load the given strategy from the set of built-in
                        strategies

Cash and Commission Scheme Args:
  --cash CASH, -cash CASH
                        Cash to set to the broker
  --commission COMMISSION, -comm COMMISSION
                        Commission value to set
  --margin MARGIN, -marg MARGIN
                        Margin type to set
  --mult MULT, -mul MULT
                        Multiplier to use

Plotting options:
  --noplot, -np         Do not plot the read data
  --plotstyle {bar,line,candle}, -ps {bar,line,candle}
                        Plot style for the input data
  --plotfigs PLOTFIGS, -pn PLOTFIGS
                        Plot using n figures

And the code:

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import argparse
import datetime
import inspect
import itertools
import random
import string
import sys

import backtrader as bt
import backtrader.feeds as btfeeds
import backtrader.indicators as btinds
import backtrader.observers as btobs
import backtrader.strategies as btstrats
import backtrader.analyzers as btanalyzers


DATAFORMATS = dict(
    btcsv=btfeeds.BacktraderCSVData,
    vchartcsv=btfeeds.VChartCSVData,
    vchart=btfeeds.VChartData,
    sierracsv=btfeeds.SierraChartCSVData,
    yahoocsv=btfeeds.YahooFinanceCSVData,
    yahoocsv_unreversed=btfeeds.YahooFinanceCSVData
)


def runstrat():
    args = parse_args()

    stdstats = not args.nostdstats

    cerebro = bt.Cerebro(stdstats=stdstats)

    for data in getdatas(args):
        cerebro.adddata(data)

    # Prepare a dictionary of extra args passed to push them to the strategy

    # pack them in pairs
    packedargs = itertools.izip_longest(*[iter(args.args)] * 2, fillvalue='')

    # prepare a string for evaluation, eval and store the result
    evalargs = 'dict('
    for key, value in packedargs:
        evalargs += key + '=' + value + ','
    evalargs += ')'
    stratkwargs = eval(evalargs)

    # Get the strategy and add it with any arguments
    strat = getstrategy(args)
    cerebro.addstrategy(strat, **stratkwargs)

    obs = getobservers(args)
    for ob in obs:
        cerebro.addobserver(ob)

    ans = getanalyzers(args)
    for an in ans:
        cerebro.addanalyzer(an)

    setbroker(args, cerebro)

    runsts = cerebro.run()
    runst = runsts[0]  # single strategy and no optimization

    if runst.analyzers:
        print('====================')
        print('== Analyzers')
        print('====================')
        for name, analyzer in runst.analyzers.getitems():
            print('## ', name)
            analysis = analyzer.get_analysis()
            for key, val in analysis.items():
                print('-- ', key, ':', val)

    if not args.noplot:
        cerebro.plot(numfigs=args.plotfigs, style=args.plotstyle)


def setbroker(args, cerebro):
    broker = cerebro.getbroker()

    if args.cash is not None:
        broker.setcash(args.cash)

    commkwargs = dict()
    if args.commission is not None:
        commkwargs['commission'] = args.commission
    if args.margin is not None:
        commkwargs['margin'] = args.margin
    if args.mult is not None:
        commkwargs['mult'] = args.mult

    if commkwargs:
        broker.setcommission(**commkwargs)


def getdatas(args):
    # Get the data feed class from the global dictionary
    dfcls = DATAFORMATS[args.csvformat]

    # Prepare some args
    dfkwargs = dict()
    if args.csvformat == 'yahoo_unreversed':
        dfkwargs['reverse'] = True

    fmtstr = '%Y-%m-%d'
    if args.fromdate:
        dtsplit = args.fromdate.split('T')
        if len(dtsplit) > 1:
            fmtstr += 'T%H:%M:%S'

        fromdate = datetime.datetime.strptime(args.fromdate, fmtstr)
        dfkwargs['fromdate'] = fromdate

    fmtstr = '%Y-%m-%d'
    if args.todate:
        dtsplit = args.todate.split('T')
        if len(dtsplit) > 1:
            fmtstr += 'T%H:%M:%S'
        todate = datetime.datetime.strptime(args.todate, fmtstr)
        dfkwargs['todate'] = todate

    datas = list()
    for dname in args.data:
        dfkwargs['dataname'] = dname
        data = dfcls(**dfkwargs)
        datas.append(data)

    return datas


def getmodclasses(mod, clstype, clsname=None):
    clsmembers = inspect.getmembers(mod, inspect.isclass)

    clslist = list()
    for name, cls in clsmembers:
        if not issubclass(cls, clstype):
            continue

        if clsname:
            if clsname == name:
                clslist.append(cls)
                break
        else:
            clslist.append(cls)

    return clslist


def loadmodule(modpath, modname=''):
    # generate a random name for the module
    if not modname:
        chars = string.ascii_uppercase + string.digits
        modname = ''.join(random.choice(chars) for _ in range(10))

    version = (sys.version_info[0], sys.version_info[1])

    if version < (3, 3):
        mod, e = loadmodule2(modpath, modname)
    else:
        mod, e = loadmodule3(modpath, modname)

    return mod, e


def loadmodule2(modpath, modname):
    import imp

    try:
        mod = imp.load_source(modname, modpath)
    except Exception, e:
        return (None, e)

    return (mod, None)


def loadmodule3(modpath, modname):
    import importlib.machinery

    try:
        loader = importlib.machinery.SourceFileLoader(modname, modpath)
        mod = loader.load_module()
    except Exception, e:
        return (None, e)

    return (mod, None)


def getstrategy(args):
    sttokens = args.strategy.split(':')

    if len(sttokens) == 1:
        modpath = sttokens[0]
        stname = None
    else:
        modpath, stname = sttokens

    if modpath:
        mod, e = loadmodule(modpath)

        if not mod:
            print('')
            print('Failed to load module %s:' % modpath, e)
            sys.exit(1)
    else:
        mod = btstrats

    strats = getmodclasses(mod=mod, clstype=bt.Strategy, clsname=stname)

    if not strats:
        print('No strategy %s / module %s' % (str(stname), modpath))
        sys.exit(1)

    return strats[0]


def getanalyzers(args):
    analyzers = list()
    for anspec in args.analyzers or []:

        tokens = anspec.split(':')

        if len(tokens) == 1:
            modpath = tokens[0]
            name = None
        else:
            modpath, name = tokens

        if modpath:
            mod, e = loadmodule(modpath)

            if not mod:
                print('')
                print('Failed to load module %s:' % modpath, e)
                sys.exit(1)
        else:
            mod = btanalyzers

        loaded = getmodclasses(mod=mod, clstype=bt.Analyzer, clsname=name)

        if not loaded:
            print('No analyzer %s / module %s' % ((str(name), modpath)))
            sys.exit(1)

        analyzers.extend(loaded)

    return analyzers


def getobservers(args):
    observers = list()
    for obspec in args.observers or []:

        tokens = obspec.split(':')

        if len(tokens) == 1:
            modpath = tokens[0]
            name = None
        else:
            modpath, name = tokens

        if modpath:
            mod, e = loadmodule(modpath)

            if not mod:
                print('')
                print('Failed to load module %s:' % modpath, e)
                sys.exit(1)
        else:
            mod = btobs

        loaded = getmodclasses(mod=mod, clstype=bt.Observer, clsname=name)

        if not loaded:
            print('No observer %s / module %s' % ((str(name), modpath)))
            sys.exit(1)

        observers.extend(loaded)

    return observers


def parse_args():
    parser = argparse.ArgumentParser(
        description='Backtrader Run Script')

    group = parser.add_argument_group(title='Data options')
    # Data options
    group.add_argument('--data', '-d', action='append', required=True,
                       help='Data files to be added to the system')

    datakeys = list(DATAFORMATS.keys())
    group.add_argument('--csvformat', '-c', required=False,
                       default='btcsv', choices=datakeys,
                       help='CSV Format')

    group.add_argument('--fromdate', '-f', required=False, default=None,
                       help='Starting date in YYYY-MM-DD[THH:MM:SS] format')

    group.add_argument('--todate', '-t', required=False, default=None,
                       help='Ending date in YYYY-MM-DD[THH:MM:SS] format')

    # Module where to read the strategy from
    group = parser.add_argument_group(title='Strategy options')
    group.add_argument('--strategy', '-st', required=True,
                       help=('Module and strategy to load with format '
                             'module_path:strategy_name.\n'
                             '\n'
                             'module_path:strategy_name will load '
                             'strategy_name from the given module_path\n'
                             '\n'
                             'module_path will load the module and return '
                             'the first available strategy in the module\n'
                             '\n'
                             ':strategy_name will load the given strategy '
                             'from the set of built-in strategies'))

    # Observers
    group = parser.add_argument_group(title='Observers and statistics')
    group.add_argument('--nostdstats', action='store_true',
                       help='Disable the standard statistics observers')

    group.add_argument('--observer', '-ob', dest='observers',
                       action='append', required=False,
                       help=('This option can be specified multiple times\n'
                             '\n'
                             'Module and observer to load with format '
                             'module_path:observer_name.\n'
                             '\n'
                             'module_path:observer_name will load '
                             'observer_name from the given module_path\n'
                             '\n'
                             'module_path will load the module and return '
                             'all available observers in the module\n'
                             '\n'
                             ':observer_name will load the given strategy '
                             'from the set of built-in strategies'))

    # Anaylzers
    group = parser.add_argument_group(title='Analyzers')
    group.add_argument('--analyzer', '-an', dest='analyzers',
                       action='append', required=False,
                       help=('This option can be specified multiple times\n'
                             '\n'
                             'Module and analyzer to load with format '
                             'module_path:analzyer_name.\n'
                             '\n'
                             'module_path:analyzer_name will load '
                             'observer_name from the given module_path\n'
                             '\n'
                             'module_path will load the module and return '
                             'all available analyzers in the module\n'
                             '\n'
                             ':anaylzer_name will load the given strategy '
                             'from the set of built-in strategies'))

    # Broker/Commissions
    group = parser.add_argument_group(title='Cash and Commission Scheme Args')
    group.add_argument('--cash', '-cash', required=False, type=float,
                       help='Cash to set to the broker')
    group.add_argument('--commission', '-comm', required=False, type=float,
                       help='Commission value to set')
    group.add_argument('--margin', '-marg', required=False, type=float,
                       help='Margin type to set')

    group.add_argument('--mult', '-mul', required=False, type=float,
                       help='Multiplier to use')

    # Plot options
    group = parser.add_argument_group(title='Plotting options')
    group.add_argument('--noplot', '-np', action='store_true', required=False,
                       help='Do not plot the read data')

    group.add_argument('--plotstyle', '-ps', required=False, default='bar',
                       choices=['bar', 'line', 'candle'],
                       help='Plot style for the input data')

    group.add_argument('--plotfigs', '-pn', required=False, default=1,
                       type=int, help='Plot using n figures')

    # Extra arguments
    parser.add_argument('args', nargs=argparse.REMAINDER,
                        help='args to pass to the loaded strategy')

    return parser.parse_args()


if __name__ == '__main__':
    runstrat()