Skip to content

Evaluating external historical performance

With release 1.9.55.122, backtrader can now be used to evaluate the performance of an external set of orders. This can be used for example:

  • To evaluate a set of orders/trades which for which judgmental trading (i.e.: human discretional decision) was used

  • To evaluate orders created in another platform and verify the analyzers of that platform

  • And obviously in the other direction to evaluate the things returned by backtrader against well-known results from other platforms

Usage pattern

...
cerebro.adddata(mydata)
...
cerebro.add_order_history(orders, notify=True or False)
...
cerebro.run()

The obvious question here is how orders has to look like. Let’s quote the docs:

  • orders: is an iterable (ex: list, tuple, iterator, generator) in which each element will be also an iterable (with length) with the following sub-elements (2 formats are possible)

    [datetime, size, price] or [datetime, size, price, data]

    Note: it must be sorted (or produce sorted elements) by

    datetime ascending
    

    where:

    • datetime is a python date/datetime instance or a string with format YYYY-MM-DD[THH:MM:SS[.us]] where the elements in brackets are optional

    • size is an integer (positive to buy, negative to sell)

    • price is a float/integer

    • data if present can take any of the following values

      • None - The 1st data feed will be used as target

      • integer - The data with that index (insertion order in Cerebro) will be used

      • string - a data with that name, assigned for example with cerebro.addata(data, name=value), will be the target

In the case of notify:

  • notify (default: True)

    If True the 1st strategy inserted in the system will be notified of the artificial orders created following the information from each order in orders

Note

Notice how the example above is adding a data feed. Yes this is needed.

A practical example of how orders could look like

ORDER_HISTORY = (
    ('2005-02-01', 1, 2984.63),
    ('2005-03-04', -1, 3079.93),
    ...
    ('2006-12-18', 1, 4140.99),
)

An iterable with 3 elements, which could have been perfectly loaded from a CSV file.

An example

The sample below does two things:

  1. Execute a simple SMA Crossover strategy

  2. Add a history of orders which executes the same operations as the SMA CrossOver strategy

    In this 2nd case an empty strategy is added to receive order and trade notifications over notify_order and notify_trade

In both cases a set of analyzers (TimeReturn in Months and Years and a TradeAnalyzer) are loaded … and they should return the same values.

Run 1: SMA Crossover

$ ./order-history.py --plot --cerebro writer=True

Which produces a chart

!image

And some textual output (capped for brevity):

Creating Signal Strategy
2005-02-01,1,2984.63
2005-03-04,-1,3079.93
...
2006-12-01,-1,3993.03
profit 177.9000000000001
2006-12-18,1,4140.99
===============================================================================
Cerebro:
...
        - timereturn1:
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          - Params:
            - timeframe: 8
            - compression: None
            - _doprenext: True
            - data: None
            - firstopen: True
            - fund: None
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          - Analysis:
            - 2005-12-31: 0.03580099999999975
            - 2006-12-31: 0.01649448108275653
        .......................................................................
        - tradeanalyzer:
          - Params: None
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          - Analysis:
            """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
            - total:
              - total: 14
              - open: 1
              - closed: 13
            """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
            - streak:
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              - won:
                - current: 2
                - longest: 2
...

Run 2: Order history

$ ./order-history.py --plot --cerebro writer=True --order-history

Which produces a chart which seems to have no differences

!image

And some textual output (capped again for brevity):

Creating Empty Strategy
2005-02-01,1,2984.63
2005-03-04,-1,3079.93
...
        .......................................................................
        - timereturn1:
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          - Params:
            - timeframe: 8
            - compression: None
            - _doprenext: True
            - data: None
            - firstopen: True
            - fund: None
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          - Analysis:
            - 2005-12-31: 0.03580099999999975
            - 2006-12-31: 0.01649448108275653
        .......................................................................
        - tradeanalyzer:
          - Params: None
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          - Analysis:
            """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
            - total:
              - total: 14
              - open: 1
              - closed: 13
            """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
            - streak:
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              - won:
                - current: 2
                - longest: 2
...

And the values as expected match those of the reference.

Conclusion

Measuring the performance of judgmental trading can be measured for example. This is sometimes used in combination with algotrading, where the algo generates signals, but the human has the final decision on whether the signal has to translate into an actual trade.

Sample Usage

$ ./order-history.py --help
usage: order-history.py [-h] [--data0 DATA0] [--fromdate FROMDATE]
                        [--todate TODATE] [--order-history] [--cerebro kwargs]
                        [--broker kwargs] [--sizer kwargs] [--strat kwargs]
                        [--plot [kwargs]]

Order History Sample

optional arguments:
  -h, --help           show this help message and exit
  --data0 DATA0        Data to read in (default:
                       ../../datas/2005-2006-day-001.txt)
  --fromdate FROMDATE  Date[time] in YYYY-MM-DD[THH:MM:SS] format (default: )
  --todate TODATE      Date[time] in YYYY-MM-DD[THH:MM:SS] format (default: )
  --order-history      use order history (default: False)
  --cerebro kwargs     kwargs in key=value format (default: )
  --broker kwargs      kwargs in key=value format (default: )
  --sizer kwargs       kwargs in key=value format (default: )
  --strat kwargs       kwargs in key=value format (default: )
  --plot [kwargs]      kwargs in key=value format (default: )

Sample Code

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import argparse
import datetime

import backtrader as bt


ORDER_HISTORY = (
    ('2005-02-01', 1, 2984.63),
    ('2005-03-04', -1, 3079.93),
    ('2005-03-08', 1, 3113.82),
    ('2005-03-22', -1, 3040.55),
    ('2005-04-08', 1, 3092.07),
    ('2005-04-20', -1, 2957.92),
    ('2005-05-13', 1, 2991.71),
    ('2005-08-19', -1, 3284.35),
    ('2005-08-22', 1, 3328.84),
    ('2005-08-25', -1, 3293.69),
    ('2005-09-12', 1, 3361.1),
    ('2005-10-18', -1, 3356.73),
    ('2005-11-09', 1, 3361.92),
    ('2006-01-24', -1, 3544.78),
    ('2006-02-06', 1, 3678.87),
    ('2006-03-13', -1, 3801.03),
    ('2006-03-20', 1, 3833.25),
    ('2006-04-13', -1, 3777.24),
    ('2006-05-02', 1, 3839.24),
    ('2006-05-16', -1, 3711.46),
    ('2006-06-30', 1, 3592.01),
    ('2006-07-21', -1, 3580.53),
    ('2006-08-01', 1, 3687.82),
    ('2006-09-14', -1, 3809.08),
    ('2006-09-25', 1, 3815.13),
    ('2006-12-01', -1, 3993.03),
    ('2006-12-18', 1, 4140.99),
)


class SmaCross(bt.SignalStrategy):
    params = dict(sma1=10, sma2=20)

    def notify_order(self, order):
        if not order.alive():
            print(','.join(str(x) for x in
                           (self.data.num2date(order.executed.dt).date(),
                            order.executed.size * 1 if order.isbuy() else -1,
                            order.executed.price)))

    def notify_trade(self, trade):
        if trade.isclosed:
            print('profit {}'.format(trade.pnlcomm))

    def __init__(self):
        print('Creating Signal Strategy')
        sma1 = bt.ind.SMA(period=self.params.sma1)
        sma2 = bt.ind.SMA(period=self.params.sma2)
        crossover = bt.ind.CrossOver(sma1, sma2)
        self.signal_add(bt.SIGNAL_LONG, crossover)


class St(bt.Strategy):
    params = dict(
    )

    def notify_order(self, order):
        if not order.alive():
            print(','.join(str(x) for x in
                           (self.data.num2date(order.executed.dt).date(),
                            order.executed.size * 1 if order.isbuy() else -1,
                            order.executed.price)))

    def notify_trade(self, trade):
        if trade.isclosed:
            print('profit {}'.format(trade.pnlcomm))

    def __init__(self):
        print('Creating Empty Strategy')
        pass

    def next(self):
        pass


def runstrat(args=None):
    args = parse_args(args)

    cerebro = bt.Cerebro()

    # Data feed kwargs
    kwargs = dict()

    # Parse from/to-date
    dtfmt, tmfmt = '%Y-%m-%d', 'T%H:%M:%S'
    for a, d in ((getattr(args, x), x) for x in ['fromdate', 'todate']):
        if a:
            strpfmt = dtfmt + tmfmt * ('T' in a)
            kwargs[d] = datetime.datetime.strptime(a, strpfmt)

    data0 = bt.feeds.BacktraderCSVData(dataname=args.data0, **kwargs)
    cerebro.adddata(data0)

    # Broker
    cerebro.broker = bt.brokers.BackBroker(**eval('dict(' + args.broker + ')'))

    # Sizer
    cerebro.addsizer(bt.sizers.FixedSize, **eval('dict(' + args.sizer + ')'))

    # Strategy
    if not args.order_history:
        cerebro.addstrategy(SmaCross, **eval('dict(' + args.strat + ')'))
    else:
        cerebro.addstrategy(St, **eval('dict(' + args.strat + ')'))
        cerebro.add_order_history(ORDER_HISTORY, notify=True)

    cerebro.addanalyzer(bt.analyzers.TimeReturn, timeframe=bt.TimeFrame.Months)
    cerebro.addanalyzer(bt.analyzers.TimeReturn, timeframe=bt.TimeFrame.Years)
    cerebro.addanalyzer(bt.analyzers.TradeAnalyzer)

    # Execute
    cerebro.run(**eval('dict(' + args.cerebro + ')'))

    if args.plot:  # Plot if requested to
        cerebro.plot(**eval('dict(' + args.plot + ')'))


def parse_args(pargs=None):
    parser = argparse.ArgumentParser(
        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
        description=(
            'Order History Sample'
        )
    )

    parser.add_argument('--data0', default='../../datas/2005-2006-day-001.txt',
                        required=False, help='Data to read in')

    # Defaults for dates
    parser.add_argument('--fromdate', required=False, default='',
                        help='Date[time] in YYYY-MM-DD[THH:MM:SS] format')

    parser.add_argument('--todate', required=False, default='',
                        help='Date[time] in YYYY-MM-DD[THH:MM:SS] format')

    parser.add_argument('--order-history', required=False, action='store_true',
                        help='use order history')

    parser.add_argument('--cerebro', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--broker', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--sizer', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--strat', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--plot', required=False, default='',
                        nargs='?', const='{}',
                        metavar='kwargs', help='kwargs in key=value format')

    return parser.parse_args(pargs)


if __name__ == '__main__':
    runstrat()