Evaluating external historical performance
With release 1.9.55.122, backtrader can now be used to evaluate the
performance of an external set of orders. This can be used for example:
-
To evaluate a set of orders/trades which for which judgmental trading (i.e.: human discretional decision) was used
-
To evaluate orders created in another platform and verify the analyzers of that platform
-
And obviously in the other direction to evaluate the things returned by backtrader against well-known results from other platforms
Usage pattern
...
cerebro.adddata(mydata)
...
cerebro.add_order_history(orders, notify=True or False)
...
cerebro.run()
The obvious question here is how orders has to look like. Let’s quote the
docs:
-
orders: is an iterable (ex: list, tuple, iterator, generator) in which each element will be also an iterable (with length) with the following sub-elements (2 formats are possible)[datetime, size, price]or[datetime, size, price, data]Note: it must be sorted (or produce sorted elements) by
datetime ascendingwhere:
-
datetimeis a pythondate/datetimeinstance or a string with format YYYY-MM-DD[THH:MM:SS[.us]] where the elements in brackets are optional -
sizeis an integer (positive to buy, negative to sell) -
priceis a float/integer -
dataif present can take any of the following values-
None - The 1st data feed will be used as target
-
integer - The data with that index (insertion order in Cerebro) will be used
-
string - a data with that name, assigned for example with
cerebro.addata(data, name=value), will be the target
-
-
In the case of notify:
-
notify(default: True)If
Truethe 1st strategy inserted in the system will be notified of the artificial orders created following the information from each order inorders
Note
Notice how the example above is adding a data feed. Yes this is needed.
A practical example of how orders could look like
ORDER_HISTORY = (
('2005-02-01', 1, 2984.63),
('2005-03-04', -1, 3079.93),
...
('2006-12-18', 1, 4140.99),
)
An iterable with 3 elements, which could have been perfectly loaded from a CSV file.
An example
The sample below does two things:
-
Execute a simple SMA Crossover strategy
-
Add a history of orders which executes the same operations as the SMA CrossOver strategy
In this 2nd case an empty strategy is added to receive order and trade notifications over
notify_orderandnotify_trade
In both cases a set of analyzers (TimeReturn in Months and Years and
a TradeAnalyzer) are loaded … and they should return the same values.
Run 1: SMA Crossover
$ ./order-history.py --plot --cerebro writer=True
Which produces a chart
And some textual output (capped for brevity):
Creating Signal Strategy
2005-02-01,1,2984.63
2005-03-04,-1,3079.93
...
2006-12-01,-1,3993.03
profit 177.9000000000001
2006-12-18,1,4140.99
===============================================================================
Cerebro:
...
- timereturn1:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Params:
- timeframe: 8
- compression: None
- _doprenext: True
- data: None
- firstopen: True
- fund: None
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Analysis:
- 2005-12-31: 0.03580099999999975
- 2006-12-31: 0.01649448108275653
.......................................................................
- tradeanalyzer:
- Params: None
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Analysis:
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
- total:
- total: 14
- open: 1
- closed: 13
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
- streak:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- won:
- current: 2
- longest: 2
...
Run 2: Order history
$ ./order-history.py --plot --cerebro writer=True --order-history
Which produces a chart which seems to have no differences
And some textual output (capped again for brevity):
Creating Empty Strategy
2005-02-01,1,2984.63
2005-03-04,-1,3079.93
...
.......................................................................
- timereturn1:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Params:
- timeframe: 8
- compression: None
- _doprenext: True
- data: None
- firstopen: True
- fund: None
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Analysis:
- 2005-12-31: 0.03580099999999975
- 2006-12-31: 0.01649448108275653
.......................................................................
- tradeanalyzer:
- Params: None
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Analysis:
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
- total:
- total: 14
- open: 1
- closed: 13
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
- streak:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- won:
- current: 2
- longest: 2
...
And the values as expected match those of the reference.
Conclusion
Measuring the performance of judgmental trading can be measured for example. This is sometimes used in combination with algotrading, where the algo generates signals, but the human has the final decision on whether the signal has to translate into an actual trade.
Sample Usage
$ ./order-history.py --help
usage: order-history.py [-h] [--data0 DATA0] [--fromdate FROMDATE]
[--todate TODATE] [--order-history] [--cerebro kwargs]
[--broker kwargs] [--sizer kwargs] [--strat kwargs]
[--plot [kwargs]]
Order History Sample
optional arguments:
-h, --help show this help message and exit
--data0 DATA0 Data to read in (default:
../../datas/2005-2006-day-001.txt)
--fromdate FROMDATE Date[time] in YYYY-MM-DD[THH:MM:SS] format (default: )
--todate TODATE Date[time] in YYYY-MM-DD[THH:MM:SS] format (default: )
--order-history use order history (default: False)
--cerebro kwargs kwargs in key=value format (default: )
--broker kwargs kwargs in key=value format (default: )
--sizer kwargs kwargs in key=value format (default: )
--strat kwargs kwargs in key=value format (default: )
--plot [kwargs] kwargs in key=value format (default: )
Sample Code
from __future__ import (absolute_import, division, print_function,
unicode_literals)
import argparse
import datetime
import backtrader as bt
ORDER_HISTORY = (
('2005-02-01', 1, 2984.63),
('2005-03-04', -1, 3079.93),
('2005-03-08', 1, 3113.82),
('2005-03-22', -1, 3040.55),
('2005-04-08', 1, 3092.07),
('2005-04-20', -1, 2957.92),
('2005-05-13', 1, 2991.71),
('2005-08-19', -1, 3284.35),
('2005-08-22', 1, 3328.84),
('2005-08-25', -1, 3293.69),
('2005-09-12', 1, 3361.1),
('2005-10-18', -1, 3356.73),
('2005-11-09', 1, 3361.92),
('2006-01-24', -1, 3544.78),
('2006-02-06', 1, 3678.87),
('2006-03-13', -1, 3801.03),
('2006-03-20', 1, 3833.25),
('2006-04-13', -1, 3777.24),
('2006-05-02', 1, 3839.24),
('2006-05-16', -1, 3711.46),
('2006-06-30', 1, 3592.01),
('2006-07-21', -1, 3580.53),
('2006-08-01', 1, 3687.82),
('2006-09-14', -1, 3809.08),
('2006-09-25', 1, 3815.13),
('2006-12-01', -1, 3993.03),
('2006-12-18', 1, 4140.99),
)
class SmaCross(bt.SignalStrategy):
params = dict(sma1=10, sma2=20)
def notify_order(self, order):
if not order.alive():
print(','.join(str(x) for x in
(self.data.num2date(order.executed.dt).date(),
order.executed.size * 1 if order.isbuy() else -1,
order.executed.price)))
def notify_trade(self, trade):
if trade.isclosed:
print('profit {}'.format(trade.pnlcomm))
def __init__(self):
print('Creating Signal Strategy')
sma1 = bt.ind.SMA(period=self.params.sma1)
sma2 = bt.ind.SMA(period=self.params.sma2)
crossover = bt.ind.CrossOver(sma1, sma2)
self.signal_add(bt.SIGNAL_LONG, crossover)
class St(bt.Strategy):
params = dict(
)
def notify_order(self, order):
if not order.alive():
print(','.join(str(x) for x in
(self.data.num2date(order.executed.dt).date(),
order.executed.size * 1 if order.isbuy() else -1,
order.executed.price)))
def notify_trade(self, trade):
if trade.isclosed:
print('profit {}'.format(trade.pnlcomm))
def __init__(self):
print('Creating Empty Strategy')
pass
def next(self):
pass
def runstrat(args=None):
args = parse_args(args)
cerebro = bt.Cerebro()
# Data feed kwargs
kwargs = dict()
# Parse from/to-date
dtfmt, tmfmt = '%Y-%m-%d', 'T%H:%M:%S'
for a, d in ((getattr(args, x), x) for x in ['fromdate', 'todate']):
if a:
strpfmt = dtfmt + tmfmt * ('T' in a)
kwargs[d] = datetime.datetime.strptime(a, strpfmt)
data0 = bt.feeds.BacktraderCSVData(dataname=args.data0, **kwargs)
cerebro.adddata(data0)
# Broker
cerebro.broker = bt.brokers.BackBroker(**eval('dict(' + args.broker + ')'))
# Sizer
cerebro.addsizer(bt.sizers.FixedSize, **eval('dict(' + args.sizer + ')'))
# Strategy
if not args.order_history:
cerebro.addstrategy(SmaCross, **eval('dict(' + args.strat + ')'))
else:
cerebro.addstrategy(St, **eval('dict(' + args.strat + ')'))
cerebro.add_order_history(ORDER_HISTORY, notify=True)
cerebro.addanalyzer(bt.analyzers.TimeReturn, timeframe=bt.TimeFrame.Months)
cerebro.addanalyzer(bt.analyzers.TimeReturn, timeframe=bt.TimeFrame.Years)
cerebro.addanalyzer(bt.analyzers.TradeAnalyzer)
# Execute
cerebro.run(**eval('dict(' + args.cerebro + ')'))
if args.plot: # Plot if requested to
cerebro.plot(**eval('dict(' + args.plot + ')'))
def parse_args(pargs=None):
parser = argparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
description=(
'Order History Sample'
)
)
parser.add_argument('--data0', default='../../datas/2005-2006-day-001.txt',
required=False, help='Data to read in')
# Defaults for dates
parser.add_argument('--fromdate', required=False, default='',
help='Date[time] in YYYY-MM-DD[THH:MM:SS] format')
parser.add_argument('--todate', required=False, default='',
help='Date[time] in YYYY-MM-DD[THH:MM:SS] format')
parser.add_argument('--order-history', required=False, action='store_true',
help='use order history')
parser.add_argument('--cerebro', required=False, default='',
metavar='kwargs', help='kwargs in key=value format')
parser.add_argument('--broker', required=False, default='',
metavar='kwargs', help='kwargs in key=value format')
parser.add_argument('--sizer', required=False, default='',
metavar='kwargs', help='kwargs in key=value format')
parser.add_argument('--strat', required=False, default='',
metavar='kwargs', help='kwargs in key=value format')
parser.add_argument('--plot', required=False, default='',
nargs='?', const='{}',
metavar='kwargs', help='kwargs in key=value format')
return parser.parse_args(pargs)
if __name__ == '__main__':
runstrat()