Writers - Write it down
With the 1.1.7.88 release backtrader gets a new addition: writers
This is probably long due and should have been there and the discussion in Issue #14 should also have kicked started the development.
But better late than never.
The Writer implementation tries to remain in line with the other objects in
the backtrader environment
-
Get added over Cerebro
-
Provide the most resonable defaults
-
Don’t force the user to do much
Of course and of much more importance is to understand what the writer actually writes. And that is:
- A CSV output of
- `datas` added to the system (can be switched off) - `strategies` (a Strategy can have named lines) - `indicators` inside the strategies (only 1st level) - `observers` inside the strategies (only 1st level) Which `indicators` and `observers` output data to the CSV stream is controlled by the attribute: `csv` in each instance The defaults are: - Observers have `csv = True` - Indicators have `csv = False` The value can be overriden for any instance created inside a strategy
Once the backtesting phase is over, Writers add a new section for the
Cerebro instance and the following subsections are added:
-
Properties of
datasin the system (name, compression, timeframe) -
Properties of
strategiesin the system (lines, params)-
Properties of
indicatorsin the strategies (lines, params) -
Properties of
observersin the strategies (lines, params) -
Analyzers with the following properties
-
Params
-
Analysis
-
With all that in mind, an example may be the easiest way to show the power (or
weakness) or the writers.
But before how to add them to cerebro.
-
Using the
writerparam tocerebro:cerebro = bt.Cerebro(writer=True)This creates a default instance.
-
Specific addition:
cerebro = bt.Cerebro() cerebro.addwriter(bt.WriterFile, csv=False)Adds (right now the only writer) a
WriterFileclass to the writer list to be later instantiated withcsv=False(no csv stream will be generated in the output.
The long due example with a long-short strategy (see below for the full code) using a Close-SMA crossover as the signal by executing:
$ ./writer-test.py
The chart:
With the following output:
===============================================================================
Cerebro:
-----------------------------------------------------------------------------
- Datas:
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
- Data0:
- Name: 2006-day-001
- Timeframe: Days
- Compression: 1
-----------------------------------------------------------------------------
- Strategies:
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
- LongShortStrategy:
*************************************************************************
- Params:
- csvcross: False
- printout: False
- onlylong: False
- stake: 1
- period: 15
*************************************************************************
- Indicators:
.......................................................................
- SMA:
- Lines: sma
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Params:
- period: 15
.......................................................................
- CrossOver:
- Lines: crossover
- Params: None
*************************************************************************
- Observers:
.......................................................................
- Broker:
- Lines: cash, value
- Params: None
.......................................................................
- BuySell:
- Lines: buy, sell
- Params: None
.......................................................................
- Trades:
- Lines: pnlplus, pnlminus
- Params: None
*************************************************************************
- Analyzers:
.......................................................................
- Value:
- Begin: 100000
- End: 100826.1
.......................................................................
- SQN:
- Params: None
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Analysis:
- sqn: 0.05
- trades: 22
After the run we have a complete summary of how the system is setup and at the end what the analzyers say. In this case the analyzers are
-
Valuewhich is a fake analyzer inside the strategy which collects the starting and ending values of the portfolio -
SQN(or SystemQualityNumber) defined by Van K. Tharp (addition tobacktrader1.1.7.88 which is telling us that it has seen 22 trades and has calculated asqnof 0.05.This is actually pretty low. We could have figured it out by looking at the small profit after a full year (luckily the system loses no money)
The test script allows us to tune the strategy to become long-only:
$ ./writer-test.py --onlylong --plot
The chart:
The output being now:
===============================================================================
Cerebro:
-----------------------------------------------------------------------------
- Datas:
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
- Data0:
- Name: 2006-day-001
- Timeframe: Days
- Compression: 1
-----------------------------------------------------------------------------
- Strategies:
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
- LongShortStrategy:
*************************************************************************
- Params:
- csvcross: False
- printout: False
- onlylong: True
- stake: 1
- period: 15
*************************************************************************
- Indicators:
.......................................................................
- SMA:
- Lines: sma
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Params:
- period: 15
.......................................................................
- CrossOver:
- Lines: crossover
- Params: None
*************************************************************************
- Observers:
.......................................................................
- Broker:
- Lines: cash, value
- Params: None
.......................................................................
- BuySell:
- Lines: buy, sell
- Params: None
.......................................................................
- Trades:
- Lines: pnlplus, pnlminus
- Params: None
*************************************************************************
- Analyzers:
.......................................................................
- Value:
- Begin: 100000
- End: 102795.0
.......................................................................
- SQN:
- Params: None
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Analysis:
- sqn: 0.91
- trades: 11
The changes in the “params” to the strategy can be seen (onlylong has turned to True) and the Analyzers tell a different story:
-
Ending value improved from 100826.1 to 102795.0
-
Trades seen by SQN down to 11 from 22
-
The SQN score grows from 0.05 to 0.91 which is much much better
But still there is no CSV output to be seen. Let’s run the script to turn it on:
$ ./writer-test.py --onlylong --writercsv
With renewed output:
===============================================================================
Id,2006-day-001,len,datetime,open,high,low,close,volume,openinterest,LongShortStrategy,len,Broker,len,cash,value,Buy
Sell,len,buy,sell,Trades,len,pnlplus,pnlminus
1,2006-day-001,1,2006-01-02 23:59:59+00:00,3578.73,3605.95,3578.73,3604.33,0.0,0.0,LongShortStrategy,1,Broker,1,1000
00.0,100000.0,BuySell,1,,,Trades,1,,
2,2006-day-001,2,2006-01-03 23:59:59+00:00,3604.08,3638.42,3601.84,3614.34,0.0,0.0,LongShortStrategy,2,Broker,2,1000
00.0,100000.0,BuySell,2,,,Trades,2,,
...
...
...
255,2006-day-001,255,2006-12-29 23:59:59+00:00,4130.12,4142.01,4119.94,4119.94,0.0,0.0,LongShortStrategy,255,Broker,255,100795.0,102795.0,BuySell,255,,,Trades,255,,
===============================================================================
Cerebro:
-----------------------------------------------------------------------------
...
...
We can skip most of the csv stream and the already seen summaries. The CSV stream has printe out the following
-
A section line separator at the beginning
-
A header row
-
The corresponding data
Note how each object gets its “length” printed. Although in this case it doesn’t offer much information, it will if multi-timeframe datas are used or data is replayed.
The writer defaults do the following:
-
No indicators are printed (neither the Simple Moving Average nor the CrossOver)
-
The obsevers are printed
Let’s run the script with an additional parameter to have the CrossOver indicator added to the CSV stream:
$ ./writer-test.py --onlylong --writercsv --csvcross
The output:
===============================================================================
Id,2006-day-001,len,datetime,open,high,low,close,volume,openinterest,LongShortStrategy,len,CrossOver,len,crossover,B
roker,len,cash,value,BuySell,len,buy,sell,Trades,len,pnlplus,pnlminus
1,2006-day-001,1,2006-01-02 23:59:59+00:00,3578.73,3605.95,3578.73,3604.33,0.0,0.0,LongShortStrategy,1,CrossOver,1,,
Broker,1,100000.0,100000.0,BuySell,1,,,Trades,1,,
...
...
This has shown some of the powers of the writers. Further documentation of the class is still a to-do.
Meanwhile the execution possibilities and code used for the example.
Usage:
$ ./writer-test.py --help
usage: writer-test.py [-h] [--data DATA] [--fromdate FROMDATE]
[--todate TODATE] [--period PERIOD] [--onlylong]
[--writercsv] [--csvcross] [--cash CASH] [--comm COMM]
[--mult MULT] [--margin MARGIN] [--stake STAKE] [--plot]
[--numfigs NUMFIGS]
MultiData Strategy
optional arguments:
-h, --help show this help message and exit
--data DATA, -d DATA data to add to the system
--fromdate FROMDATE, -f FROMDATE
Starting date in YYYY-MM-DD format
--todate TODATE, -t TODATE
Starting date in YYYY-MM-DD format
--period PERIOD Period to apply to the Simple Moving Average
--onlylong, -ol Do only long operations
--writercsv, -wcsv Tell the writer to produce a csv stream
--csvcross Output the CrossOver signals to CSV
--cash CASH Starting Cash
--comm COMM Commission for operation
--mult MULT Multiplier for futures
--margin MARGIN Margin for each future
--stake STAKE Stake to apply in each operation
--plot, -p Plot the read data
--numfigs NUMFIGS, -n NUMFIGS
Plot using numfigs figures
And the test script.
from __future__ import (absolute_import, division, print_function,
unicode_literals)
import argparse
import datetime
# The above could be sent to an independent module
import backtrader as bt
import backtrader.feeds as btfeeds
import backtrader.indicators as btind
from backtrader.analyzers import SQN
class LongShortStrategy(bt.Strategy):
'''This strategy buys/sells upong the close price crossing
upwards/downwards a Simple Moving Average.
It can be a long-only strategy by setting the param "onlylong" to True
'''
params = dict(
period=15,
stake=1,
printout=False,
onlylong=False,
csvcross=False,
)
def start(self):
pass
def stop(self):
pass
def log(self, txt, dt=None):
if self.p.printout:
dt = dt or self.data.datetime[0]
dt = bt.num2date(dt)
print('%s, %s' % (dt.isoformat(), txt))
def __init__(self):
# To control operation entries
self.orderid = None
# Create SMA on 2nd data
sma = btind.MovAv.SMA(self.data, period=self.p.period)
# Create a CrossOver Signal from close an moving average
self.signal = btind.CrossOver(self.data.close, sma)
self.signal.csv = self.p.csvcross
def next(self):
if self.orderid:
return # if an order is active, no new orders are allowed
if self.signal > 0.0: # cross upwards
if self.position:
self.log('CLOSE SHORT , %.2f' % self.data.close[0])
self.close()
self.log('BUY CREATE , %.2f' % self.data.close[0])
self.buy(size=self.p.stake)
elif self.signal < 0.0:
if self.position:
self.log('CLOSE LONG , %.2f' % self.data.close[0])
self.close()
if not self.p.onlylong:
self.log('SELL CREATE , %.2f' % self.data.close[0])
self.sell(size=self.p.stake)
def notify_order(self, order):
if order.status in [bt.Order.Submitted, bt.Order.Accepted]:
return # Await further notifications
if order.status == order.Completed:
if order.isbuy():
buytxt = 'BUY COMPLETE, %.2f' % order.executed.price
self.log(buytxt, order.executed.dt)
else:
selltxt = 'SELL COMPLETE, %.2f' % order.executed.price
self.log(selltxt, order.executed.dt)
elif order.status in [order.Expired, order.Canceled, order.Margin]:
self.log('%s ,' % order.Status[order.status])
pass # Simply log
# Allow new orders
self.orderid = None
def notify_trade(self, trade):
if trade.isclosed:
self.log('TRADE PROFIT, GROSS %.2f, NET %.2f' %
(trade.pnl, trade.pnlcomm))
elif trade.justopened:
self.log('TRADE OPENED, SIZE %2d' % trade.size)
def runstrategy():
args = parse_args()
# Create a cerebro
cerebro = bt.Cerebro()
# Get the dates from the args
fromdate = datetime.datetime.strptime(args.fromdate, '%Y-%m-%d')
todate = datetime.datetime.strptime(args.todate, '%Y-%m-%d')
# Create the 1st data
data = btfeeds.BacktraderCSVData(
dataname=args.data,
fromdate=fromdate,
todate=todate)
# Add the 1st data to cerebro
cerebro.adddata(data)
# Add the strategy
cerebro.addstrategy(LongShortStrategy,
period=args.period,
onlylong=args.onlylong,
csvcross=args.csvcross,
stake=args.stake)
# Add the commission - only stocks like a for each operation
cerebro.broker.setcash(args.cash)
# Add the commission - only stocks like a for each operation
cerebro.broker.setcommission(commission=args.comm,
mult=args.mult,
margin=args.margin)
cerebro.addanalyzer(SQN)
cerebro.addwriter(bt.WriterFile, csv=args.writercsv, rounding=2)
# And run it
cerebro.run()
# Plot if requested
if args.plot:
cerebro.plot(numfigs=args.numfigs, volume=False, zdown=False)
def parse_args():
parser = argparse.ArgumentParser(description='MultiData Strategy')
parser.add_argument('--data', '-d',
default='../../datas/2006-day-001.txt',
help='data to add to the system')
parser.add_argument('--fromdate', '-f',
default='2006-01-01',
help='Starting date in YYYY-MM-DD format')
parser.add_argument('--todate', '-t',
default='2006-12-31',
help='Starting date in YYYY-MM-DD format')
parser.add_argument('--period', default=15, type=int,
help='Period to apply to the Simple Moving Average')
parser.add_argument('--onlylong', '-ol', action='store_true',
help='Do only long operations')
parser.add_argument('--writercsv', '-wcsv', action='store_true',
help='Tell the writer to produce a csv stream')
parser.add_argument('--csvcross', action='store_true',
help='Output the CrossOver signals to CSV')
parser.add_argument('--cash', default=100000, type=int,
help='Starting Cash')
parser.add_argument('--comm', default=2, type=float,
help='Commission for operation')
parser.add_argument('--mult', default=10, type=int,
help='Multiplier for futures')
parser.add_argument('--margin', default=2000.0, type=float,
help='Margin for each future')
parser.add_argument('--stake', default=1, type=int,
help='Stake to apply in each operation')
parser.add_argument('--plot', '-p', action='store_true',
help='Plot the read data')
parser.add_argument('--numfigs', '-n', default=1,
help='Plot using numfigs figures')
return parser.parse_args()
if __name__ == '__main__':
runstrategy()