If you have read through the Backtrader: First Script post or seen any of the other code snippets on this site, you will see that most examples work with just one data feed. Similarly, the number of indicators to be used in a strategy is well-defined in advance. This is for good reason. As the site is targeted more towards the beginner, it makes sense for each post to keep the code as simple as possible and focus only on the subject at hand. As such, this is the first post to focus on backtesting with multiple data feeds. If you want to be able to change the number of data feeds without changing the code, dynamically swap out indicators that produce buy/sell signals or assign the same indicator to multiple feeds, this post is for you.
Background
The Backtrader blog has a good tutorial that shows you the basics of how to work with multiple data feeds. However, I do think value can be added here with a more gentle introduction aimed beginners and by expanding on some of the concepts in the official blog post. For example, no indicators are used in the blog post and that may leave you wondering how to initialize them if you are new to programming. Luckily enough, Python has us covered and I am going to show you how to dynamically create indicators in a loop rather than hard-coding attribute (variable) names during __init__().
The official blog post can be found here:
https://www.backtrader.com/blog/posts/2017-04-09-multi-example/multi-example.html
Getting Data
The examples in this post will use data I downloaded from Oanda. You can replace the data I am using with your own. However, if you prefer to just copy, paste and run, then take a copy of the data files used below. Just make sure you place them in a “data” folder above the script so that they can be found.
If you decide NOT to work with the data provided, please note that in the following examples, I am using a bt.feeds.GenericCSVData sub-class to tell cerebro which columns in the CSV file are for open, high, low and close data. You will not need this code.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
class OandaCSVData(bt.feeds.GenericCSVData): params = ( ('nullvalue', float('NaN')), ('dtformat', '%Y-%m-%dT%H:%M:%S.%fZ'), ('datetime', 6), ('time', -1), ('open', 5), ('high', 3), ('low', 4), ('close', 1), ('volume', 7), ('openinterest', -1), ) |
See: https://www.backtrader.com/docu/dataautoref.html
Feeding Multiple Data Feeds Into Cerebro
Let’s go through the process step by step. The first part is perhaps the easiest. Adding the data. It is done in exactly the same way you add data for a single instrument. You just create the data object, feed it into cerebro, rinse and repeat. I suggest creating a list or dictionary of data feeds you want to use. This will allow you to loop through the list without having separate lines of code for each data feed. Additionally, you could create the list at run-time using argparse or a config file to easily swap out data feeds or add more without changing the code. For an introduction to argparse see: Using argparse to change strategy parameters
The code snippet below shows an example of adding data feeds from a list. In it, you can see that the data list contains nested lists of both the data location and the name of the data. This list is then looped through to add the data into cerebro. It is import to make sure you use the “name” keyword when adding the data object to cerebro, this will allow you to easily differentiate the feeds when logging and printing to the terminal. More on that later.
|
1 2 3 4 5 6 7 8 9 |
datalist = [ ('data/CAD_CHF-2005-2017-D1.csv', 'CADCHF'), ('data/EUR_USD-2005-2017-D1.csv', 'EURUSD'), ('data/GBP_AUD-2005-2017-D1.csv', 'GBPAUD'), ] for i in range(len(datalist)): data = OandaCSVData(dataname=datalist[i][0]) cerebro.adddata(data, name=datalist[i][1]) |
Working with the data during next()
The key lines of code in the official blog post which allow you to work with multiple data sources easily are:
|
1 2 3 4 5 |
def next(self): for i, d in enumerate(self.datas): dt, dn = self.datetime.date(), d._name pos = self.getposition(d).size if not pos: |
Here we are looping through the data feeds one by one. The d variable contains the data object in question and it is this object that we need to primarily work with. In addition, d._name returns the data name we supplied when adding the data into cerebro. This helps us identify which feed we are working with. (useful for printing, logging and debugging etc). Finally, another different with our normal implementations is that we also do not use the strategies self.position attribute to determine whether we are in a position. Instead, the position size needs to be checked for each data feed. If it returns None, it means we are not in a position and can potentially make a trade.
Plotting on the same master
In the Backtrader blog above, the author uses a nice plot info parameter to make all the data feeds appear on the same chart. This is nice in the example but if you have too many data-feeds, things can get messy quick! Therefore I personally prefer to chart them separately. If I have a lot of data feeds, it might be worthwhile to turn plotting off altogether.
Another note about the example code is that author creates all data feeds during the setup of cerebro using separate lines of code like so:
|
1 2 3 4 5 6 7 8 9 10 11 |
# Data feed data0 = bt.feeds.YahooFinanceCSVData(dataname=args.data0, **kwargs) cerebro.adddata(data0, name='d0') data1 = bt.feeds.YahooFinanceCSVData(dataname=args.data1, **kwargs) data1.plotinfo.plotmaster = data0 cerebro.adddata(data1, name='d1') data2 = bt.feeds.YahooFinanceCSVData(dataname=args.data2, **kwargs) data2.plotinfo.plotmaster = data0 cerebro.adddata(data2, name='d2') |
Since I am advocating looping, I suggest setting the plotmaster attribute during the strategies __init__() method. An example of how this is done is contained in the “adding indicators” code snippet below.
Adding Indicators
The challenge with trying to support an unknown amount of data feeds is that we cannot hard code indicator names. E.g.
|
1 2 3 4 |
self.ind1 = bt.indicators.IndicatorName() self.ind2 = bt.indicators.IndicatorName() self.ind3 = bt.indicators.IndicatorName() self.ind4 = bt.indicators.IndicatorName() |
and so on…
My suggestion to takle this is to use a dictionary. We can create a dictionary where the data object is the key and the indicator objects are stored as values. Doing this allows the objects (and therefore values) to be easily accessed during each next() call. When we loop through the data feeds, we can simply pass that data feed object to the dictionary as a key reference to then access the indicators for that particular feed. The example below shows how to do this and forms part of the complete code. Additionally, whilst we are looping through the data feeds we can set the plotmaster attribute discussed in the section above.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
def __init__(self): ''' Create an dictionary of indicators so that we can dynamically add the indicators to the strategy using a loop. This mean the strategy will work with any numner of data feeds. ''' self.inds = dict() for i, d in enumerate(self.datas): self.inds[d] = dict() self.inds[d]['sma1'] = bt.indicators.SimpleMovingAverage( d.close, period=self.params.sma1) self.inds[d]['sma2'] = bt.indicators.SimpleMovingAverage( d.close, period=self.params.sma2) self.inds[d]['cross'] = bt.indicators.CrossOver(self.inds[d]['sma1'],self.inds[d]['sma2']) if i > 0: #Check we are not on the first loop of data feed: if self.p.oneplot == True: d.plotinfo.plotmaster = self.datas[0] |
Putting it all together
The full strategy below uses a simple moving average crossover strategy on multiple data feeds. By default, plotting is done on subplots. Should you want to change this, you can change the line:
|
1 2 |
#Add our strategy cerebro.addstrategy(maCross, oneplot=False) |
The Code
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 |
''' Author: www.backtest-rookies.com MIT License Copyright (c) 2017 backtest-rookies.com Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ''' import backtrader as bt from datetime import datetime class maCross(bt.Strategy): ''' For an official backtrader blog on this topic please take a look at: https://www.backtrader.com/blog/posts/2017-04-09-multi-example/multi-example.html oneplot = Force all datas to plot on the same master. ''' params = ( ('sma1', 40), ('sma2', 200), ('oneplot', True) ) def __init__(self): ''' Create an dictionary of indicators so that we can dynamically add the indicators to the strategy using a loop. This mean the strategy will work with any numner of data feeds. ''' self.inds = dict() for i, d in enumerate(self.datas): self.inds[d] = dict() self.inds[d]['sma1'] = bt.indicators.SimpleMovingAverage( d.close, period=self.params.sma1) self.inds[d]['sma2'] = bt.indicators.SimpleMovingAverage( d.close, period=self.params.sma2) self.inds[d]['cross'] = bt.indicators.CrossOver(self.inds[d]['sma1'],self.inds[d]['sma2']) if i > 0: #Check we are not on the first loop of data feed: if self.p.oneplot == True: d.plotinfo.plotmaster = self.datas[0] def next(self): for i, d in enumerate(self.datas): dt, dn = self.datetime.date(), d._name pos = self.getposition(d).size if not pos: # no market / no orders if self.inds[d]['cross'][0] == 1: self.buy(data=d, size=1000) elif self.inds[d]['cross'][0] == -1: self.sell(data=d, size=1000) else: if self.inds[d]['cross'][0] == 1: self.close(data=d) self.buy(data=d, size=1000) elif self.inds[d]['cross'][0] == -1: self.close(data=d) self.sell(data=d, size=1000) def notify_trade(self, trade): dt = self.data.datetime.date() if trade.isclosed: print('{} {} Closed: PnL Gross {}, Net {}'.format( dt, trade.data._name, round(trade.pnl,2), round(trade.pnlcomm,2))) class OandaCSVData(bt.feeds.GenericCSVData): params = ( ('nullvalue', float('NaN')), ('dtformat', '%Y-%m-%dT%H:%M:%S.%fZ'), ('datetime', 6), ('time', -1), ('open', 5), ('high', 3), ('low', 4), ('close', 1), ('volume', 7), ('openinterest', -1), ) #Variable for our starting cash startcash = 10000 #Create an instance of cerebro cerebro = bt.Cerebro() #Add our strategy cerebro.addstrategy(maCross, oneplot=False) #create our data list datalist = [ ('data/CAD_CHF-2005-2017-D1.csv', 'CADCHF'), #[0] = Data file, [1] = Data name ('data/EUR_USD-2005-2017-D1.csv', 'EURUSD'), ('data/GBP_AUD-2005-2017-D1.csv', 'GBPAUD'), ] #Loop through the list adding to cerebro. for i in range(len(datalist)): data = OandaCSVData(dataname=datalist[i][0]) cerebro.adddata(data, name=datalist[i][1]) # Set our desired cash start cerebro.broker.setcash(startcash) # Run over everything cerebro.run() #Get final portfolio Value portvalue = cerebro.broker.getvalue() pnl = portvalue - startcash #Print out the final result print('Final Portfolio Value: ${}'.format(portvalue)) print('P/L: ${}'.format(pnl)) #Finally plot the end results cerebro.plot(style='candlestick') |
The Result
And there we have it. A fully working strategy using multiple data feeds and indicators.
PS: If you are wondering why the default settings for the SMA’s are 40 and 200, see the SMA crossover review where we learned that these settings performed the best across all markets tested.
Backtrader Simple Moving Average Crossover Review
Hi, I followed the code and I can get the code to work if the stocks are of similiar price, but once they differs a lot, it will have this error:
“””dst[i] = math.fsum(src[i – period + 1:i + 1]) / period
IndexError: array assignment index out of range “””
Not sure how to fix myself
Hi Jacky,
Thanks for following along…
It is hard to say without seeing the full code. I guess some key differences will be the data you are using and the code edits made to use that data.
Feel free to post the full code and the full error below and I can see if I spot anything that might be causing it.
Full Codes below, I have only changed the area where how the data is imported. My method of import utilises Panda feed import instead. The CSV file is downloaded in the Google Csv format.
import backtrader as bt
import datetime
import pandas as pd
class maCross(bt.Strategy):
”’
For an official backtrader blog on this topic please take a look at:
https://www.backtrader.com/blog/posts/2017-04-09-multi-example/multi-example.html
oneplot = Force all datas to plot on the same master.
”’
params = (
(‘sma1’, 4),
(‘sma2’, 20),
(‘oneplot’, True)
)
def __init__(self):
”’
Create an dictionary of indicators so that we can dynamically add the
indicators to the strategy using a loop. This mean the strategy will
work with any numner of data feeds.
”’
self.inds = dict()
for i, d in enumerate(self.datas):
print(“B : ” + str(i))
self.inds[d] = dict()
self.inds[d][‘sma1’] = bt.indicators.SimpleMovingAverage(
d.close, period=self.params.sma1)
self.inds[d][‘sma2’] = bt.indicators.SimpleMovingAverage(
d.close, period=self.params.sma2)
self.inds[d][‘cross’] = bt.indicators.CrossOver(self.inds[d][‘sma1’],self.inds[d][‘sma2’])
if i > 0: #Check we are not on the first loop of data feed:
if self.p.oneplot == True:
d.plotinfo.plotmaster = self.datas[0]
def next(self):
for i, d in enumerate(self.datas):
dt, dn = self.datetime.date(), d._name
pos = self.getposition(d).size
if not pos: # no market / no orders
if self.inds[d][‘cross’][0] == 1:
self.buy(data=d, size=1000)
elif self.inds[d][‘cross’][0] == -1:
self.sell(data=d, size=1000)
else:
if self.inds[d][‘cross’][0] == 1:
self.close(data=d)
self.buy(data=d, size=1000)
elif self.inds[d][‘cross’][0] == -1:
self.close(data=d)
self.sell(data=d, size=1000)
def notify_trade(self, trade):
dt = self.data.datetime.date()
if trade.isclosed:
print(‘{} {} Closed: PnL Gross {}, Net {}’.format(
dt,
trade.data._name,
round(trade.pnl,2),
round(trade.pnlcomm,2)))
#
#Variable for our starting cash
startcash = 10000
#Create an instance of cerebro
cerebro = bt.Cerebro()
#Add our strategy
cerebro.addstrategy(maCross, oneplot=False)
datapath = “C:/Users/.spyder-py3/STOCKS/BACKTRADER/data/STOCK_DATA.txt”
df = pd.read_csv(datapath, parse_dates=True, index_col=0)
#create our data list
datalist = [
(3389, ‘Stock1’),
(5, ‘Stock2’),
]
#Loop through the list adding to cerebro.
for i in range(len(datalist)):
print(‘A : ‘ + str(i))
df = df[df.stock_code == datalist[i][0]]
data = bt.feeds.PandasData(dataname=df, fromdate=datetime.datetime(2016, 12, 12), todate=datetime.datetime(2018, 1, 4))
cerebro.adddata(data, name=datalist[i][1])
# Set our desired cash start
cerebro.broker.setcash(startcash)
# Run over everything
cerebro.run()
#Get final portfolio Value
#portvalue = cerebro.broker.getvalue()
#pnl = portvalue – startcash
#
#Print out the final result
#print(‘Final Portfolio Value: ${}’.format(portvalue))
#print(‘P/L: ${}’.format(pnl))
#Finally plot the end results
cerebro.plot(style=’candlestick’)
runfile(‘C:/Users/43924746/.spyder-py3/STOCKS/BACKTRADER/MAIN FOLDER/TALIB/MULTI_STOCKS.py’, wdir=’C:/Users/43924746/.spyder-py3/STOCKS/BACKTRADER/MAIN FOLDER/TALIB’)
A : 0
A : 1
B : 0
B : 1
Traceback (most recent call last):
File “”, line 1, in
runfile(‘C:/Users/43924746/.spyder-py3/STOCKS/BACKTRADER/MAIN FOLDER/TALIB/MULTI_STOCKS.py’, wdir=’C:/Users/43924746/.spyder-py3/STOCKS/BACKTRADER/MAIN FOLDER/TALIB’)
File “C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py”, line 880, in runfile
execfile(filename, namespace)
File “C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py”, line 102, in execfile
exec(compile(f.read(), filename, ‘exec’), namespace)
File “C:/Users/43924746/.spyder-py3/STOCKS/BACKTRADER/MAIN FOLDER/TALIB/MULTI_STOCKS.py”, line 102, in
cerebro.run()
File “C:\ProgramData\Anaconda3\lib\site-packages\backtrader-1.9.59.122-py3.6.egg\backtrader\cerebro.py”, line 1127, in run
runstrat = self.runstrategies(iterstrat)
File “C:\ProgramData\Anaconda3\lib\site-packages\backtrader-1.9.59.122-py3.6.egg\backtrader\cerebro.py”, line 1290, in runstrategies
self._runonce(runstrats)
File “C:\ProgramData\Anaconda3\lib\site-packages\backtrader-1.9.59.122-py3.6.egg\backtrader\cerebro.py”, line 1648, in _runonce
strat._once()
File “C:\ProgramData\Anaconda3\lib\site-packages\backtrader-1.9.59.122-py3.6.egg\backtrader\lineiterator.py”, line 292, in _once
indicator._once()
File “C:\ProgramData\Anaconda3\lib\site-packages\backtrader-1.9.59.122-py3.6.egg\backtrader\lineiterator.py”, line 292, in _once
indicator._once()
File “C:\ProgramData\Anaconda3\lib\site-packages\backtrader-1.9.59.122-py3.6.egg\backtrader\lineiterator.py”, line 312, in _once
self.oncestart(self._minperiod – 1, self._minperiod)
File “C:\ProgramData\Anaconda3\lib\site-packages\backtrader-1.9.59.122-py3.6.egg\backtrader\lineiterator.py”, line 322, in oncestart
self.once(start, end)
File “C:\ProgramData\Anaconda3\lib\site-packages\backtrader-1.9.59.122-py3.6.egg\backtrader\indicators\basicops.py”, line 364, in once
dst[i] = math.fsum(src[i – period + 1:i + 1]) / period
IndexError: array assignment index out of range
My first guess is that there is something wrong with your data feed. The error you are seeing caused by the simple moving average indicator. It is possible that there is not enough data for one of the stocks you are adding.
A similar issue was reported here some time ago:
https://community.backtrader.com/topic/407/indexerror-array-assignment-index-out-of-range
However, compared to that post, your date range looks good but the error is the same. So maybe try to isolate the issue. You can try to replace the stock data you are using with a simple Quandl feed using the Wiki data.