Backtrader: Multiple Data Feeds & Indicators

If you have read through the Backtrader: First Script post or seen any of the other code snippets on this site, you will see that most examples work with just one data feed. Similarly, the number of indicators to be used in a strategy is well-defined in advance. This is for good reason. As the site is targeted more towards the beginner, it makes sense for each post to keep the code as simple as possible and focus only on the subject at hand. As such, this is the first post to focus on backtesting with multiple data feeds. If you want to be able to change the number of data feeds without changing the code, dynamically swap out indicators that produce buy/sell signals or assign the same indicator to multiple feeds, this post is for you.

Background

The Backtrader blog has a good tutorial that shows you the basics of how to work with multiple data feeds. However, I do think value can be added here with a more gentle introduction aimed beginners and by expanding on some of the concepts in the official blog post. For example, no indicators are used in the blog post and that may leave you wondering how to initialize them if you are new to programming. Luckily enough, Python has us covered and I am going to show you how to dynamically create indicators in a loop rather than hard-coding attribute (variable) names during __init__(). The official blog post can be found here: https://www.backtrader.com/blog/posts/2017-04-09-multi-example/multi-example.html

Getting Data

The examples in this post will use data I downloaded from Oanda. You can replace the data I am using with your own. However, if you prefer to just copy, paste and run, then take a copy of the data files used below. Just make sure you place them in a “data” folder above the script so that they can be found.

1. CAD_CHF-2005-2017-D1
2. EUR_USD-2005-2017-D1
3. GBP_AUD-2005-2017-D1

If you decide NOT to work with the data provided, please note that in the following examples, I am using a bt.feeds.GenericCSVData sub-class to tell cerebro which columns in the CSV file are for open, high, low and close data. You will not need this code.

class OandaCSVData(bt.feeds.GenericCSVData):
    params = (
        ('nullvalue', float('NaN')),
        ('dtformat', '%Y-%m-%dT%H:%M:%S.%fZ'),
        ('datetime', 6),
        ('time', -1),
        ('open', 5),
        ('high', 3),
        ('low', 4),
        ('close', 1),
        ('volume', 7),
        ('openinterest', -1),
    )

See: https://www.backtrader.com/docu/dataautoref.html

Feeding Multiple Data Feeds Into Cerebro

Let’s go through the process step by step. The first part is perhaps the easiest. Adding the data. It is done in exactly the same way you add data for a single instrument. You just create the data object, feed it into cerebro, rinse and repeat. I suggest creating a list or dictionary of data feeds you want to use. This will allow you to loop through the list without having separate lines of code for each data feed. Additionally, you could create the list at run-time using argparse or a config file to easily swap out data feeds or add more without changing the code. For an introduction to argparse see: Using argparse to change strategy parameters The code snippet below shows an example of adding data feeds from a list. In it, you can see that the data list contains nested lists of both the data location and the name of the data. This list is then looped through to add the data into cerebro. It is import to make sure you use the “name” keyword when adding the data object to cerebro, this will allow you to easily differentiate the feeds when logging and printing to the terminal. More on that later.

datalist = [
    ('data/CAD_CHF-2005-2017-D1.csv', 'CADCHF'),
    ('data/EUR_USD-2005-2017-D1.csv', 'EURUSD'),
    ('data/GBP_AUD-2005-2017-D1.csv', 'GBPAUD'),
]

for i in range(len(datalist)):
    data = OandaCSVData(dataname=datalist[i][0])
    cerebro.adddata(data, name=datalist[i][1])

Working with the data during next()

The key lines of code in the official blog post which allow you to work with multiple data sources easily are:

def next(self):
    for i, d in enumerate(self.datas):
        dt, dn = self.datetime.date(), d._name
        pos = self.getposition(d).size
        if not pos:

Here we are looping through the data feeds one by one. The d variable contains the data object in question and it is this object that we need to primarily work with. In addition, d._name returns the data name we supplied when adding the data into cerebro. This helps us identify which feed we are working with. (useful for printing, logging and debugging etc). Finally, another different with our normal implementations is that we also do not use the strategies self.position attribute to determine whether we are in a position. Instead, the position size needs to be checked for each data feed. If it returns None, it means we are not in a position and can potentially make a trade.

Plotting on the same master

In the Backtrader blog above, the author uses a nice plot info parameter to make all the data feeds appear on the same chart. This is nice in the example but if you have too many data-feeds, things can get messy quick! Therefore I personally prefer to chart them separately. If I have a lot of data feeds, it might be worthwhile to turn plotting off altogether. Another note about the example code is that author creates all data feeds during the setup of cerebro using separate lines of code like so:

# Data feed
data0 = bt.feeds.YahooFinanceCSVData(dataname=args.data0, **kwargs)
cerebro.adddata(data0, name='d0')

data1 = bt.feeds.YahooFinanceCSVData(dataname=args.data1, **kwargs)
data1.plotinfo.plotmaster = data0
cerebro.adddata(data1, name='d1')

data2 = bt.feeds.YahooFinanceCSVData(dataname=args.data2, **kwargs)
data2.plotinfo.plotmaster = data0
cerebro.adddata(data2, name='d2')

Since I am advocating looping, I suggest setting the plotmaster attribute during the strategies __init__() method. An example of how this is done is contained in the “adding indicators” code snippet below.

Adding Indicators

The challenge with trying to support an unknown amount of data feeds is that we cannot hard code indicator names. E.g.

self.ind1 = bt.indicators.IndicatorName()
self.ind2 = bt.indicators.IndicatorName()
self.ind3 = bt.indicators.IndicatorName()
self.ind4 = bt.indicators.IndicatorName()

and so on… My suggestion to takle this is to use a dictionary. We can create a dictionary where the data object is the key and the indicator objects are stored as values. Doing this allows the objects (and therefore values) to be easily accessed during each next() call. When we loop through the data feeds, we can simply pass that data feed object to the dictionary as a key reference to then access the indicators for that particular feed. The example below shows how to do this and forms part of the complete code. Additionally, whilst we are looping through the data feeds we can set the plotmaster attribute discussed in the section above.

    def __init__(self):
        '''
        Create an dictionary of indicators so that we can dynamically add the
        indicators to the strategy using a loop. This mean the strategy will
        work with any numner of data feeds. 
        '''
        self.inds = dict()
        for i, d in enumerate(self.datas):
            self.inds[d] = dict()
            self.inds[d]['sma1'] = bt.indicators.SimpleMovingAverage(
                d.close, period=self.params.sma1)
            self.inds[d]['sma2'] = bt.indicators.SimpleMovingAverage(
                d.close, period=self.params.sma2)
            self.inds[d]['cross'] = bt.indicators.CrossOver(self.inds[d]['sma1'],self.inds[d]['sma2'])

            if i > 0: #Check we are not on the first loop of data feed:
                if self.p.oneplot == True:
                    d.plotinfo.plotmaster = self.datas[0]

Putting it all together

The full strategy below uses a simple moving average crossover strategy on multiple data feeds. By default, plotting is done on subplots. Should you want to change this, you can change the line:

#Add our strategy
cerebro.addstrategy(maCross, oneplot=False)

The Code

'''
Author: www.backtest-rookies.com

MIT License

Copyright (c) 2017 backtest-rookies.com

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
'''

import backtrader as bt
from datetime import datetime


class maCross(bt.Strategy):
    '''
    For an official backtrader blog on this topic please take a look at:

    https://www.backtrader.com/blog/posts/2017-04-09-multi-example/multi-example.html

    oneplot = Force all datas to plot on the same master.
    '''
    params = (
    ('sma1', 40),
    ('sma2', 200),
    ('oneplot', True)
    )

    def __init__(self):
        '''
        Create an dictionary of indicators so that we can dynamically add the
        indicators to the strategy using a loop. This mean the strategy will
        work with any numner of data feeds. 
        '''
        self.inds = dict()
        for i, d in enumerate(self.datas):
            self.inds[d] = dict()
            self.inds[d]['sma1'] = bt.indicators.SimpleMovingAverage(
                d.close, period=self.params.sma1)
            self.inds[d]['sma2'] = bt.indicators.SimpleMovingAverage(
                d.close, period=self.params.sma2)
            self.inds[d]['cross'] = bt.indicators.CrossOver(self.inds[d]['sma1'],self.inds[d]['sma2'])

            if i > 0: #Check we are not on the first loop of data feed:
                if self.p.oneplot == True:
                    d.plotinfo.plotmaster = self.datas[0]

    def next(self):
        for i, d in enumerate(self.datas):
            dt, dn = self.datetime.date(), d._name
            pos = self.getposition(d).size
            if not pos:  # no market / no orders
                if self.inds[d]['cross'][0] == 1:
                    self.buy(data=d, size=1000)
                elif self.inds[d]['cross'][0] == -1:
                    self.sell(data=d, size=1000)
            else:
                if self.inds[d]['cross'][0] == 1:
                    self.close(data=d)
                    self.buy(data=d, size=1000)
                elif self.inds[d]['cross'][0] == -1:
                    self.close(data=d)
                    self.sell(data=d, size=1000)

    def notify_trade(self, trade):
        dt = self.data.datetime.date()
        if trade.isclosed:
            print('{} {} Closed: PnL Gross {}, Net {}'.format(
                                                dt,
                                                trade.data._name,
                                                round(trade.pnl,2),
                                                round(trade.pnlcomm,2)))


class OandaCSVData(bt.feeds.GenericCSVData):
    params = (
        ('nullvalue', float('NaN')),
        ('dtformat', '%Y-%m-%dT%H:%M:%S.%fZ'),
        ('datetime', 6),
        ('time', -1),
        ('open', 5),
        ('high', 3),
        ('low', 4),
        ('close', 1),
        ('volume', 7),
        ('openinterest', -1),
    )


#Variable for our starting cash
startcash = 10000

#Create an instance of cerebro
cerebro = bt.Cerebro()

#Add our strategy
cerebro.addstrategy(maCross, oneplot=False)

#create our data list
datalist = [
    ('data/CAD_CHF-2005-2017-D1.csv', 'CADCHF'), #[0] = Data file, [1] = Data name
    ('data/EUR_USD-2005-2017-D1.csv', 'EURUSD'),
    ('data/GBP_AUD-2005-2017-D1.csv', 'GBPAUD'),
]

#Loop through the list adding to cerebro.
for i in range(len(datalist)):
    data = OandaCSVData(dataname=datalist[i][0])
    cerebro.adddata(data, name=datalist[i][1])


# Set our desired cash start
cerebro.broker.setcash(startcash)

# Run over everything
cerebro.run()

#Get final portfolio Value
portvalue = cerebro.broker.getvalue()
pnl = portvalue - startcash

#Print out the final result
print('Final Portfolio Value: ${}'.format(portvalue))
print('P/L: ${}'.format(pnl))

#Finally plot the end results
cerebro.plot(style='candlestick')

The Result

And there we have it. A fully working strategy using multiple data feeds and indicators. PS: If you are wondering why the default settings for the SMA’s are 40 and 200, see the SMA crossover review where we learned that these settings performed the best across all markets tested. Backtrader Simple Moving Average Crossover Review