Backtrader: Manage Dividends and Splits with Adjusted Close Data

This post follows on from Backtesting 101: Dividends and Adjustments. In that post, we discussed the importance of accounting for and handling dividends when backtesting. Today, we will take a deeper look at how to download and work with adjusted data in Backtrader.

Prerequisites

The example code will require users to download data from Alpha Vantage. A free API key is needed for this. As such, users should head over to their homepage, sign up and create an API key. The API is free to use but you will be limited to around 5 API calls per minute. In addition to Alpha Vantage, we will be making use of a couple of additional Python modules that will make our lives easier. Those modules are:

• pandas: https://pandas.pydata.org/
• numpy: http://www.numpy.org/

Note: To find instructions on how to install python modules, take a look at the Getting Setup tutorial or the official docs.

Backtrader Dividends and Splits Scope

The process is not quite as simple as downloading data from another source. If we want to use OHLCVvalues, we also need to adjust some parts of the data. As such, during the course of this post we will cover the following:

• Downloading data from Alpha Vantage
• Adjusting the open,highand lowdata.
• Creating a data feed in Backtrader with PandasData
• Comparing Adjusted vs Unadjusted returns

Example Code

'''
Author: www.backtest-rookies.com

MIT License

Copyright (c) 2018 backtest-rookies.com

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
'''

from alpha_vantage.timeseries import TimeSeries
import backtrader as bt
from datetime import datetime
import argparse
import pandas as pd
import numpy as np


def parse_args():
    parser = argparse.ArgumentParser(description='CCXT Market Data Downloader')


    parser.add_argument('-s','--symbol',
                        type=str,
                        required=True,
                        help='The Symbol of the Instrument/Currency Pair To Download')


    parser.add_argument('--adjusted',
                        action='store_true',
                        help='Use Adjusted Data')

    parser.add_argument('--rounding',
                        default=4,
                        help='Round adjusted data to nearest x')

    return parser.parse_args()


class testStrategy(bt.Strategy):

    def next(self):
        if not self.position:
            self.order_target_percent(target=1)


def adjust(date, close, adj_close, in_col, rounding=4):
    '''
    If using forex or Crypto - Change the rounding accordingly!
    '''
    try:
        factor = adj_close / close
        return round(in_col * factor, rounding)
    except ZeroDivisionError:
        print('WARNING: DIRTY DATA >> {} Close: {} | Adj Close {} | in_col: {}'.format(date, close, adj_close, in_col))
        return 0


args = parse_args()

apikey = 'INSERT YOUR API KEY'

#Variable for our starting cash
startcash = 10000

#Create an instance of cerebro
cerebro = bt.Cerebro()

#Add our strategy
cerebro.addstrategy(testStrategy)

#Get Apple data from ALPHA Vantage.
# -------------------------------
# Submit our API and create a session
alpha_ts = TimeSeries(key=apikey, output_format='pandas')

# Get the data
if args.adjusted:

    data, meta_data = alpha_ts.get_daily_adjusted(symbol=args.symbol, outputsize='full')

    #Convert the index to datetime.
    data.index = pd.to_datetime(data.index)

    # NOTE: Pandas Keys = '1. open', '2. high', '3. low', '4. close', '5. adjusted close',
    #                     '6. volume','7. dividend', '8. split coefficient'

    # Adjust the rest of the data
    data['adj open'] = np.vectorize(adjust)(data.index.date, data['4. close'], data['5. adjusted close'], data['1. open'], rounding=args.rounding)
    data['adj high'] = np.vectorize(adjust)(data.index.date, data['4. close'], data['5. adjusted close'], data['2. high'], rounding=args.rounding)
    data['adj low'] = np.vectorize(adjust)(data.index.date, data['4. close'], data['5. adjusted close'], data['3. low'], rounding=args.rounding)

    # Extract the colums we want to work with and rename them.
    data = data[['adj open', 'adj high', 'adj low','5. adjusted close','6. volume']]
    data.columns = ['Open','High','Low','Close','Volume']

else:

    data, meta_data = alpha_ts.get_daily(symbol=args.symbol, outputsize='full')
    #Convert the index to datetime.
    data.index = pd.to_datetime(data.index)
    # Rename columns
    data.columns = ['Open','High','Low','Close','Volume']

#Add the data to Cerebro
data_in = bt.feeds.PandasData(dataname=data,
                timeframe=bt.TimeFrame.Days,
                compression=1)

cerebro.adddata(data_in)

# Set our desired cash start
cerebro.broker.setcash(startcash)

# Run over everything
cerebro.run()

#Get final portfolio Value
portvalue = round(cerebro.broker.getvalue(),2)
pnl = portvalue - startcash

#Print out the final result
print('Final Portfolio Value: ${}'.format(portvalue))
print('P/L: ${}'.format(pnl))

#Finally plot the end results
cerebro.plot(style='candlestick')

Code Commentary

First things first! In order to run the script, you need to insert your API key. Look for the following line and update it accordingly: apikey = 'INSERT YOUR API KEY' To allow us to easily compare adjusted vs unadjusted returns, the example code allows arguments to be set at runtime using the argparsemodule. For those new to coding, there is an in-depth tutorial showing how to use argparsewith backtrader on this site. Our test strategy is simple. We just buy and hold! This is probably the shortest strategy snippet on the site. You might notice that we are not simply using self.buy()here. Instead, we use order_target_percentwhich allows us to try and place x% of our cash into the asset. For more information regarding order targets, see here.

Data Acquisition

Next, we move onto something a little more complex. We need to download, adjust and enter the data into Backtrader. First, we simply download the data using the alpha_vantagemodule. When we do this, data is returned from the module in a pandasdataframe. If we then print()the returned dataframe we will see something like this: As we can see, only the close has been adjusted. If we want to work with OHLC data, we need to make some adjustments to the open,highand lowcolumns. To do this we need to dive a little further into the world of pandasand get a little help from a useful function in the numpy module. The formula used to adjust the values is quite simple. We just work out the adjustment factor by dividing the adj close with the close. We can then use to same factor to adjust the open,high,lowandclose. However, we cannot run a traditional for loop over the data frame. This is where numpycomes to the rescue. Numpy’s vectorize function allows us to apply the adjust()function to each row in the dataframe. You can think of it as similar to a loop since it applies the function to every row.  Having said that, I am not familiar with the underlying mechanics so take that statement as a conceptual aid. data['Adj Open'] = np.vectorize(adjust)(data['Close'], data['Adj Close'], data['Open'], rounding=args.rounding) Rounding allows us to set ensure our final values are appropriate for the asset. The default here is 4 decimal places as that is what Alpha Vantage provide data to for equities. Note that if you are trading Crypto or Forex, you may wish to change the rounding parameter. Finally, we drop the unadjusted data and rename the columns so that Backtrader will automatically recognize which column contains which type of data. (e.g which column is volume).

Running the Code

First, let’s run the code with unadjusted data to see our baseline. You can do this with the following command. python3 .\adjusted_data_example.py -s AAPL Note: The example assumes you have saved the file as adjusted_data_example.py This should result in a chart which looks similar to this: Now let’s add the --adjustedswitch to the command like so: python3 .\adjusted_data_example.py -s AAPL --adjusted Immediately we can see we have quite different results! If you paid attention to the final PnL, you will see a difference of a hundred or so thousand vs millions in profit! Hopefully, now you can really see how those stock splits negatively affected the returns in our backtest even though they would not have had any impact in the real world. Also, we gained extra returns from the dividends received during this period. Note that since we are using the full data-set provided by Alpha Vantage, this means your results will not exactly match the ones posted here. Apple will continue to move up and down in the future. If you want to see the effect of dividends only without the splits or focus on a specific time period, you can filter the backtest start and end days by changing:

#Add the data to Cerebro
data_in = bt.feeds.PandasData(dataname=data,
                timeframe=bt.TimeFrame.Days,
                compression=1)

To this:

data_in = bt.feeds.PandasData(dataname=data,
                timeframe=bt.TimeFrame.Days,
                compression=1,
                fromdate=datetime(2016,1,1),
                todate=datetime(2017,1,1))

This time, the difference in chart appearance is not as obvious as the last example. However, the difference in PnL is still significant. During my test, unadjusted data returned $946.57 vs $1,191.69 with adjusted data. And that is all there is to it. Remember, to use adjusted data whenever backtesting over a long period of time with equities to capture those dividend gains!