QuantConnect: Handling Dividends

As we learned in Backtesting 101: Dividends and Adjustments, dividends are a fundamental part of Investing. Yet they are often overlooked in the backtesting world. This might be because generally, frameworks do not provide the specific data or tools required to handle a dividend. As a result, end users need to find adjusted data from 3rd parties such as Alpha Vantage. Luckily for QuantConnect users, dividend data and appropriate tools to handle them are readily available in the platform. Furthermore, this can be as simple as setting a single initialization option.

Scope

As you might have guessed by now, we are going to look at how to access dividend information and the various options for working with them in a buy and hold strategy. Note: Whilst aimed at beginners, this post assumes the reader is familiar with how the basic mechanics of a script works on QuantConnect. If you are arriving fresh from another platform or a complete beginner, please take a look through the other posts in the getting started series.

Accessing Dividends

On QuantConnect we do not need to add a special dividend data feed. Instead, dividend “events” are pushed to our strategies in the same way as OHLCV data when they happen. The events are contained within the data “slice” that is passed to theOnData() function. The official documentation provides an overview of the different types of data slices that are pushed to our algorithms here: https://www.quantconnect.com/docs/algorithm-reference/handling-data Whilst that page does describe how to access dividend data, I personally think, QuantConnect’s DividendAlgorithm.py example makes it much more clear. The example is worth more than a thousand words and QuantConnect’s is so good that we will use as a base to create our code example later. To summarize the example, we can access the dividend (if any) through a special data object/type contained in our data slice. To be more specific, we access them using data.Dividends. Note, Dividendscan be iterated (looped) through to access the dividend information. If no dividend is available (on days where there is no dividend), then the code inside the loop will not be executed or cause an error. The official example achieves this with the line: for kvp in data.Dividends:

Handling Dividends

So now we know where we can access dividend information. Next, we need to understand how to handle them. As mentioned earlier QuantConnect does provide us with a few options for how the dividends can be handled. We are not limited to simply adjusting the data like other platforms (although we can if we want!). Instead, we can also decide to receive the cash dividends to our account. These options are not covered in the online documentation link above or the DividendAlgorithm.py example but can be found with a little digging around. The options fall under a category called “Data Normalization Modes” and setting a different data normalization modes will affect whether we receive cash balance updates or perform a price adjustment. Note: During my search, I was unable to find data normalization modes in the official online docs. If you do know the location, please leave a comment below and some links will be added. I discovered them through the forums and here is a block quote from that post:

Do you want raw asset prices, with dividends paid as cash and splits adjusting the quantity of your holdings? This is the closest possible to reality, and can be achieved with the new setting: DataNormalizationMode.Raw Would you prefer splits adjusted into the price, but dividends paid in cash? This is ideal for charting since the price is continuous, but the dividends are still paid in cash. You can use this mode with: DataNormalizationMode.SplitAdjusted How about Total Return charting? Where dividends are added into the price of the asset, so the total asset price includes the sum of dividends? This is a popular technique to account for the receipt and reinvestment of dividends. You can access this mode with: DataNormalizationMode.TotalReturn

Source: https://www.quantconnect.com/forum/discussion/508/update-dividends-splits-and-custom-price-normalization/p1 Data normalization modes are set during the initialization of our algorithm and a full example is provided below.

Example Code

import numpy as np

### <summary>
### Basic template algorithm simply initializes the date range and cash. This is a skeleton
### framework you can use for designing an algorithm.
### </summary>
class BasicTemplateAlgorithm(QCAlgorithm):
    '''Basic template algorithm simply initializes the date range and cash'''

    def Initialize(self):
        '''Initialise the data and resolution required, as well as the cash and start-end dates for your algorithm. All algorithms must initialized.'''
        
        self.ticker = 'MSFT'
        self.raw_handling = True
        
        self.SetStartDate(2018,5,11)   #Set Start Date
        self.SetEndDate(2018,5,22)    #Set End Date
        self.SetCash(10000)           #Set Strategy Cash
        # Find more symbols here: http://quantconnect.com/data
        self.AddEquity(self.ticker, Resolution.Daily)
        
        # Set Dividend Handling Method
        # https://www.quantconnect.com/forum/discussion/508/update-dividends-splits-and-custom-price-normalization/p1
        if self.raw_handling:
            self.Securities[self.ticker].SetDataNormalizationMode(DataNormalizationMode.Raw)
        else:
            self.Securities[self.ticker].SetDataNormalizationMode(DataNormalizationMode.TotalReturn)

    def OnData(self, data):
        '''OnData event is the primary entry point for your algorithm. Each new data point will be pumped in here.

        Arguments:
            data: Slice object keyed by symbol containing the stock data
        '''
        if not self.Portfolio.Invested:
            self.SetHoldings(self.ticker, 1)
            
        for kvp in data.Dividends: # update this to Dividends dictionary
            div_ticker = kvp.Key
            div_distribution = kvp.Value.Distribution
            div_total_value = div_distribution * self.Portfolio[self.ticker].Quantity
            self.Log("{0} >> DIVIDEND >> {1} - ${2} - ${3}".format(self.Time, div_ticker, div_distribution, div_total_value))

        self.Log("{0} >> SUMMARY >> {1} | Port Cash: {2} | Port Value: {3} | Holdings: {4} | Price {5}".format(self.Time, 
            self.ticker, self.Portfolio.Cash, self.Portfolio.TotalPortfolioValue, self.Portfolio[self.ticker].Quantity, self.Portfolio[self.ticker].Price))

Code Commentary

Since this article is focused on dividends and not splits, the code example code focuses on testing raw and total returns data normalization modes. We set our normalization mode during Initialize()with a simple, single line: self.Securities[self.ticker].SetDataNormalizationMode(DataNormalizationMode.XYZ) Of course, in the example code, we have two versions of this line. They are wrapped in an if/else statement to allow easy switching between the modes. We also replace XYZwith the actual mode names (Rawand TotalReturns). Technically, that is all we have to do! Select our preferred normalization method and dividends will be handled automatically. Your account will either be credited with cash or the price will be adjusted automatically. It is that simple. However, since we are learning, exploring and investigating how dividends work in QuantConnect, some extra log information was added so we can look at the mechanics. This allows us to take obtain a better understanding of how the engine works.

Verifying Our Output

To see how the normalizations modes affect the backtest, we can take a look at our logs around the date that the dividend happens. Note that in our example, the date range used is quite specific to Microsoft. The data starts a few days before a known divided and finishes just after.

Raw Data Normalization

The output below is a copy of the log file generated from running the example code “as is” above.

2018-05-11 00:00:00 :	Launching analysis for 51299e7b283ab1a462baca7e8fcb4532 with LEAN Engine v2.4.0.0.4858
2018-05-11 00:00:00 :	2018-05-11 00:00:00 >> SUMMARY >> MSFT | Port Cash: 10000.0 | Port Value: 10000.00000 | Holdings: 0 | Price 97.9000
2018-05-12 00:00:00 :	2018-05-12 00:00:00 >> SUMMARY >> MSFT | Port Cash: 132.310 | Port Value: 10001.02000 | Holdings: 101 | Price 97.7100
2018-05-15 00:00:00 :	2018-05-15 00:00:00 >> SUMMARY >> MSFT | Port Cash: 132.310 | Port Value: 10034.35000 | Holdings: 101 | Price 98.0400
2018-05-16 00:00:00 :	2018-05-16 00:00:00 >> DIVIDEND >> MSFT - $0.42 - $42.42
2018-05-16 00:00:00 :	2018-05-16 00:00:00 >> SUMMARY >> MSFT | Port Cash: 174.730 | Port Value: 10004.05000 | Holdings: 101 | Price 97.3200
2018-05-17 00:00:00 :	2018-05-17 00:00:00 >> SUMMARY >> MSFT | Port Cash: 174.730 | Port Value: 9986.88000 | Holdings: 101 | Price 97.1500
2018-05-18 00:00:00 :	2018-05-18 00:00:00 >> SUMMARY >> MSFT | Port Cash: 174.730 | Port Value: 9888.91000 | Holdings: 101 | Price 96.1800
2018-05-19 00:00:00 :	2018-05-19 00:00:00 >> SUMMARY >> MSFT | Port Cash: 174.730 | Port Value: 9907.09000 | Holdings: 101 | Price 96.3600
2018-05-22 00:00:00 :	2018-05-22 00:00:00 >> SUMMARY >> MSFT | Port Cash: 174.730 | Port Value: 10035.36000 | Holdings: 101 | Price 97.6300
2018-05-23 00:00:00 :	2018-05-23 00:00:00 >> SUMMARY >> MSFT | Port Cash: 174.730 | Port Value: 10029.30000 | Holdings: 101 | Price 97.5700
2018-05-23 00:00:00 :	Algorithm Id:(51299e7b283ab1a462baca7e8fcb4532) completed in 1.18 seconds at 0k data points per second. Processing total of 11 data points.

If we take a look at the logs closely, we can see that on 2018-05-16we received a dividend of $42.42. We can also see that would cash rises to $174.73 from $132.31 the day before (an increase of $42.42). What you do with that cash is now up to you. You could re-invest it or perhaps, you want to simulate retirement and assume you will need to keep x% and only re-invest the leftovers? With RAW data normalization mode, the option is there for you to choose!

Total Return Data Normalization

To test the TotalReurn normalization mode, we need to change the following line from: self.raw_handling = True to: self.raw_handling = False The will activate the TotalReturnnormalization mode and produce the following output.

2018-05-11 00:00:00 :	Launching analysis for 4e2a805c2f7699067879e8bbea9ddd3a with LEAN Engine v2.4.0.0.4858
2018-05-11 00:00:00 :	2018-05-11 00:00:00 >> SUMMARY >> MSFT | Port Cash: 10000.0 | Port Value: 10000.00000 | Holdings: 0 | Price 97.9000
2018-05-12 00:00:00 :	2018-05-12 00:00:00 >> SUMMARY >> MSFT | Port Cash: 132.310 | Port Value: 10001.02000 | Holdings: 101 | Price 97.7100
2018-05-15 00:00:00 :	2018-05-15 00:00:00 >> SUMMARY >> MSFT | Port Cash: 132.310 | Port Value: 10034.35000 | Holdings: 101 | Price 98.0400
2018-05-16 00:00:00 :	2018-05-16 00:00:00 >> DIVIDEND >> MSFT - $0.42 - $42.42
2018-05-16 00:00:00 :	2018-05-16 00:00:00 >> SUMMARY >> MSFT | Port Cash: 132.310 | Port Value: 9961.63000 | Holdings: 101 | Price 97.3200
2018-05-17 00:00:00 :	2018-05-17 00:00:00 >> SUMMARY >> MSFT | Port Cash: 132.310 | Port Value: 9986.88000 | Holdings: 101 | Price 97.5700
2018-05-18 00:00:00 :	2018-05-18 00:00:00 >> SUMMARY >> MSFT | Port Cash: 132.310 | Port Value: 9888.91000 | Holdings: 101 | Price 96.6000
2018-05-19 00:00:00 :	2018-05-19 00:00:00 >> SUMMARY >> MSFT | Port Cash: 132.310 | Port Value: 9907.09000 | Holdings: 101 | Price 96.7800
2018-05-22 00:00:00 :	2018-05-22 00:00:00 >> SUMMARY >> MSFT | Port Cash: 132.310 | Port Value: 10035.36000 | Holdings: 101 | Price 98.0500
2018-05-23 00:00:00 :	2018-05-23 00:00:00 >> SUMMARY >> MSFT | Port Cash: 132.310 | Port Value: 10029.30000 | Holdings: 101 | Price 97.9900
2018-05-23 00:00:00 :	Algorithm Id:(4e2a805c2f7699067879e8bbea9ddd3a) completed in 1.26 seconds at 0k data points per second. Processing total of 11 data points.

Here we can see our stock price and account cash are in sync up until the dividend. Following the dividend, the two examples diverge. In this case, our cash remains the same as before the dividend and the price is then adjusted from 2018-05-17.  $97.57 in TotalReturn mode vs  $97.15 in Rawmode.

Ex-Div Dates

Before we finish, it is worth noting that when the data normalization mode is set to Raw, cash is paid into the account on the ex-dividend date and not on the payment date. We can see this if we compare the distribution with the ex-dividend dates on other online resources. This is something to be aware of as you would not normally receive the cash for the dividend on the ex-div date. Payments are usually around 1 month after the ex-div date. If you then reinvest the cash immediately, that money will be in the market for much longer than it could be in the real world. This, in turn, can affect the realism of the backtesting results.