One of the reasons backtesting often does not provide an accurate indication of real-world performance is that we receive a large chunk of information all at the same time. Specifically, the open, high, low and close data for a given time period. As such, we are missing important information about what happened between the open and the close. We know the highs price reached but we don’t know how it got there. Was price action volatile? Was it a steady climb? Did price hit the lows first? The higher the timeframe you trade on, the more information you are missing. This is where replaying data from a lower time-frame can help add an extra layer of realism to your backtests (assuming you can find the lower time-frame data). Replaying data works in a similar fashion to re-sampling but with one key advantage. The lower time-frame data that you are using to create the higher time frame candle is used to paint a picture of what happened between the open and close. For example, you can work on the daily timeframe but also know that price dipped before reaching the high of the day by replaying minute data. Another benefit is that replaying data addresses an issue some of you may have encountered where both a stop loss and a take profit are hit on the same bar. This is usually due to a large volatile movement in one candle. The reason they can be both triggered is that the engine has no idea if the take profit was hit first or the stop loss. It only knows that price crossed both of the thresholds during that bar. When replaying data, this will not happen as one will be triggered first as the bar is built. As with everything, replaying data is not without its downsides. There are some added complexities introduced and things to be aware of when replaying data. Therefore, I think tackling these points is where this post can add some value.
Official DocsThe good news is, getting setup and replaying data is very straightforward. The official documentation gives some good examples and can be found here: https://www.backtrader.com/docu/data-replay/data-replay.html
Candles are built with each tick of the lower time-frameThe first thing you need to be aware of when replaying data is that candles are built gradually. If you are referencing the
closeof a candle, the values can and will update with every tick of the data. This might appear to be obvious when written down but it is easy to overlook. Especially if you are used to working on a higher time-frame and of reference
self.data.close. This is because…
Next() is called on every tick of the lower time-frameThe
next()method is called every time a bar on the input data is replayed. For example, if you replay data on the daily time-frame and using minute data,
next()will be called every minute. The example image above was created by just printing OHLC data on each call of
next()is called so many times, we cannot simply run a script written for a higher timeframe. We must first add a few lines that prevent us from making entries/exits as the candle forms. More on that in a bit.
Bar numbers/data length only increase on when a bar is completeIn Backtrader, you can easily find out which bar number you are looking at by calling
len(self.data). Usually, (when not replaying data) the bar number will increase with each call of
next(). However, when replaying data, it will only increase when each bar on the higher timeframe is complete. This allows you to identify when a bar is complete and a new one is starting. This also means that referencing the index position
[-1]will return a full bar’s worth of OHLC data for the candle that has just closed on the higher timeframe. On the other hand, referencing the index position
will only return data for a tick of the lower time frame. As mentioned above, it is a constantly evolving value with every call of
Identifying when you have a new barThe following is a simple example of the
next()method that can be copied into your own strategy to determine when to take actions on the higher timeframe.
def __init__(self): #For tracking HTF self.last_bar = len(self) def next(self): bar = len(self) #Check we are not in a new position and we have a bar on HTF. if not self.position and bar > self.last_bar: pass self.last_bar = barAs you can see it is pretty straightforward. We are just checking whether the current bar is greater than the last bar recorded. Since
self.last_baris recorded with every tick,
barwill only be different to
self.last_baron the first tick of the new bar. Furthermore, you can reference index
[-1]to get the full OHLC of the most recent full bar since it will only be called once per bar.
But I want the work on the last tick of the barIf you prefer to work from the last tick of the bar, then you will need to have a counter. With each call of next, increase the counter by one. Then once a new bar is identified, reset the counter. You can then perform actions
if self.counter = x:. For example, if you are replaying minute data on the hourly timeframe you could perform an action every time the counter == 60.