Analysis – Performance of Indicator Trade Signals

In this post, we are going to use Backtrader together with Pandas to check the performance of trade signals generated by a simple indicator. I am interested whether certain signals are better at predicting moves in the market over a particular time period. In other words, do they appear to be more profitable in 10 bars following the signal or 200 bars? As such, the idea will be to generate a list of signals in backtrader and then plot the performance of each signal over its subsequent bars to see if any patterns emerge.

Code

'''
Author: www.backtest-rookies.com

MIT License

Copyright (c) 2019 backtest-rookies.com

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Backtrader to Pandas analysis of signal direction predicability
'''
import pandas as pd
import numpy as np
import backtrader as bt
from datetime import datetime
import argparse
import matplotlib.pyplot as plt

def parse_args():
    parser = argparse.ArgumentParser(description="COT BACKTESTER")

    parser.add_argument('--data',
                        default='data/EUR_USD-D.csv',
                        type=str,
                        help='Price Data CSV File')

    parser.add_argument('--results',
                        default='SignalAnalysis-Results.csv',
                        type=str,
                        help='Price Data CSV File')

    parser.add_argument('--bars',
                        default=200,
                        type=int,
                        help='Analyze the following x bars')

    parser.add_argument('--period',
                        default=200,
                        type=int,
                        help='SMA Lookback Period')

    parser.add_argument('--analyze_only',
                        action='store_true',
                        help='Perform Pandas Analysis Only')

    return parser.parse_args()


class SignalAnalysis(bt.Strategy):

    params = (
        ('sma_lkb', 200),
        )

    def __init__(self):

        self.setup_log_file()
        # Core Indicators
        self.sma = bt.indicators.SMA(self.data, period=self.p.sma_lkb)
        self.cross = bt.indicators.CrossOver(self.data.close, self.sma)

    def next(self):
        dt = self.datas[0].datetime.datetime()
        bar = len(self.datas[0])
        o = self.datas[0].open[0]
        h = self.datas[0].high[0]
        l = self.datas[0].low[0]
        c = self.datas[0].close[0]
        v = self.datas[0].volume[0]

        # Create some signal Aliases so we can easily replace them with
        # better signals later
        long = self.cross == 1
        short = self.cross == -1

        # This part can be left alone - Just create the long and short conditions
        # above and everything will work!
        if long:
            sig = 'Long'
        elif short:
            sig = 'Short'
        else:
            sig = ''

        with open(self.logfile, 'a+') as f:
            log = "{},{},{},{},{},{},{}\n".format(dt,o,h,l,c,v,sig)
            f.write(log)


    def setup_log_file(self):
        '''
        Function to setup log files.
        '''
        cn = self.__class__.__name__
        self.logfile ='{}-Results.csv'.format(cn)

        #Write the header to the trade log.
        log_header = 'Datetime,Open,High,Low,Close,Volume,Signal'

        with open(self.logfile, 'w') as file:
            file.write(log_header + "\n")


args = parse_args()

if not args.analyze_only:
    # Create an instance of cerebro
    cerebro = bt.Cerebro()

    # Add our strategy
    cerebro.addstrategy(SignalAnalysis, sma_lkb=args.period)

    # Create a Data Feed
    data = bt.feeds.GenericCSVData(
        timeframe=bt.TimeFrame.Days,
        compression=1,
        dataname=args.data,
        nullvalue=0.0,
        dtformat=('%Y-%m-%d %H:%M:%S'),
        datetime=0,
        time=-1,
        high=2,
        low=3,
        open=1,
        close=4,
        volume=5,
        openinterest=-1 #-1 means not used
        )

    cerebro.adddata(data)

    # Run the strategy
    strats = cerebro.run()



# Time For PANDAS
df = pd.read_csv(args.results, parse_dates=True, index_col=[0])

# Get long and short signal dates
longs = df[df['Signal'] == 'Long']
shorts = df[df['Signal'] == 'Short']

# Need to get the index values for each signal. We will use these later for
# looping and creating a new dataframe
long_entries = longs.index.values
short_entries = shorts.index.values

# Create two new dataframes for the results
long_analysis = pd.DataFrame()
short_analysis = pd.DataFrame()


def pnl_calc(x, open_value, short=False):
    '''
    open_value: Float, the close value on the day of the signal.
    short: Bool, will invert the percentage change calc for shorts
    because negative is good!
    '''
    try:
        pnl = (x - open_value) / open_value

        if short:
            return -pnl
        else:
            return pnl
    except ZeroDivisionError:
        return 0

for signal_date in long_entries:
    # This will get the close values for the 200 closes following the signal
    closes = pd.DataFrame(df['Close'].loc[signal_date:].head(args.bars + 1),).reset_index()

    # Then we calculate the difference
    closes['PNL'] = closes['Close'].apply(pnl_calc, open_value=closes['Close'].iloc[0], short=False)
    long_analysis[signal_date] = closes['PNL']

for signal_date in short_entries:
    closes = pd.DataFrame(df['Close'].loc[signal_date:].head(args.bars + 1),).reset_index()

    closes['PNL'] = closes['Close'].apply(pnl_calc, open_value=closes['Close'].iloc[0], short=True)
    short_analysis[signal_date] = closes['PNL']

# Get the average PNL
long_average = long_analysis.mean(axis=1)
short_average = short_analysis.mean(axis=1)

# Start Plotting!
# ------------------------------------------------------------------------------
fig, axes = plt.subplots(nrows=2, ncols=2)

# All long Entries
ax1 = long_analysis.plot(ax=axes[0,0], kind='line', legend=False, title='Long Signal PnL / Bar', )
ax1.set_ylabel("PnL %")

# All short Entries
ax2 = short_analysis.plot(ax=axes[0,1], kind='line',  legend=False, title='Short Signal PnL / Bar')
ax2.set_ylabel("PnL %")

# Average long Entries
ax3 = long_average.plot(ax=axes[1,0], kind='line', legend=False, title='Long Average')
ax3.set_xlabel("Bar No")

# Average Short Entries
ax4 = short_average.plot(ax=axes[1,1], kind='line', legend=False, title='Short Average')
ax4.set_xlabel("Bar No")

fig.set_size_inches(10.5, 6.5)

# Show the figure
plt.show()

Code Commentary

As this is not a tutorial, the commentary on each piece of code will be light. Instead, this section will provide some general notes on running the script and what the code does at a higher level. First of all, the code in this example expects a CSV file for data input. You can get a copy of the data used in this post here: EUR_USD-D. If you use the example file, be sure to place it in a sub-directory of the main script called “data”. A few runtime arguments have been added. These can be used to change the data feed or skip backtrader signal generation completely. This might be useful if you have already performed signal generation and just want to see the output chart again. Note: If you are new to argparse and setting run-time options, see this article.  In addition, if you do use a run-time option to analyse your own data, you need to make sure the structure of the file and date format of your data the same as the example file. That means the Open,High,Low, CloseandVolumecolumns are in the same order. Finally, you will need to make sure the date format is the same. If your data does not follow this format and you don’t want to edit your data, you can edit the following section to match your CSV file format:
    # Create a Data Feed
    data = bt.feeds.GenericCSVData(
        timeframe=bt.TimeFrame.Days,
        compression=1,
        dataname=args.data,
        nullvalue=0.0,
        dtformat=('%Y-%m-%d %H:%M:%S'),
        datetime=0,
        time=-1,
        high=2,
        low=3,
        open=1,
        close=4,
        volume=5,
        openinterest=-1 #-1 means not used
        )
If you don’t know how to do this, have a look at the following links: On launching the code, Backtrader will be used to run through a simple strategy. Then, during each call of next() the strategy will log OHLCVdata along with whether a Longor Shortsignal was generated. This log will be written to a CSV file that will subsequently make its way into Panda’s for analysis. The signals generated in the test strategy are just examples to keep the code and short and clean. All we do is generate signals when we have a close above or below a Simple Moving Average. You can easily swap this out for any complex signal of your choice by editing the Longand Short boolean variables. Pandas proved to be the trickier part for the author to implement. As such, I am sure there must be more efficient ways of coding this section. However, it appears to work (Thanks to google and stack overflow) and that is the main thing! Here we attempt to extract the date of each long and short signal and then pull xnumber of subsequent close prices. Once we have the close prices, we can then calculate the running PNL for each bar and insert it into new dataframe for plotting.

Results

Running the script with the default settings and example data will output a chart that looks like this: Analysis over 200 days The results are fascinating! So what we see is that generally long signals were more profitable than short signals over the test period. Personally, I think this makes sense as we know that the EURO has generally risen against the USD over the long term. Furthermore, the drops in the EURO have tended to be faster moves in comparison with the periods of appreciation. Finally, we are using a long look back on the SMA which means the results span 200 days. In other words, each signal is getting a benefit from the longer term trend as we move towards the second hundred bar.
EUR/USD Since 2003
So what happens if we reduce the period of analysis to just 20 days? Well, this happens: Analysis over 20 days I will leave you to come to your own conclusion on this chart. Like a candlestick chart, everyone will see something different! That is the beauty of this game. Moving onward, I would be interested to see how more complex signals perform over the shorter term and will be definitely swapping out the SMA in my own tests.

Find This Post Useful?

If this post saved you time and effort, please consider support the site! There are many ways to support us and some won’t even cost you a penny. 

Backtest Rookies is a registered with Brave publisher!

Brave users can drop us a tip.

Alternatively, support us by switching to Brave using this referral link and we will receive some BAT! 


https://brave.com/bac691

Referral Link

Enjoying the content and thinking of subscribing to Tradingview? Support this site by clicking the referral link before you sign up! 

https://tradingview.go2cloud.org/SHpB


Donate with PayPal using any payment method you are comfortable with! 


3HxNVyh5729ieTfPnTybrtK7J7QxJD9gjD

0x9a2f88198224d59e5749bacfc23d79507da3d431

M8iUBnb8KamHR18gpgi2sSG7jopddFJi8S