CCXT is a cross-language library for interacting with Crypto exchange API’s. At the time of writing, the library provides a single interface to over 100 cryptocurrency exchanges. In this post, we are going to use the library to build a tool that will allow us to download cryptocurrency data for backtesting. For more information on the library, visit CCXT’s GitHub page (referenced below). For those of you who prefer a quick overview, here is a snippet straight from the horse’s mouth:
It is intended to be used by coders, developers, technically-skilled traders, data-scientists and financial analystsfor building trading algorithms on top of it.
Current feature list:
- support for many exchange markets, even more upcoming soon
- fully implemented public and private APIs for all exchanges
- all currencies, altcoins and symbols, prices, order books, trades, tickers, etc…
- optional normalized data for cross-exchange or cross-currency analytics and arbitrage
- an out-of-the box unified all-in-one API extremely easy to integrate
- works in Node 7.6+, Python 2 and 3, PHP 5.3+, web browsers
You can find out more about the project here: https://github.com/ccxt/ccxt
Installing the Library
As with most modules, installing CCXT is made simple via pip. Just open up the command line and type:
pip install ccxt
If you happen to havepython2
installed alongsidepython3
and want to install it for python3
, simply replace pip
withpip3
.
Other Dependencies
If this is your first time visiting the site, it is assumed you know how to install Python 3 and run some basic scripts. If that is not you, try looking at the Getting Setup: Python and Backtrader post first.
Lastly, the script below requires pandas
to also be installed. However, the code could be updated to not require it. The download tool only uses pandas
to massage and export the data received. To install it use the same pip
command as above but replace ccxt
withpandas
.
Reference: https://pandas.pydata.org/
Download Tool Overview
In the first iteration, the download tool shall be a simple command line tool. It shall allow simple arguments to be specified at runtime using the argeparse
module for flexibility. This means we can avoid altering the code everytime we want to download different data. For a deeper look at argparse
and how it can be used with Backtrader, see here: Using argparse to change strategy parameters
The download tool shall allow users to specify the following options:
- Instrument (Currency Pair)
- Timeframe (1m, 5m, 1 hour etc)
- Exchange
Notice how there are no inputs for start and end dates? The tool shall download everything that is available using CCXT’s fetchOHLCV
method. Once the data is downloaded, it shall be exported to a CSV file in the current folder. You may wish to tweak this and add anotherargparse
argument to specify the output directory.
The Code
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 |
''' Author: www.backtest-rookies.com MIT License Copyright (c) 2018 backtest-rookies.com Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ''' import ccxt from datetime import datetime, timedelta, timezone import math import argparse import pandas as pd def parse_args(): parser = argparse.ArgumentParser(description='CCXT Market Data Downloader') parser.add_argument('-s','--symbol', type=str, required=True, help='The Symbol of the Instrument/Currency Pair To Download') parser.add_argument('-e','--exchange', type=str, required=True, help='The exchange to download from') parser.add_argument('-t','--timeframe', type=str, default='1d', choices=['1m', '5m','15m', '30m','1h', '2h', '3h', '4h', '6h', '12h', '1d', '1M', '1y'], help='The timeframe to download') parser.add_argument('--debug', action ='store_true', help=('Print Sizer Debugs')) return parser.parse_args() # Get our arguments args = parse_args() # Get our Exchange try: exchange = getattr (ccxt, args.exchange) () except AttributeError: print('-'*36,' ERROR ','-'*35) print('Exchange "{}" not found. Please check the exchange is supported.'.format(args.exchange)) print('-'*80) quit() # Check if fetching of OHLC Data is supported if exchange.has["fetchOHLCV"] != True: print('-'*36,' ERROR ','-'*35) print('{} does not support fetching OHLC data. Please use another exchange'.format(args.exchange)) print('-'*80) quit() # Check requested timeframe is available. If not return a helpful error. if (not hasattr(exchange, 'timeframes')) or (args.timeframe not in exchange.timeframes): print('-'*36,' ERROR ','-'*35) print('The requested timeframe ({}) is not available from {}\n'.format(args.timeframe,args.exchange)) print('Available timeframes are:') for key in exchange.timeframes.keys(): print(' - ' + key) print('-'*80) quit() # Check if the symbol is available on the Exchange exchange.load_markets() if args.symbol not in exchange.symbols: print('-'*36,' ERROR ','-'*35) print('The requested symbol ({}) is not available from {}\n'.format(args.symbol,args.exchange)) print('Available symbols are:') for key in exchange.symbols: print(' - ' + key) print('-'*80) quit() # Get data data = exchange.fetch_ohlcv(args.symbol, args.timeframe) header = ['Timestamp', 'Open', 'High', 'Low', 'Close', 'Volume'] df = pd.DataFrame(data, columns=header).set_index('Timestamp') # Save it symbol_out = args.symbol.replace("/","") filename = '{}-{}-{}.csv'.format(args.exchange, symbol_out,args.timeframe) df.to_csv(filename) |
The code is quite straightforward. After using argparse
to capture our exchange, instrument and timezone requirements, the bulk of the work is spent error checking before and calling CCXT’s fetch_ohlcv
method.
Speaking of error checking, not all exchanges support fetching of OHLCV data. Furthermore, not all exchanges support the same timeframes and currency pairs. Since there are over 100 exchanges supported at the time of writing, it is easy to imagine that there will be a lot of variances between exchanges. To avoid frustration with bumping into error after error trying to guess which timeframes and/or currency pairs are supported, the code performs a series of checks before attempting to download data. It detected something is not supported, it will notify the user and provide a list of alternative parameters where applicable.
Other than that, it is worth paying attention to the exchange.load_markets()
call if you plan to expand upon the code or work with the library yourself. Admittedly, I spent a little too much some time scratching my head, trying to figure out why every exchange I tested seemed to have empty markets
, symbols
and markets_by_id
dictionaries. It turns out that snuggled in the documentation is the answer, you need to call load_markets()
before any data is available. This was a good reminder to read the documentation carefully! A little time spent reading can save a lot of time.
Finally, if you are new to programming, the getattr(ccxt, args.exchange)
might be new to you. getattr()
is a python built-in function which allows you to “get an attribute” from an object using a string. In this case, we are getting the correct exchange object from the given argument string.
Reference: https://docs.python.org/3/library/functions.html#getattr
Usage Examples
Please note that the following usage examples assume the script is saved as ccxt_market_data.py
.
ETH/NEO Binance Daily
python .ccxt_market_data.py -s NEO/ETH -e binance -t 1d
Unsupported Timeframe
The example demonstrates the expected result when an unsupported timeframe is given as an argument:
python .ccxt_market_data.py -s NEO/ETH -e bittrex -t 12h
1 2 3 4 5 6 7 8 9 |
------------------------------------ ERROR ----------------------------------- The requested timeframe (12h) is not available from bittrex Available timeframes are: - 1m - 5m - 30m - 1h - 1d -------------------------------------------------------------------------------- |
Unsupported Symbol
The second example has a fair bit of output. It demonstrates the expected result when an unsupported symbol is given as an argument:
python .ccxt_market_data.py -s ATP/USD -e bittrex -t 5m
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 |
------------------------------------ ERROR ----------------------------------- The requested symbol (ATP/USD) is not available from bittrex Available symbols are: - 1ST/BTC - 1ST/ETH - 2GIVE/BTC - ABY/BTC - ADA/BTC - ADA/ETH - ADA/USDT - ADT/BTC - ADT/ETH - ADX/BTC - ADX/ETH - AEON/BTC - AGRS/BTC - AMP/BTC - ANT/BTC - ANT/ETH - ARDR/BTC - ARK/BTC - AUR/BTC - BAT/BTC - BAT/ETH - BAY/BTC - BCH/BTC - BCH/ETH - BCH/USDT - BCPT/BTC - BCPT/ETH - BCY/BTC - BITB/BTC - BLITZ/BTC - BLK/BTC - BLOCK/BTC - BNT/BTC - BNT/ETH - BRK/BTC - BRX/BTC - BSD/BTC - BTC/USDT - BTG/BTC - BTG/ETH - BTG/USDT - BURST/BTC - BYC/BTC - CANN/BTC - CFI/BTC - CFI/ETH - CLAM/BTC - CLOAK/BTC - CLUB/BTC - COVAL/BTC - CPC/BTC - CRB/BTC - CRB/ETH - CRW/BTC - CURE/BTC - CVC/BTC - CVC/ETH - DASH/BTC - DASH/ETH - DASH/USDT - DCR/BTC - DCT/BTC - DGB/BTC - DGB/ETH - DMD/BTC - DNT/BTC - DNT/ETH - DOGE/BTC - DOPE/BTC - DTB/BTC - DYN/BTC - EBST/BTC - EDG/BTC - EFL/BTC - EGC/BTC - EMC/BTC - EMC2/BTC - ENG/BTC - ENG/ETH - ENRG/BTC - ERC/BTC - ETC/BTC - ETC/ETH - ETC/USDT - ETH/BTC - ETH/USDT - EXCL/BTC - EXP/BTC - FAIR/BTC - FCT/BTC - FCT/ETH - FLDC/BTC - FLO/BTC - FTC/BTC - GAM/BTC - GAME/BTC - GBG/BTC - GBYTE/BTC - GCR/BTC - GEO/BTC - GLD/BTC - GNO/BTC - GNO/ETH - GNT/BTC - GNT/ETH - GOLOS/BTC - GRC/BTC - GRS/BTC - GUP/BTC - GUP/ETH - HMQ/BTC - HMQ/ETH - IGNIS/BTC - INCNT/BTC - INFX/BTC - IOC/BTC - ION/BTC - IOP/BTC - KMD/BTC - KORE/BTC - LBC/BTC - LGD/BTC - LGD/ETH - LMC/BTC - LSK/BTC - LTC/BTC - LTC/ETH - LTC/USDT - LUN/BTC - LUN/ETH - MAID/BTC - MANA/BTC - MANA/ETH - MCO/BTC - MCO/ETH - MEME/BTC - MER/BTC - MLN/BTC - MONA/BTC - MUE/BTC - MUSIC/BTC - NAV/BTC - NBT/BTC - NEO/BTC - NEO/ETH - NEO/USDT - NEOS/BTC - NLG/BTC - NMR/BTC - NMR/ETH - NXC/BTC - NXS/BTC - NXT/BTC - NXT/USDT - OK/BTC - OMG/BTC - OMG/ETH - OMG/USDT - OMNI/BTC - PART/BTC - PAY/BTC - PAY/ETH - PDC/BTC - PINK/BTC - PIVX/BTC - PKB/BTC - POT/BTC - POWR/BTC - POWR/ETH - PPC/BTC - PTC/BTC - PTOY/BTC - PTOY/ETH - QRL/BTC - QRL/ETH - QTUM/BTC - QTUM/ETH - QWARK/BTC - RADS/BTC - RBY/BTC - RCN/BTC - RCN/ETH - RDD/BTC - REP/BTC - REP/ETH - RLC/BTC - RLC/ETH - SALT/BTC - SALT/ETH - SBD/BTC - SC/BTC - SC/ETH - SEQ/BTC - SHIFT/BTC - SIB/BTC - SLR/BTC - SLS/BTC - SNRG/BTC - SNT/BTC - SNT/ETH - SPHR/BTC - SPR/BTC - SRN/BTC - SRN/ETH - START/BTC - STEEM/BTC - STORJ/BTC - STORJ/ETH - STRAT/BTC - STRAT/ETH - SWIFT/BTC - SWT/BTC - SYNX/BTC - SYS/BTC - THC/BTC - TIX/BTC - TIX/ETH - TKS/BTC - TRST/BTC - TRST/ETH - TRUST/BTC - TRX/BTC - TRX/ETH - TUSD/BTC - TX/BTC - UBQ/BTC - UKG/BTC - UKG/ETH - UNB/BTC - VEE/BTC - VEE/ETH - VIA/BTC - VIB/BTC - VIB/ETH - VOX/BTC - VRC/BTC - VRM/BTC - VTC/BTC - VTR/BTC - WAVES/BTC - WAVES/ETH - WAX/BTC - WAX/ETH - WINGS/BTC - WINGS/ETH - XCP/BTC - XDN/BTC - XEL/BTC - XEM/BTC - XEM/ETH - XLM/BTC - XLM/ETH - XMG/BTC - XMR/BTC - XMR/ETH - XMR/USDT - XMY/BTC - XRP/BTC - XRP/ETH - XRP/USDT - XST/BTC - XVC/BTC - XVG/BTC - XVG/USDT - XWC/BTC - XZC/BTC - ZCL/BTC - ZEC/BTC - ZEC/ETH - ZEC/USDT - ZEN/BTC - ZRX/BTC - ZRX/ETH -------------------------------------------------------------------------------- |
Unsupported Exachange
The final example demostrates the expected result when an exchange does not support fethcing of OHLCV data.
python .ccxt_market_data.py -s BTC/USD -e gdax -t 15m
1 2 3 |
------------------------------------ ERROR ----------------------------------- gdax does not support fetching OHLC data. Please use another exchange -------------------------------------------------------------------------------- |
Issues and Improvements
The code tries to be user-friendly in that it will tell the user when incorrect data is provided. Having said that, not all edge cases have been considered and there is definitely room for improvement in that regard.
In addition to this, the amount of data available on most exchanges for download is limited. CCXT does provide a since
parameter for the fetchOHLCV()
method but I did not see any detailed description for this in the fetchOHLCV()
documentation. Having said that, an explanation of a parameter with the same name does appear the “Trades, Executions and Transactions” section as follows (Note: That the parameter appears frequently, so I assume it will have consistent functionality but that is no guarantee!):
The second optional argument
since
reduces the array by timestamp, the thirdlimit
argument reduces by number (count) of returned items.
Reference: https://github.com/ccxt/ccxt/wiki/Manual#trades-executions-transactions
So taking that into account, it appears since
should reduce the amount of data provided. However, it would be good to know for sure. If anyone pokes through the codebase and has an answer, I would be happy to hear it. Leave a comment below!
Hi,
many thanks for the tutorial.
I am trying to import the data into your optimise strategy which I have modified.
However it does not seem to work.
How would you modify your optimise example to include ccxt?
Hi Jaspal,
It is hard to say without seeing the strategy. However, one thing I can think of is to check the way you are adding the data. You will need to import the data downloaded as CSV and make sure the columns are mapped correctly.
Try taking a look at the GenericCSV docs here:
https://www.backtrader.com/docu/datafeed.html#genericcsvdata
Hi, I managed to figure it out. I had to declare the format of the csv file in my code with:
# CSV INPUT FILE FORMAT CONFIGURATION
class dataFeed(btfeed.GenericCSVData):
params = (
(‘dtformat’, ‘%Y-%m-%d %H:%M:%S’),
(‘datetime’, 0),
(‘open’, 1),
(‘high’, 2),
(‘low’, 3),
(‘close’, 4),
(‘volume’, 5),
(‘openinterest’, -1)
)
Thanks again!
Awesome!
@Jaspal : exactly what I was looking for! thank you
Amazing! Thanks a lot for sharing this post, it´s really useful
Another thing…watching the CSV file generated with the call..the first column corresponding to Timestamp…it would be better if we change the time format using datetime in order to get something readable.
Hello,
Thanks for the great code.
I thought I’d share, with all teh other noobs, a problem I had.
… If you name this code ccxt.py, specifically, it will not fire.
It will come back with an “Exchange “bitfinex” not found. Please check the exchange is supported.” error, no matter what exchange is entered.
Renaming the file to any name other than ccxt.py will solve this issue.
Some Guy
Hi Some Guy, I have the same exchange not supported error no matter what exchange is entered. I tried changing the name of my file to no avail. Any ideas?
error begin 20180902
python /root/ccxt_market_data.py -s BTC/USDT -e binance -t 6h
1535911200000,7298.38,7333.33,7237.06,7302.01,7507.444406
1535932800000,7302.0,7325.0,7191.63,7225.0,7222.945765
1535954400000,7225.02,7260.0,7213.25,7250.79,1612.071928
the close is ok, but the open,high,low are error.
https://imgur.com/a/kD6fGL4
Hi Garufeng,
The script simply saved the results returned from the exchange. If you think there are data errors, it might be worth checking it with the exchange.
Some exchanges have API explorers where you can make test calls to the API and compare it against the results.
Good luck!
Thank you for the help . how do you change the format of date in the csv file?
Hello!
Thanks for the post!
Do you know how can I get more than just one pair from the same exchange? Do you I have to separate the markets with a comma or something like that?
Thanks!
Hi Lucas!
The code example would need to be expanded to support multiple downloads. We would need to handle a list of pairs with some in argparse and also run the download in a loop.
Alternatively, you could create a bash script to run this script multiple times, each time with a different pair.
Cheers!
Thanks, Rookie! We created a bash script here and run smoothly 🙂
Thank you so much for the code.
One last question, if I may, is there a way to get 1 year timeframe in all exchanges supported or only the timeframes provided by the specific API?
Bittrex, for instance, I couldn’t get this information. What I specifically need is the volume of that specific pair in Bittrex in the last year. Is there a way to do it or I would have to buy some market data, such as the one from CoinMktCap?
Hello! Noob here. For some reason, I always return “Exchange “coinbase” not found. Please check the exchange is supported.” with the same error for any exchange. Any idea what I’m doing wrong?
Hi Stepan – Sorry for the late reply.
Coinbase does not support downloading OHLC data. The error message should tell you this.
Try to use GDAX instead. (CoinbasePro)
Just tested – This line is working:
python .\ccxt_market_data.py -exchange -e gdax -s BTC/USD -t 1m
Replace python with python3 if you also have python2 installed!
Hello! I described an error that I was having with CCXT and BackTest code and got an insightful answer from one of the maintainers, you should check it out! 🙂
https://github.com/ccxt/ccxt/issues/4356
Cheers!
Awesome Lucas! It is always good to improve the code.
The suggestions from the maintainer have been added to the code above.
Hello, how are you?
When I run this to get the volume from bitfinex, it only gives me 101 entries, starting and ending in 2017.
Is there some way I can get only the latest 3 months (~101 entries)?
Thanks!
Hi Gabriel,
I think Bitfinex provide their data in reverse order and limit the amount of data points returned.
As such, you would need add some extra parameters to the
exchange.fetch_ohlcv()
call.The parameters you need are
since
limit
andparams
since
controls the date we request from,limit
controls the download size (for requesting more than 101 data points) andparams
can be used to request the data in reverse order.See here for more info:
https://github.com/ccxt/ccxt/wiki/Manual#ohlcv-candlestick-charts
Here is a snippet regarding
since
” Some exchanges will return candles from the beginning of time, others will return most recent candles only, the exchanges’ default behaviour is expected. Thus, without specifying since the range of returned candles will be exchange-specific. One should pass the since argument to ensure getting precisely the history range needed.”
Dear all,
Could you help me how to download ccxt program and install in my computer?
Thank you for the syntax!
I am completely new to pyhton and get stuck to the line
# Get our arguments
args = parse_args()
It returns:
File “”, line 31
args = parse_args()
Are there any solution for that?
Thank you so much for your help!
I have figured that out and it works now!
Such a silly question I asked.
Thank you for the script!!
Admittedly I’m a complete novice in python, and you could argue I should go way back and learn the basics before attempting this….
But I just cant get from one step to the next. If I copy the above script for the argparse and save it down, it does not allow me to call that script running the above command.
If I type in “python3 argparse-tutorial.py –help” and run it I get the following error:
———————————————————————————————-
C:\Users\xxxx\PycharmProjects\untitled\venv\Scripts\python.exe C:/Users/xxxx/PycharmProjects/untitled/venv/Lib/site-packages/backtrader/e.py
File “C:/Users/xxxx/PycharmProjects/untitled/venv/Lib/site-packages/backtrader/e.py”, line 1
python argparse-tutorial.py –help
^
SyntaxError: invalid syntax
Process finished with exit code 1
——————————————————————————————————-
I can’t understand what’s going wrong, it seems like there is a step missing in this tutorial, and the one for the Cyrpto data, which is what you do with the argparse script, and how you run another script querying it?
Sorry for the dumb question.
H
Ignore me, I’m obviously too noob to comprehend. I will explain the issue so any other noobs don’t run into the same issue.
When you save the script you need to run the command in command prompt(windows) or bash(Mac), for example:
> open command prompt
> go to the directory the script is saved in
> run command (eg. “python .ccxt_market_data.py -s NEO/ETH -e binance -t 1d”)
For mega noobs like me, I thought i was running a SECOND python script which was dependant on the argparse script, but it was in fact a command line prompt that should be ran.
(I know how stupid I look, but I wanted to detail the issue I had so that I might save another noob the time I just spent trying to work it out)
Hi Henry,
Don’t worry about it! It is never stupid to ask a question. We are all here to learn.
Actually, the fact you had an issue means you are likely not the only one and so your comment here will help others.
Thanks and good luck with the learning!
Hello,
Currently I can only get 500 row of data.
Is it possible to get more?
Thank you!
Hi,
@Rookie1 Thanks very much for the code. This is really helpful.
I am also facing this issue. Can only fetch 500 rows of data in csv.
Could someone please suggest how to control for start and end dates.
Thanks!
Hi,
As other users tell in their comments, format of timestamp as printed in the CSV file is not that easy to use (at least for a newbie).
More specifically, I would like to import back the data in backtrader.
As you have used panda.Dataframe for export, it would seem that using Panda data feed in backtrader would me seamless.
I used the script given in backtrader doc here:
https://www.backtrader.com/docu/pandas-datafeed/pandas-datafeed.html?highlight=panda
But I have trouble, date format is not recognised.
I don’t know how solving the issue.
Please, do you think you could show an updated version of your script or below script to be able to make such data transfer between CCXT & Backtrader, relying on panda.dataFrame?
from __future__ import (absolute_import, division, print_function,
unicode_literals)
import argparse
import backtrader as bt
import backtrader.feeds as btfeeds
import pandas
def runstrat():
args = parse_args()
# Create a cerebro entity
cerebro = bt.Cerebro(stdstats=False)
# Add a strategy
cerebro.addstrategy(bt.Strategy)
# Get a pandas dataframe
datapath = ('../../datas/2006-day-001.txt')
# Simulate the header row isn't there if noheaders requested
skiprows = 1 if args.noheaders else 0
header = None if args.noheaders else 0
dataframe = pandas.read_csv(datapath,
skiprows=skiprows,
header=header,
parse_dates=True,
index_col=0)
if not args.noprint:
print('--------------------------------------------------')
print(dataframe)
print('--------------------------------------------------')
# Pass it to the backtrader datafeed and add it to the cerebro
data = bt.feeds.PandasData(dataname=dataframe)
cerebro.adddata(data)
# Run over everything
cerebro.run()
# Plot the result
cerebro.plot(style='bar')
def parse_args():
parser = argparse.ArgumentParser(
description='Pandas test script')
parser.add_argument('--noheaders', action='store_true', default=False,
required=False,
help='Do not use header rows')
parser.add_argument('--noprint', action='store_true', default=False,
help='Print the dataframe')
return parser.parse_args()
if __name__ == '__main__':
runstrat()
I hope you will be able to consider positively this demand.
Have a good day and thanks to share all your tutorials.
Bests,
Pierrot
Hi,
to the interested readers that would like to write timestamp in iso format, I found the solution given in CCXT bug tracker here:
https://github.com/ccxt/ccxt/issues/4478
So a tiniy single line does the trick (maybe to be included in your script David?)
# convert timestamps to iso8601
data = [[exchange.iso8601(candle[0])] + candle[1:] for candle in data]
To be added in the “Get data” section:
# Get data
data = exchange.fetch_ohlcv(args.symbol, args.timeframe)
# convert timestamps to iso8601
data = [[exchange.iso8601(candle[0])] + candle[1:] for candle in data]
header = ['Timestamp', 'Open', 'High', 'Low', 'Close', 'Volume']
df = pd.DataFrame(data, columns=header).set_index('Timestamp')
Hope this will be of some help.
Have a good afternoon.
Best regards,
Pierrot
Hello.
How I can add a ‘symbol’ to CSV?
and more specific question. What I can do to have exact format in CSV like this one:
Date,Symbol,Open,High,Low,Close,Volume BTC,Volume USD
2019-05-17 01-AM,BTCUSD,7881.86,7897.92,7881.86,7897.92,0.0,0.0
Correct read data from saved csv to backtrader:
filename = ‘{}-{}-{}.csv’.format(‘binance’, symbol.replace(“/”,””), timeframe)
df = pd.read_csv(filename,
header=0,
index_col=’Timestamp’,
parse_dates=[‘Timestamp’],
date_parser=lambda x: pd.to_datetime(int(x)/1000,unit=’s’)
)
data = bt.feeds.PandasData(dataname=df)
only fetch 500 rows of data in csv.
Could someone please suggest how to control for start and end dates. thanks!
Regarding the
since
parameter https://github.com/ccxt/ccxt/wiki/Manual#ohlcv-candlestick-charts states:> The since argument is an integer UTC timestamp in milliseconds
> (everywhere throughout the library with all unified methods).
With Python 3.3 and above one could use something along the lines of this for the
since
parameter:int(datetime.strptime('1.1.2017 1:00', '%d.%m.%Y %H:%M').timestamp() * 1000)
As a sidenote: E.g. with Binance you can download a maximum limit of 1000 lines of ohlc data with one go. See:
https://github.com/binance-exchange/binance-official-api-docs/blob/master/rest-api.md#klinecandlestick-data
I have to correct myself.
The above time conversion would assume the local timezone. To get around that
UTC
should be specified explicitly like this:int(datetime.strptime(args.since, '%d.%m.%Y %H:%M').replace(tzinfo=timezone.utc).timestamp() * 1000)
You could regenerate the resulting timestamp with a website like https://www.epochconverter.com/timezones?q=1578445200000&tz=UTC and crosscheck the ohlc data from the file for this candle with the one in tradingview.com
Can you show how to insert this code into the above script? I have been struggling to implement the suggestion you have here.
Hello!! Thanks very much for the code 😉
I cant run the code. The error message should tell you this:
Error 2
Looking for stack:
> File “C:\Users\Shinobi\AppData\Local\temporary projects\pythonapplication1\pythonapplication1.py”, line 33, in parse_args
> return parser.parse_args()
> File “C:\Users\Shinobi\AppData\Local\temporary projects\pythonapplication1\pythonapplication1.py”, line 36, in
> args = parse_args()
Please, can you help me? thanks