Download Cryptocurrency Data with CCXT

CCXT is a cross-language library for interacting with Crypto exchange API’s. At the time of writing, the library provides a single interface to over 100 cryptocurrency exchanges. In this post, we are going to use the library to build a tool that will allow us to download cryptocurrency data for backtesting. For more information on the library, visit CCXT’s GitHub page (referenced below). For those of you who prefer a quick overview, here is a snippet straight from the horse’s mouth:

It is intended to be used by coders, developers, technically-skilled traders, data-scientists and financial analystsfor building trading algorithms on top of it.

Current feature list:

support for many exchange markets, even more upcoming soon

fully implemented public and private APIs for all exchanges

all currencies, altcoins and symbols, prices, order books, trades, tickers, etc…

optional normalized data for cross-exchange or cross-currency analytics and arbitrage

an out-of-the box unified all-in-one API extremely easy to integrate

works in Node 7.6+, Python 2 and 3, PHP 5.3+, web browsers

You can find out more about the project here: https://github.com/ccxt/ccxt

Installing the Library

As with most modules, installing CCXT is made simple via pip. Just open up the command line and type:

pip install ccxt

If you happen to havepython2installed alongsidepython3and want to install it for python3, simply replace pipwithpip3.

Other Dependencies

If this is your first time visiting the site, it is assumed you know how to install Python 3 and run some basic scripts. If that is not you, try looking at the Getting Setup: Python and Backtrader post first.

Lastly, the script below requires pandasto also be installed. However, the code could be updated to not require it. The download tool only uses pandas to massage and export the data received. To install it use the same pipcommand as above but replace ccxtwithpandas.

Reference: https://pandas.pydata.org/

Download Tool Overview

In the first iteration, the download tool shall be a simple command line tool. It shall allow simple arguments to be specified at runtime using the argeparsemodule for flexibility. This means we can avoid altering the code everytime we want to download different data. For a deeper look at argparseand how it can be used with Backtrader, see here: Using argparse to change strategy parameters

The download tool shall allow users to specify the following options:

Instrument (Currency Pair)
Timeframe (1m, 5m, 1 hour etc)
Exchange

Notice how there are no inputs for start and end dates? The tool shall download everything that is available using CCXT’s fetchOHLCVmethod. Once the data is downloaded, it shall be exported to a CSV file in the current folder. You may wish to tweak this and add anotherargparseargument to specify the output directory.

The Code

'''
Author: www.backtest-rookies.com

MIT License

Copyright (c) 2018 backtest-rookies.com

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
'''

import ccxt
from datetime import datetime, timedelta, timezone
import math
import argparse
import pandas as pd


def parse_args():
    parser = argparse.ArgumentParser(description='CCXT Market Data Downloader')


    parser.add_argument('-s','--symbol',
                        type=str,
                        required=True,
                        help='The Symbol of the Instrument/Currency Pair To Download')

    parser.add_argument('-e','--exchange',
                        type=str,
                        required=True,
                        help='The exchange to download from')

    parser.add_argument('-t','--timeframe',
                        type=str,
                        default='1d',
                        choices=['1m', '5m','15m', '30m','1h', '2h', '3h', '4h', '6h', '12h', '1d', '1M', '1y'],
                        help='The timeframe to download')


    parser.add_argument('--debug',
                            action ='store_true',
                            help=('Print Sizer Debugs'))

    return parser.parse_args()

# Get our arguments
args = parse_args()

# Get our Exchange
try:
    exchange = getattr (ccxt, args.exchange) ()
except AttributeError:
    print('-'*36,' ERROR ','-'*35)
    print('Exchange "{}" not found. Please check the exchange is supported.'.format(args.exchange))
    print('-'*80)
    quit()

# Check if fetching of OHLC Data is supported
if exchange.has["fetchOHLCV"] == False:
    print('-'*36,' ERROR ','-'*35)
    print('{} does not support fetching OHLC data. Please use another exchange'.format(args.exchange))
    print('-'*80)
    quit()

# Check requested timeframe is available. If not return a helpful error.
if args.timeframe not in exchange.timeframes:
    print('-'*36,' ERROR ','-'*35)
    print('The requested timeframe ({}) is not available from {}\n'.format(args.timeframe,args.exchange))
    print('Available timeframes are:')
    for key in exchange.timeframes.keys():
        print('  - ' + key)
    print('-'*80)
    quit()

# Check if the symbol is available on the Exchange
exchange.load_markets()
if args.symbol not in exchange.symbols:
    print('-'*36,' ERROR ','-'*35)
    print('The requested symbol ({}) is not available from {}\n'.format(args.symbol,args.exchange))
    print('Available symbols are:')
    for key in exchange.symbols:
        print('  - ' + key)
    print('-'*80)
    quit()


# Get data
data = exchange.fetch_ohlcv(args.symbol, args.timeframe)
header = ['Timestamp', 'Open', 'High', 'Low', 'Close', 'Volume']
df = pd.DataFrame(data, columns=header).set_index('Timestamp')
# Save it
symbol_out = args.symbol.replace("/","")
filename = '{}-{}-{}.csv'.format(args.exchange, symbol_out,args.timeframe)
df.to_csv(filename)

100

101

102

103

104

105

106

107

108

109

'''

Author: www.backtest-rookies.com

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy

of this software and associated documentation files (the "Software"), to deal

in the Software without restriction, including without limitation the rights

to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

copies of the Software, and to permit persons to whom the Software is

furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all

copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE

AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,

OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

SOFTWARE.

'''

import ccxt

from datetime import datetime, timedelta, timezone

import math

import argparse

import pandas as pd

def parse_args():

parser = argparse.ArgumentParser(description='CCXT Market Data Downloader')

parser.add_argument('-s','--symbol',

type=str,

required=True,

help='The Symbol of the Instrument/Currency Pair To Download')

parser.add_argument('-e','--exchange',

type=str,

required=True,

help='The exchange to download from')

parser.add_argument('-t','--timeframe',

type=str,

default='1d',

choices=['1m', '5m','15m', '30m','1h', '2h', '3h', '4h', '6h', '12h', '1d', '1M', '1y'],

help='The timeframe to download')

parser.add_argument('--debug',

action ='store_true',

help=('Print Sizer Debugs'))

return parser.parse_args()

# Get our arguments

args = parse_args()

# Get our Exchange

try:

exchange = getattr (ccxt, args.exchange) ()

except AttributeError:

print('-'*36,' ERROR ','-'*35)

print('Exchange "{}" not found. Please check the exchange is supported.'.format(args.exchange))

print('-'*80)

quit()

# Check if fetching of OHLC Data is supported

if exchange.has["fetchOHLCV"] == False:

print('-'*36,' ERROR ','-'*35)

print('{} does not support fetching OHLC data. Please use another exchange'.format(args.exchange))

print('-'*80)

quit()

# Check requested timeframe is available. If not return a helpful error.

if args.timeframe not in exchange.timeframes:

print('-'*36,' ERROR ','-'*35)

print('The requested timeframe ({}) is not available from {}\n'.format(args.timeframe,args.exchange))

print('Available timeframes are:')

for key in exchange.timeframes.keys():

print(' - ' + key)

print('-'*80)

quit()

# Check if the symbol is available on the Exchange

exchange.load_markets()

if args.symbol not in exchange.symbols:

print('-'*36,' ERROR ','-'*35)

print('The requested symbol ({}) is not available from {}\n'.format(args.symbol,args.exchange))

print('Available symbols are:')

for key in exchange.symbols:

print(' - ' + key)

print('-'*80)

quit()

# Get data

data = exchange.fetch_ohlcv(args.symbol, args.timeframe)

header = ['Timestamp', 'Open', 'High', 'Low', 'Close', 'Volume']

df = pd.DataFrame(data, columns=header).set_index('Timestamp')

# Save it

symbol_out = args.symbol.replace("/","")

filename = '{}-{}-{}.csv'.format(args.exchange, symbol_out,args.timeframe)

df.to_csv(filename)

The code is quite straightforward. After using argparseto capture our exchange, instrument and timezone requirements, the bulk of the work is spent error checking before and calling CCXT’s fetch_ohlcvmethod.

Speaking of error checking, not all exchanges support fetching of OHLCV data. Furthermore, not all exchanges support the same timeframes and currency pairs. Since there are over 100 exchanges supported at the time of writing, it is easy to imagine that there will be a lot of variances between exchanges. To avoid frustration with bumping into error after error trying to guess which timeframes and/or currency pairs are supported, the code performs a series of checks before attempting to download data. It detected something is not supported, it will notify the user and provide a list of alternative parameters where applicable.

Other than that, it is worth paying attention to the exchange.load_markets()call if you plan to expand upon the code or work with the library yourself. Admittedly, I spent a little too much some time scratching my head, trying to figure out why every exchange I tested seemed to have empty markets, symbolsand markets_by_iddictionaries. It turns out that snuggled in the documentation is the answer, you need to call load_markets()before any data is available. This was a good reminder to read the documentation carefully! A little time spent reading can save a lot of time.

Finally, if you are new to programming, the getattr(ccxt, args.exchange)might be new to you. getattr()is a python built-in function which allows you to “get an attribute” from an object using a string. In this case, we are getting the correct exchange object from the given argument string.

Reference: https://docs.python.org/3/library/functions.html#getattr

Usage Examples

Please note that the following usage examples assume the script is saved as ccxt_market_data.py.

ETH/NEO Binance Daily

python .\ccxt_market_data.py -s NEO/ETH -e binance -t 1d

BTC/USD

Unsupported Timeframe

The example demonstrates the expected result when an unsupported timeframe is given as an argument:

python .\ccxt_market_data.py -s NEO/ETH -e bittrex -t 12h

------------------------------------  ERROR  -----------------------------------
The requested timeframe (12h) is not available from bittrex

Available timeframes are:
  - 1m
  - 5m
  - 30m
  - 1h
  - 1d
--------------------------------------------------------------------------------

------------------------------------ ERROR -----------------------------------

The requested timeframe (12h) is not available from bittrex

Available timeframes are:

- 1m

- 5m

- 30m

- 1h

- 1d

--------------------------------------------------------------------------------

Unsupported Symbol

The second example has a fair bit of output. It demonstrates the expected result when an unsupported symbol is given as an argument:

python .\ccxt_market_data.py -s ATP/USD -e bittrex -t 5m

------------------------------------  ERROR  -----------------------------------
The requested symbol (ATP/USD) is not available from bittrex

Available symbols are:
  - 1ST/BTC
  - 1ST/ETH
  - 2GIVE/BTC
  - ABY/BTC
  - ADA/BTC
  - ADA/ETH
  - ADA/USDT
  - ADT/BTC
  - ADT/ETH
  - ADX/BTC
  - ADX/ETH
  - AEON/BTC
  - AGRS/BTC
  - AMP/BTC
  - ANT/BTC
  - ANT/ETH
  - ARDR/BTC
  - ARK/BTC
  - AUR/BTC
  - BAT/BTC
  - BAT/ETH
  - BAY/BTC
  - BCH/BTC
  - BCH/ETH
  - BCH/USDT
  - BCPT/BTC
  - BCPT/ETH
  - BCY/BTC
  - BITB/BTC
  - BLITZ/BTC
  - BLK/BTC
  - BLOCK/BTC
  - BNT/BTC
  - BNT/ETH
  - BRK/BTC
  - BRX/BTC
  - BSD/BTC
  - BTC/USDT
  - BTG/BTC
  - BTG/ETH
  - BTG/USDT
  - BURST/BTC
  - BYC/BTC
  - CANN/BTC
  - CFI/BTC
  - CFI/ETH
  - CLAM/BTC
  - CLOAK/BTC
  - CLUB/BTC
  - COVAL/BTC
  - CPC/BTC
  - CRB/BTC
  - CRB/ETH
  - CRW/BTC
  - CURE/BTC
  - CVC/BTC
  - CVC/ETH
  - DASH/BTC
  - DASH/ETH
  - DASH/USDT
  - DCR/BTC
  - DCT/BTC
  - DGB/BTC
  - DGB/ETH
  - DMD/BTC
  - DNT/BTC
  - DNT/ETH
  - DOGE/BTC
  - DOPE/BTC
  - DTB/BTC
  - DYN/BTC
  - EBST/BTC
  - EDG/BTC
  - EFL/BTC
  - EGC/BTC
  - EMC/BTC
  - EMC2/BTC
  - ENG/BTC
  - ENG/ETH
  - ENRG/BTC
  - ERC/BTC
  - ETC/BTC
  - ETC/ETH
  - ETC/USDT
  - ETH/BTC
  - ETH/USDT
  - EXCL/BTC
  - EXP/BTC
  - FAIR/BTC
  - FCT/BTC
  - FCT/ETH
  - FLDC/BTC
  - FLO/BTC
  - FTC/BTC
  - GAM/BTC
  - GAME/BTC
  - GBG/BTC
  - GBYTE/BTC
  - GCR/BTC
  - GEO/BTC
  - GLD/BTC
  - GNO/BTC
  - GNO/ETH
  - GNT/BTC
  - GNT/ETH
  - GOLOS/BTC
  - GRC/BTC
  - GRS/BTC
  - GUP/BTC
  - GUP/ETH
  - HMQ/BTC
  - HMQ/ETH
  - IGNIS/BTC
  - INCNT/BTC
  - INFX/BTC
  - IOC/BTC
  - ION/BTC
  - IOP/BTC
  - KMD/BTC
  - KORE/BTC
  - LBC/BTC
  - LGD/BTC
  - LGD/ETH
  - LMC/BTC
  - LSK/BTC
  - LTC/BTC
  - LTC/ETH
  - LTC/USDT
  - LUN/BTC
  - LUN/ETH
  - MAID/BTC
  - MANA/BTC
  - MANA/ETH
  - MCO/BTC
  - MCO/ETH
  - MEME/BTC
  - MER/BTC
  - MLN/BTC
  - MONA/BTC
  - MUE/BTC
  - MUSIC/BTC
  - NAV/BTC
  - NBT/BTC
  - NEO/BTC
  - NEO/ETH
  - NEO/USDT
  - NEOS/BTC
  - NLG/BTC
  - NMR/BTC
  - NMR/ETH
  - NXC/BTC
  - NXS/BTC
  - NXT/BTC
  - NXT/USDT
  - OK/BTC
  - OMG/BTC
  - OMG/ETH
  - OMG/USDT
  - OMNI/BTC
  - PART/BTC
  - PAY/BTC
  - PAY/ETH
  - PDC/BTC
  - PINK/BTC
  - PIVX/BTC
  - PKB/BTC
  - POT/BTC
  - POWR/BTC
  - POWR/ETH
  - PPC/BTC
  - PTC/BTC
  - PTOY/BTC
  - PTOY/ETH
  - QRL/BTC
  - QRL/ETH
  - QTUM/BTC
  - QTUM/ETH
  - QWARK/BTC
  - RADS/BTC
  - RBY/BTC
  - RCN/BTC
  - RCN/ETH
  - RDD/BTC
  - REP/BTC
  - REP/ETH
  - RLC/BTC
  - RLC/ETH
  - SALT/BTC
  - SALT/ETH
  - SBD/BTC
  - SC/BTC
  - SC/ETH
  - SEQ/BTC
  - SHIFT/BTC
  - SIB/BTC
  - SLR/BTC
  - SLS/BTC
  - SNRG/BTC
  - SNT/BTC
  - SNT/ETH
  - SPHR/BTC
  - SPR/BTC
  - SRN/BTC
  - SRN/ETH
  - START/BTC
  - STEEM/BTC
  - STORJ/BTC
  - STORJ/ETH
  - STRAT/BTC
  - STRAT/ETH
  - SWIFT/BTC
  - SWT/BTC
  - SYNX/BTC
  - SYS/BTC
  - THC/BTC
  - TIX/BTC
  - TIX/ETH
  - TKS/BTC
  - TRST/BTC
  - TRST/ETH
  - TRUST/BTC
  - TRX/BTC
  - TRX/ETH
  - TUSD/BTC
  - TX/BTC
  - UBQ/BTC
  - UKG/BTC
  - UKG/ETH
  - UNB/BTC
  - VEE/BTC
  - VEE/ETH
  - VIA/BTC
  - VIB/BTC
  - VIB/ETH
  - VOX/BTC
  - VRC/BTC
  - VRM/BTC
  - VTC/BTC
  - VTR/BTC
  - WAVES/BTC
  - WAVES/ETH
  - WAX/BTC
  - WAX/ETH
  - WINGS/BTC
  - WINGS/ETH
  - XCP/BTC
  - XDN/BTC
  - XEL/BTC
  - XEM/BTC
  - XEM/ETH
  - XLM/BTC
  - XLM/ETH
  - XMG/BTC
  - XMR/BTC
  - XMR/ETH
  - XMR/USDT
  - XMY/BTC
  - XRP/BTC
  - XRP/ETH
  - XRP/USDT
  - XST/BTC
  - XVC/BTC
  - XVG/BTC
  - XVG/USDT
  - XWC/BTC
  - XZC/BTC
  - ZCL/BTC
  - ZEC/BTC
  - ZEC/ETH
  - ZEC/USDT
  - ZEN/BTC
  - ZRX/BTC
  - ZRX/ETH
--------------------------------------------------------------------------------

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

------------------------------------ ERROR -----------------------------------

The requested symbol (ATP/USD) is not available from bittrex

Available symbols are:

- 1ST/BTC

- 1ST/ETH

- 2GIVE/BTC

- ABY/BTC

- ADA/BTC

- ADA/ETH

- ADA/USDT

- ADT/BTC

- ADT/ETH

- ADX/BTC

- ADX/ETH

- AEON/BTC

- AGRS/BTC

- AMP/BTC

- ANT/BTC

- ANT/ETH

- ARDR/BTC

- ARK/BTC

- AUR/BTC

- BAT/BTC

- BAT/ETH

- BAY/BTC

- BCH/BTC

- BCH/ETH

- BCH/USDT

- BCPT/BTC

- BCPT/ETH

- BCY/BTC

- BITB/BTC

- BLITZ/BTC

- BLK/BTC

- BLOCK/BTC

- BNT/BTC

- BNT/ETH

- BRK/BTC

- BRX/BTC

- BSD/BTC

- BTC/USDT

- BTG/BTC

- BTG/ETH

- BTG/USDT

- BURST/BTC

- BYC/BTC

- CANN/BTC

- CFI/BTC

- CFI/ETH

- CLAM/BTC

- CLOAK/BTC

- CLUB/BTC

- COVAL/BTC

- CPC/BTC

- CRB/BTC

- CRB/ETH

- CRW/BTC

- CURE/BTC

- CVC/BTC

- CVC/ETH

- DASH/BTC

- DASH/ETH

- DASH/USDT

- DCR/BTC

- DCT/BTC

- DGB/BTC

- DGB/ETH

- DMD/BTC

- DNT/BTC

- DNT/ETH

- DOGE/BTC

- DOPE/BTC

- DTB/BTC

- DYN/BTC

- EBST/BTC

- EDG/BTC

- EFL/BTC

- EGC/BTC

- EMC/BTC

- EMC2/BTC

- ENG/BTC

- ENG/ETH

- ENRG/BTC

- ERC/BTC

- ETC/BTC

- ETC/ETH

- ETC/USDT

- ETH/BTC

- ETH/USDT

- EXCL/BTC

- EXP/BTC

- FAIR/BTC

- FCT/BTC

- FCT/ETH

- FLDC/BTC

- FLO/BTC

- FTC/BTC

- GAM/BTC

- GAME/BTC

- GBG/BTC

- GBYTE/BTC

- GCR/BTC

- GEO/BTC

- GLD/BTC

- GNO/BTC

- GNO/ETH

- GNT/BTC

- GNT/ETH

- GOLOS/BTC

- GRC/BTC

- GRS/BTC

- GUP/BTC

- GUP/ETH

- HMQ/BTC

- HMQ/ETH

- IGNIS/BTC

- INCNT/BTC

- INFX/BTC

- IOC/BTC

- ION/BTC

- IOP/BTC

- KMD/BTC

- KORE/BTC

- LBC/BTC

- LGD/BTC

- LGD/ETH

- LMC/BTC

- LSK/BTC

- LTC/BTC

- LTC/ETH

- LTC/USDT

- LUN/BTC

- LUN/ETH

- MAID/BTC

- MANA/BTC

- MANA/ETH

- MCO/BTC

- MCO/ETH

- MEME/BTC

- MER/BTC

- MLN/BTC

- MONA/BTC

- MUE/BTC

- MUSIC/BTC

- NAV/BTC

- NBT/BTC

- NEO/BTC

- NEO/ETH

- NEO/USDT

- NEOS/BTC

- NLG/BTC

- NMR/BTC

- NMR/ETH

- NXC/BTC

- NXS/BTC

- NXT/BTC

- NXT/USDT

- OK/BTC

- OMG/BTC

- OMG/ETH

- OMG/USDT

- OMNI/BTC

- PART/BTC

- PAY/BTC

- PAY/ETH

- PDC/BTC

- PINK/BTC

- PIVX/BTC

- PKB/BTC

- POT/BTC

- POWR/BTC

- POWR/ETH

- PPC/BTC

- PTC/BTC

- PTOY/BTC

- PTOY/ETH

- QRL/BTC

- QRL/ETH

- QTUM/BTC

- QTUM/ETH

- QWARK/BTC

- RADS/BTC

- RBY/BTC

- RCN/BTC

- RCN/ETH

- RDD/BTC

- REP/BTC

- REP/ETH

- RLC/BTC

- RLC/ETH

- SALT/BTC

- SALT/ETH

- SBD/BTC

- SC/BTC

- SC/ETH

- SEQ/BTC

- SHIFT/BTC

- SIB/BTC

- SLR/BTC

- SLS/BTC

- SNRG/BTC

- SNT/BTC

- SNT/ETH

- SPHR/BTC

- SPR/BTC

- SRN/BTC

- SRN/ETH

- START/BTC

- STEEM/BTC

- STORJ/BTC

- STORJ/ETH

- STRAT/BTC

- STRAT/ETH

- SWIFT/BTC

- SWT/BTC

- SYNX/BTC

- SYS/BTC

- THC/BTC

- TIX/BTC

- TIX/ETH

- TKS/BTC

- TRST/BTC

- TRST/ETH

- TRUST/BTC

- TRX/BTC

- TRX/ETH

- TUSD/BTC

- TX/BTC

- UBQ/BTC

- UKG/BTC

- UKG/ETH

- UNB/BTC

- VEE/BTC

- VEE/ETH

- VIA/BTC

- VIB/BTC

- VIB/ETH

- VOX/BTC

- VRC/BTC

- VRM/BTC

- VTC/BTC

- VTR/BTC

- WAVES/BTC

- WAVES/ETH

- WAX/BTC

- WAX/ETH

- WINGS/BTC

- WINGS/ETH

- XCP/BTC

- XDN/BTC

- XEL/BTC

- XEM/BTC

- XEM/ETH

- XLM/BTC

- XLM/ETH

- XMG/BTC

- XMR/BTC

- XMR/ETH

- XMR/USDT

- XMY/BTC

- XRP/BTC

- XRP/ETH

- XRP/USDT

- XST/BTC

- XVC/BTC

- XVG/BTC

- XVG/USDT

- XWC/BTC

- XZC/BTC

- ZCL/BTC

- ZEC/BTC

- ZEC/ETH

- ZEC/USDT

- ZEN/BTC

- ZRX/BTC

- ZRX/ETH

--------------------------------------------------------------------------------

Unsupported Exachange

The final example demostrates the expected result when an exchange does not support fethcing of OHLCV data.

python .\ccxt_market_data.py -s BTC/USD -e gdax -t 15m

------------------------------------  ERROR  -----------------------------------
gdax does not support fetching OHLC data. Please use another exchange
--------------------------------------------------------------------------------

------------------------------------ ERROR -----------------------------------

gdax does not support fetching OHLC data. Please use another exchange

--------------------------------------------------------------------------------

Issues and Improvements

The code tries to be user-friendly in that it will tell the user when incorrect data is provided. Having said that, not all edge cases have been considered and there is definitely room for improvement in that regard.

In addition to this, the amount of data available on most exchanges for download is limited. CCXT does provide a sinceparameter for the fetchOHLCV()method but I did not see any detailed description for this in the fetchOHLCV()documentation. Having said that, an explanation of a parameter with the same name does appear the “Trades, Executions and Transactions” section as follows (Note: That the parameter appears frequently, so I assume it will have consistent functionality but that is no guarantee!):

The second optional argument since reduces the array by timestamp, the third limit argument reduces by number (count) of returned items.

Reference: https://github.com/ccxt/ccxt/wiki/Manual#trades-executions-transactions

So taking that into account, it appears sinceshould reduce the amount of data provided. However, it would be good to know for sure. If anyone pokes through the codebase and has an answer, I would be happy to hear it. Leave a comment below!

7 Responses

Jaspal March 17, 2018 at 2:26 am Reply

Hi,

many thanks for the tutorial.

I am trying to import the data into your optimise strategy which I have modified.

However it does not seem to work.

How would you modify your optimise example to include ccxt?

Rookie1 March 17, 2018 at 9:06 pm Reply

Hi Jaspal,
It is hard to say without seeing the strategy. However, one thing I can think of is to check the way you are adding the data. You will need to import the data downloaded as CSV and make sure the columns are mapped correctly.

Try taking a look at the GenericCSV docs here:

https://www.backtrader.com/docu/datafeed.html#genericcsvdata

Jaspal March 22, 2018 at 3:45 am Reply

Hi, I managed to figure it out. I had to declare the format of the csv file in my code with:

# CSV INPUT FILE FORMAT CONFIGURATION
class dataFeed(btfeed.GenericCSVData):
params = (
(‘dtformat’, ‘%Y-%m-%d %H:%M:%S’),
(‘datetime’, 0),
(‘open’, 1),
(‘high’, 2),
(‘low’, 3),
(‘close’, 4),
(‘volume’, 5),
(‘openinterest’, -1)
)

Thanks again!

Rookie1 March 22, 2018 at 9:03 am Reply

Awesome!

gito May 9, 2018 at 4:58 pm Reply

@Jaspal : exactly what I was looking for! thank you
Jorge May 12, 2018 at 5:28 am Reply

Amazing! Thanks a lot for sharing this post, it´s really useful

Another thing…watching the CSV file generated with the call..the first column corresponding to Timestamp…it would be better if we change the time format using datetime in order to get something readable.
Some May 29, 2018 at 10:28 pm Reply

Hello,

Thanks for the great code.
I thought I’d share, with all teh other noobs, a problem I had.

… If you name this code ccxt.py, specifically, it will not fire.
It will come back with an “Exchange “bitfinex” not found. Please check the exchange is supported.” error, no matter what exchange is entered.

Renaming the file to any name other than ccxt.py will solve this issue.

Some Guy

Download Cryptocurrency Data with CCXT

Installing the Library

Other Dependencies

Download Tool Overview

The Code

Usage Examples

Unsupported Timeframe

Unsupported Symbol

Unsupported Exachange

Issues and Improvements

Related

7 Responses

Leave a Reply

Installing the Library

Other Dependencies

Download Tool Overview

The Code

Usage Examples

Unsupported Timeframe

Unsupported Symbol

Unsupported Exachange

Issues and Improvements

Share this:

Related

7 Responses

Leave a Reply