对加密货币盘口与逐笔交易数据的回放展现,可帮忙量化钻研人员测验量化策略,也有助于交易员复盘,加深对市场的洞察。DolphinDB可实现盘口和逐笔交易数据的高速回放,以及对回放后果逐点查问。
DolphinDB database反对将多个分布式表同步回放并公布到流数据表,例如对盘口和交易这两个表进行同步回放。前端JavaScript应用DolphinDB Web API来轮询回放输入的流数据表,实现盘口和交易数据的可视化回放。DolphinDB自带Web服务器,整个流程可在DolphinDB内实现,无内部依赖。
加密货币盘口与逐笔交易数据回放可通过以下4个步骤来实现。用户亦可应用docker疾速体验回放性能,具体请参考文末介绍。
部署DolphinDB节点到官网下载DolphinDB最新版本,并部署集群。部署教程请参考单服务器集群部署教程。
下载盘口和逐笔交易数据本文应用的是火币研究院提供的加密货币交易数据,能够通过火币数据API获取。获取数据的示例代码能够参考python示例代码或java示例代码。
导入数据到DolphinDB本文将获取的orderBook的tick级数据保留为csv文件,通过loadTextEx函数疾速地将文件导入到数据库。用户也能够通过Python API或Java API将数据导入到DolphinDB中。以下代码在DolphinDB GUI中执行。
(1)数据预处理
如果保留的csv文件中第一行是无关信息,能够采纳上面脚本进行数据预处理,解决好的文件保留到某个目录,本案例将两个文件别离保留到/hdd/data/orderBook-processed和/hdd/data/tick-processes目录。如果csv文件第一行没有无关信息,可疏忽这一步骤。
//删除数据文件第一行无关信息def dataPreProcess(DIR){ if(!exists(DIR+ "-processed/")) mkdir(DIR+ "-processed/") fileList = exec filename from files(DIR) where isDir = false, filename like "%.csv" for(filename in fileList){ f = file(DIR + "/" + filename) y = f.readLines(1000000).removeHead!(1) saveText(y, DIR+ "-processed/" + filename) }}dataPreProcess("/hdd/data/orderBook")dataPreProcess("/hdd/data/tick")(2)创立DolphinDB数据库
依据数据量以及查问字段,数据库可依照交易标的代码和业务工夫进行组合分区。本案例中,数据库的名称为dfs://huobiDB。如果须要批改,必须同时批改replay.html中数据库的名称。
def createDB(){ if(existsDatabase("dfs://huobiDB")) dropDatabase("dfs://huobiDB") //依照数据集的时间跨度,请自行调整VALUE分区日期范畴 db1 = database(, VALUE, 2018.09.01..2018.09.30) db2 = database(, HASH, [SYMBOL,20]) db = database("dfs://huobiDB", COMPO, [db1,db2])}def createTick(){ tick = table(100:0, `aggregate_ID`server_time`price`amount`buy_or_sell`first_trade_ID`last_trade_ID`product , [INT,TIMESTAMP,DOUBLE,DOUBLE,CHAR,INT,INT,SYMBOL]) db = database("dfs://huobiDB") return db.createPartitionedTable(tick, `tick, `server_time`product)}def createOrderBook(){ orderData = table(100:0, `lastUpdateId`server_time`buy_1_price`buy_2_price`buy_3_price`buy_4_price`buy_5_price`buy_6_price`buy_7_price`buy_8_price`buy_9_price`buy_10_price`buy_11_price`buy_12_price`buy_13_price`buy_14_price`buy_15_price`buy_16_price`buy_17_price`buy_18_price`buy_19_price`buy_20_price`sell_1_price`sell_2_price`sell_3_price`sell_4_price`sell_5_price`sell_6_price`sell_7_price`sell_8_price`sell_9_price`sell_10_price`sell_11_price`sell_12_price`sell_13_price`sell_14_price`sell_15_price`sell_16_price`sell_17_price`sell_18_price`sell_19_price`sell_20_price`buy_1_amount`buy_2_amount`buy_3_amount`buy_4_amount`buy_5_amount`buy_6_amount`buy_7_amount`buy_8_amount`buy_9_amount`buy_10_amount`buy_11_amount`buy_12_amount`buy_13_amount`buy_14_amount`buy_15_amount`buy_16_amount`buy_17_amount`buy_18_amount`buy_19_amount`buy_20_amount`sell_1_amount`sell_2_amount`sell_3_amount`sell_4_amount`sell_5_amount`sell_6_amount`sell_7_amount`sell_8_amount`sell_9_amount`sell_10_amount`sell_11_amount`sell_12_amount`sell_13_amount`sell_14_amount`sell_15_amount`sell_16_amount`sell_17_amount`sell_18_amount`sell_19_amount`sell_20_amount`product,[INT,TIMESTAMP,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,SYMBOL]) db = database("dfs://huobiDB") return db.createPartitionedTable(orderData, `orderBook, `server_time`product)} (3)将文本数据导入数据库
...