参考资料
https://github.com/alibaba/Da...
mysqlwriter
1. 简介
DataX 是阿里巴巴团体内被宽泛应用的离线数据同步工具/平台,实现包含 MySQL、SQL Server、Oracle、PostgreSQL、HDFS、Hive、HBase、OTS、ODPS 等各种异构数据源之间高效的数据同步性能。
2. 装置
2.1 linux 下装置jdk并且配置环境变量
https://blog.csdn.net/xiaogua...
2.2 linux 下装置python2.7并且配置环境变量
https://www.python.org/downlo...
2.3 装置DataX
DataX下载地址
下载后解压至本地某个目录,进入bin目录,即可运行同步作业:
$ cd {YOUR_DATAX_HOME}/bin$ python datax.py {YOUR_JOB.json}
自检脚本:
python {YOUR_DATAX_HOME}/bin/datax.py {YOUR_DATAX_HOME}/job/job.json
3. sql server 同步到mysql
次要配置{YOUR_DATAX_HOME}/job/job.json下的json文件
{ "job": { "setting": { "speed": { "byte":10485760 }, "errorLimit": { "record": 0, "percentage": 0.02 } }, "content": [ { "reader": { "name": "sqlserverreader", "parameter": { "username": "sa", "password": "123456", "connection": [ { "jdbcUrl": [ "jdbc:sqlserver://10.0.201.255:1433;database=wzz" ], "querySql": [ "SELECT userid,name,age FROM dbo.users" ] } ] } }, "writer": { "name": "mysqlwriter", "parameter": { "username": "lambdacal", "password": "123456", "column": [ "userid", "name", "age" ], "connection": [ { "jdbcUrl": "jdbc:mysql://10.0.0.0:3306/wzz_db?charset=utf8", "table": [ "users" ] } ] } } } ] }}
执行
python ./datax/bin/datax.py ./datax/job/job.json