前言
Elasticsearch 是一个开源的分布式 RESTful 搜索和分析引擎。它可以在近实时条件下，存储，查询和分析海量的数据。它还支持将快照备份至HDFS/S3上面，而阿里云OSS兼容S3的API，本文将介绍如何使用ES的Repository-S3插件将快照备份至OSS。
部署与配置
首先，我们需要安装repository-s3，可以参考官方文档：
https://www.elastic.co/guide/en/elasticsearch/plugins/7.2/repository-s3.html
启动ES，我们可以从log中看到，ES已经load了这个plugin：
[2019-07-15T14:12:09,225][INFO ][o.e.p.PluginsService ] [master] loaded module [aggs-matrix-stats][2019-07-15T14:12:09,225][INFO ][o.e.p.PluginsService ] [master] loaded module [analysis-common][2019-07-15T14:12:09,225][INFO ][o.e.p.PluginsService ] [master] loaded module [ingest-common][2019-07-15T14:12:09,226][INFO ][o.e.p.PluginsService ] [master] loaded module [ingest-geoip][2019-07-15T14:12:09,226][INFO ][o.e.p.PluginsService ] [master] loaded module [ingest-user-agent][2019-07-15T14:12:09,226][INFO ][o.e.p.PluginsService ] [master] loaded module [lang-expression][2019-07-15T14:12:09,226][INFO ][o.e.p.PluginsService ] [master] loaded module [lang-mustache][2019-07-15T14:12:09,227][INFO ][o.e.p.PluginsService ] [master] loaded module [lang-painless][2019-07-15T14:12:09,227][INFO ][o.e.p.PluginsService ] [master] loaded module [mapper-extras][2019-07-15T14:12:09,227][INFO ][o.e.p.PluginsService ] [master] loaded module [parent-join][2019-07-15T14:12:09,227][INFO ][o.e.p.PluginsService ] [master] loaded module [percolator][2019-07-15T14:12:09,227][INFO ][o.e.p.PluginsService ] [master] loaded module [rank-eval][2019-07-15T14:12:09,228][INFO ][o.e.p.PluginsService ] [master] loaded module [reindex][2019-07-15T14:12:09,228][INFO ][o.e.p.PluginsService ] [master] loaded module [repository-url][2019-07-15T14:12:09,228][INFO ][o.e.p.PluginsService ] [master] loaded module [transport-netty4][2019-07-15T14:12:09,228][INFO ][o.e.p.PluginsService ] [master] loaded plugin [repository-s3][2019-07-15T14:12:12,375][INFO ][o.e.d.DiscoveryModule ] [master] using discovery type [zen] and seed hosts providers [settings][2019-07-15T14:12:12,801][INFO ][o.e.n.Node ] [master] initialized[2019-07-15T14:12:12,802][INFO ][o.e.n.Node ] [master] starting ...然后，我们需要将OSS使用的Access Key和Secret Key配置到ES去，分别执行下面的命令：
bin/elasticsearch-keystore add s3.client.default.access_keybin/elasticsearch-keystore add s3.client.default.secret_key运行
首先，我们创建一个备份：
[root@master ~]# curl -XPUT 'http://localhost:9200/_snapshot/test' -H 'Content-Type: application/json' -d '{ "type": "s3", "settings": { "bucket": "hadoop-oss-test", "endpoint": "oss-cn-zhangjiakou-internal.aliyuncs.com"} }'{"acknowledged":true}NOTE: 上面的命令默认使用https协议来传输数据，如果想使用http协议，需要将"protocol": "http", "disable_chunked_encoding": true加到settings里面(这个特性将会在新版本发布后可用)。
可以使用下面的命令来确实创建是否成功：
[root@master ~]# curl -XGET localhost:9200/_snapshot/test?pretty{ "test" : { "type" : "s3", "settings" : { "bucket" : "hadoop-oss-test", "endpoint" : "oss-cn-zhangjiakou-internal.aliyuncs.com" } }}我们可以写入一些测试数据到ES，然后看下目前集群的索引信息:
[root@master ~]# curl -X GET "localhost:9200/_cat/indices?v"health status index uuid pri rep docs.count docs.deleted store.size pri.store.sizegreen open sales 89ouBy6RQsuT34QRbn_jeQ 10 0 271786 0 15mb 15mbgreen open customer fQCMEvXsQOu0UgMm1SAJlA 5 0 10000 0 717kb 717kb假设我们只备份sales索引：
[root@master ~]# curl -XPUT 'http://localhost:9200/_snapshot/test/sales' -H 'Content-Type: application/json' -d '{ "indices": "sales" }'{"accepted":true}然后我们可以从OSS控制台看到备份的结果：
现在我们再往sales索引里面写一些数据：
[root@master ~]# curl -X GET "localhost:9200/_cat/indices?v"health status index uuid pri rep docs.count docs.deleted store.size pri.store.sizegreen open sales 89ouBy6RQsuT34QRbn_jeQ 10 0 281502 0 15.6mb 15.6mbgreen open customer fQCMEvXsQOu0UgMm1SAJlA 5 0 10000 0 717kb 717kb我们利用刚才备份到OSS的快照来恢复sales索引，分别执行下面的命令：
[root@master ~]# curl -XPOST localhost:9200/sales/_close{"acknowledged":true,"shards_acknowledged":true,"indices":{"sales":{"closed":true}}}[root@master ~]# curl -XPOST 'http://localhost:9200/_snapshot/test/sales/_restore?pretty'{ "accepted" : true}[root@master ~]# curl -X GET "localhost:9200/_cat/indices?v"health status index uuid pri rep docs.count docs.deleted store.size pri.store.sizegreen open sales 89ouBy6RQsuT34QRbn_jeQ 10 0 271786 0 15mb 15mbgreen open customer fQCMEvXsQOu0UgMm1SAJlA 5 0 10000 0 717kb 717kb我们可以看到，sales索引跟之前的一致。
参考资料
https://www.elastic.co/guide/en/elasticsearch/plugins/7.2/repository-s3.html
https://www.elastic.co/cn/products/elasticsearch
本文作者：冷月_wjh
阅读原文
本文为云栖社区原创内容，未经允许不得转载。