最近有个项目需要将es 3.0T的数据从腾讯云迁移到线下，带宽只有100M小水管，时间为一周，总是就是干，干就完了

首先遇到的就是问题就是网络，网络通过ipsec打通，这样可以与本地环境的跳板机直直连，架构设计就是简单为下图

1,在腾讯云管理平台对象存储里新增存储桶es-hand-bakcup-xxxxxxx

2,在访问管理里，新建用户禁止登陆，权限只对你新增存储桶有读写权限，增加权限
QcloudCOSGetServiceAccess，QcloudCollApiKeyManageAccess，es-backup-qianyi，其中es-backup-qianyi为自定义权限，需要自己设置，如下

{
    "version": "2.0",
    "statement": [
        {
            "effect": "allow",
            "resource": [
                "qcs::cos:ap-beijing:uid/xxxxxxx:es-hand-bakcup-xxxxxxxx/*"
            ],
            "action": [
                "name/cos:*"
            ]
        }
    ]
}

3,在kibana上注册次cos为快照库，并执行备份命令

#注册快照库
PUT _snapshot/my_cos_backup
{
    "type": "cos",
    "settings": {
        "app_id": "xxxxxxx",
        "access_key_id": "xxxxxx",
        "access_key_secret": "xxxxxxx",
        "bucket": "es-hand-bakcup-xxxxxxxx",
        "region": "ap-beijing",
        "compress": true,
        "chunk_size": "500mb",
        "base_path": "/"
    }
}

#执行备份后台模式
PUT _snapshot/my_cos_backup/snapshot_1

#执行备份阻塞模式
PUT _snapshot/my_cos_backup/snapshot_1?wait_for_completion=true

#也可以指定不同的名字
PUT _snapshot/my_cos_backup/snapshot_2

查询单个快照信息：

#GET _snapshot/my_cos_backup/snapshot_1
该命令会返回快照的相关信息：

{
    "snapshots": [
        {
            "snapshot": "snapshot_1",
            "uuid": "zUSugNiGR-OzH0CCcgcLmQ",
            "version_id": 5060499,
            "version": "5.6.4",
            "indices": [
                "index_1",
                "index_2"
            ],
            "state": "SUCCESS",
            "start_time": "2018-05-04T11:44:15.975Z",
            "start_time_in_millis": 1525434255975,
            "end_time": "2018-05-04T11:45:29.395Z",
            "end_time_in_millis": 1525434329395,
            "duration_in_millis": 73420,
            "failures": [],
            "shards": {
                "total": 3,
                "failed": 0,
                "successful": 3
            }
        }
    ]
}

#删除指定的快照：
DELETE _snapshot/my_cos_backup/snapshot_1

#查看所有快照信息
GET _snapshot/my_cos_backup/_all?pretty

4, 将备份好数据的cos存储通过cosfs的方式挂载到跳板机上CVM上，通过sersync推送到

#软件安装与配置
https://cloud.tencent.com/document/product/436/6883

#挂载磁盘
cosfs es-hand-bakcup-xxxxxxx /es_all_mnt -ourl=https://cos.ap-beijing.myqcloud.com -odbglevel=info -onoxattr -oallow_other

#启动传输
/opt/tools/sersync2.5.4/sersync2 -d -r -o /opt/tools/sersync2.5.4/confxml.xml

5,es配置快照仓库

#快照仓库需要所有节点可以共享到存储，这里通过传统的nfs网络存储实现
yum install nfs-utils -y   #配置/data/snap_data目录并同时挂载到es节点

#在elasticsearch.yml中增加以下配置并重启
path.repo: ["/data/snap_data"]

#在es节点执行注册仓库的命令
curl -XPUT 'http://localhost:9200/_snapshot/my_cos_backup' -H "Content-Type: application/json"  -d 
'{
    "type": "fs",
    "settings": {
        "location": "/data/snap_data",
        "chunk_size": "500mb",
        "compress": true
    }
}'

6,本地环境跳板机通过sersync实时异步同步到快照仓库

启动传输

/opt/tools/sersync2.5.4/sersync2 -d -r -o /opt/tools/sersync2.5.4/confxml.xml

7,在本地es环境上执行恢复命令

#执行恢复索引命令，这里需要注意，仅恢复业务相关的索引即可，如果恢复全部会遇到一些问题，不建议恢复全部
POST _snapshot/my_cos_backup/snapshot_1/_restore' -d '
{
  "indices": "community*,kylm*,prominent*,traffic*",
  "ignore_unavailable": true
}'