使用cos备份迁移es数据，新集群restore报错snapshot_restore_exception

原创

WeldonWang

修改于 2023-03-31 13:50:16

7650

文章被收录于专栏：腾讯云大数据与AI专家服务腾讯云大数据与AI专家服务

报错赏析

原因分析

需要在原集群创建好 repository 和 snapshot 后
再去新集群创建相同的 repository，才可以看到对应的 snapshot
如果遇到了以上问题，请严格按照下面的步骤来执行，切勿嫌麻烦

一、源集群创建仓库

PUT _snapshot/my_cos_backup
{
    "type": "cos",
    "settings": {
        "app_id": "xxxxxxx",
        "access_key_id": "xxxxxx",
        "access_key_secret": "xxxxxxx",
        "bucket": "xxxxxx",
        "region": "ap-guangzhou",
        "compress": true,
        "chunk_size": "500mb",
        "base_path": "/my_cos_backup",
        "max_restore_bytes_per_sec":"40m",
        "max_snapshot_bytes_per_sec":"40m"
    }
}

bucket：COS Bucket Name，不能带 "-{appId}" 后缀。

二、创建快照

PUT _snapshot/my_cos_backup/snapshot_1
{
  "indices":"*,-.*,-ilm-*"
}

这里 indices 取值可以为单个索引 name，也可以使用通配符，多个value之间用英文逗号分割

三、目标集群创建相同仓库

PUT _snapshot/my_cos_backup
{
    "type": "cos",
    "settings": {
        "app_id": "xxxxxxx",
        "access_key_id": "xxxxxx",
        "access_key_secret": "xxxxxxx",
        "bucket": "xxxxxx",
        "region": "ap-guangzhou",
        "compress": true,
        "chunk_size": "500mb",
        "base_path": "/my_cos_backup",
        "max_restore_bytes_per_sec":"40m",
        "max_snapshot_bytes_per_sec":"40m"
    }
}

bucket：COS Bucket Name，不能带 "-{appId}" 后缀。

四、目标集群恢复快照

POST _snapshot/my_cos_backup/snapshot_1/_restore

# 如果只想恢复部分index，也可以
POST _snapshot/my_cos_backup/snapshot_1/_restore
{
  "indices":"*,-.*,-ilm-*"
}

同第二步，indices 取值可以为单个索引 name，也可以使用通配符，多个value之间用英文逗号分割

其他操作

# 获取仓库列表
GET _snapshot
# 获取快照列表
GET _cat/snapshots?v
# 获取快照详情
GET _snapshot/my_cos_backup/snapshot_1
# 获取snapshot开头的快照，且按照name排序并只返回第一个
GET /_snapshot/my_cos_backup/snapshot*?size=1&sort=name
# 分页查询，使用上一步返回的next值继续查看剩余snapshot开头的快照
GET /_snapshot/my_cos_backup/snapshot*?size=1&sort=name&after=c25hcHNob3RfMSxteV9jb3NfYmFja3VwLHNuYXBzaG90XzE=
# 也可使用偏移量
GET /_snapshot/my_cos_backup/snapshot*?size=2&sort=name&offset=2
# 前缀匹配并T除不需要的snapshot，使用 "-" 来排除
GET /_snapshot/my_cos_backup/snapshot*,-snapshot_3?sort=name
# 获取排序后"snapshot_1"之后的snapshot
GET /_snapshot/my_cos_backup/*?sort=name&from_sort_value=snapshot_1

# 获取快照状态
GET _snapshot/_status
GET _snapshot/<repository>/_status
GET _snapshot/<repository>/<snapshot>/_status

# 恢复快照，根据情况自行选择所需参数
POST /_snapshot/my_cos_backup/snapshot_1/_restore?wait_for_completion=true
{
  "indices": "index_1,index_2",
  "ignore_index_settings":[
    "index.codec",
    "index.vector.algorithm"
  ],
  "ignore_unavailable": true,
  "include_global_state": false,
  "rename_pattern": "index_(.+)",
  "rename_replacement": "restored_index_$1",
  "include_aliases": false
}

# 删除仓库
DELETE /_snapshot/my_cos_backup
# 删除快照
DELETE /_snapshot/my_cos_backup/snapshot_1