[o.e.x.w.e.ExecutionService] [16142xxxxxx20932] could not store triggered watch with id [xxxxx4-2022-07-07T04:04:33.872Z]: [ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]]
当磁盘使用率超过85%,或者达到100%,会导致 Elasticsearch 集群或 Kibana 无法正常提供服务,可能会出现以下几种问题场景:
{[FORBIDDEN/12/index read-only/allow delete(api)];","type":"cluster_block_exception"}
的报错。[FORBIDDEN/13/cluster read-only / allow delete (api)]
的报错。GET _cat/allocation?v
命令查看),并且存在未分配的分片(可通过 GET _cat/allocation?v
命令查看)。上述问题是由于磁盘使用率过高所导致。数据节点的磁盘使用率存在以下三个水位线,超过水位线可能会影响 Elasticsearch 或 Kibana 服务。
用户可以通过访问【Kibana】>【Dev Tools】删除过期索引释放磁盘空间。步骤如下:警告: 数据删除后将无法恢复,请谨慎操作。您也可以选择保留数据,但需进行磁盘扩容。
第一步:开启集群索引批量操作权限。
PUT _cluster/settings { "persistent": { "action.destructive_requires_name": "false" } }
第二步:删除数据,例如 DELETE NginxLog-12*。
DELETE index-name-*
关闭索引只读状态,执行如下命令:
PUT _all/_settings { "index.blocks.read_only_allow_delete": null }
关闭集群只读状态,执行如下命令:
PUT _cluster/settings { "persistent": { "cluster.blocks.read_only_allow_delete": null } }
read_only
状态,索引写入是否恢复正常。https://cloud.tencent.com/document/product/845/56276
# -.- coding:utf-8 -.-
import json
import logging
import sys
from typing import List, Tuple
import requests
def get_nodes_stats(url: str, auth: Tuple[str, str], timeout: int = 20) -> dict:
try:
response = requests.get(f"{url}/_nodes/stats", auth=auth, timeout=timeout)
response.raise_for_status()
return response.json()
except requests.exceptions.RequestException as e:
logging.exception(e)
sys.exit(1)
def print_node_stats(nodes: dict):
for node_id, node in nodes.items():
print(f"NodeName: {node['name']}\tIP: {node['ip']}\tTemperature: {node['attributes']['temperature']}")
for path in node["fs"]["data"]:
print(f"{path['mount']}\ttotal: {path['total_in_bytes']/1024/1024/1024}\tpercent: {1 - path['available_in_bytes'] / path['total_in_bytes']:.2%}")
def main():
url = input("请输入url(http://xxxx:9200): ")
user = input("请输入用户名: ")
passwd = input("请输入密码: ")
auth_info = (user, passwd)
nodes_stats = get_nodes_stats(url, auth_info)
print_node_stats(nodes_stats["nodes"])
if __name__ == '__main__':
logging.basicConfig(level=logging.ERROR)
main()
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。