首页
学习
活动
专区
圈层
工具
发布
首页
学习
活动
专区
圈层
工具
社区首页 >问答首页 >如何获取GKE fluentbit相关错误的详细信息?

如何获取GKE fluentbit相关错误的详细信息?
EN

Stack Overflow用户
提问于 2021-06-07 16:35:45
回答 1查看 565关注 0票数 0

我们刚刚发现堆栈驱动程序中缺少一些日志,我们可以使用kubectl logs列出日志消息,但由于某些原因,其中一些没有发送到堆栈驱动程序日志中。

缺少日志条目的示例:

代码语言:javascript
代码运行次数:0
运行
复制
{"severity":"info","time":"2021-06-07T08:19:17.598Z","caller":"zap/options.go:212","msg":"finished unary call with code OK","grpc.start_time":"2021-06-07T08:19:17Z","system":"grpc","span.kind":"server","grpc.service":"manabie.tom.ChatService","grpc.method":"SendMessage","peer.address":"127.0.0.1:32806","userID":"xxxx","x-request-id":"xxxx","grpc.code":"OK","grpc.time_ms":48.04899978637695}

正在检查fluentbit守护进程:

代码语言:javascript
代码运行次数:0
运行
复制
kubectl logs fluentbit-gke-xxxx -c fluentbit-gke -f --tail=1 

我看到一些错误日志,如:

代码语言:javascript
代码运行次数:0
运行
复制
W0607 08:16:55.066861       1 server.go:77] Received empty or invalid msgpack for tag kube_xxxxxxxx
W0607 08:16:59.072151       1 server.go:77] Received empty or invalid msgpack for tag kube_xxxxxxxx

描述守护进程集:

代码语言:javascript
代码运行次数:0
运行
复制
kubectl describe daemonset fluentbit-gke
Name:           fluentbit-gke
Selector:       component=fluentbit-gke,k8s-app=fluentbit-gke
Node-Selector:  kubernetes.io/os=linux
Labels:         addonmanager.kubernetes.io/mode=Reconcile
                k8s-app=fluentbit-gke
                kubernetes.io/cluster-service=true
Annotations:    deprecated.daemonset.template.generation: 9
Desired Number of Nodes Scheduled: 4
Current Number of Nodes Scheduled: 4
Number of Nodes Scheduled with Up-to-date Pods: 4
Number of Nodes Scheduled with Available Pods: 4
Number of Nodes Misscheduled: 0
Pods Status:  4 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:           component=fluentbit-gke
                    k8s-app=fluentbit-gke
                    kubernetes.io/cluster-service=true
  Annotations:      EnableNodeJournal: false
                    EnablePodSecurityPolicy: false
                    SystemOnlyLogging: false
                    components.gke.io/component-name: fluentbit
                    components.gke.io/component-version: 1.4.4
                    monitoring.gke.io/path: /api/v1/metrics/prometheus
  Service Account:  fluentbit-gke
  Containers:
   fluentbit:
    Image:      gke.gcr.io/fluent-bit:v1.5.7-gke.1
    Port:       2020/TCP
    Host Port:  2020/TCP
    Limits:
      memory:  250Mi
    Requests:
      cpu:        50m
      memory:     100Mi
    Liveness:     http-get http://:2020/ delay=120s timeout=1s period=60s #success=1 #failure=3
    Environment:  <none>
    Mounts:
      /fluent-bit/etc/ from config-volume (rw)
      /var/lib/docker/containers from varlibdockercontainers (ro)
      /var/lib/kubelet/pods from varlibkubeletpods (rw)
      /var/log from varlog (rw)
      /var/run/google-fluentbit/pos-files from varrun (rw)
   fluentbit-gke:
    Image:      gke.gcr.io/fluent-bit-gke-exporter:v0.16.2-gke.0
    Port:       2021/TCP
    Host Port:  2021/TCP
    Command:
      /fluent-bit-gke-exporter
      --kubernetes-separator=_
      --stackdriver-resource-model=k8s
      --enable-pod-label-discovery
      --pod-label-dot-replacement=_
      --split-stdout-stderr
      --logtostderr
    Limits:
      memory:  250Mi
    Requests:
      cpu:        50m
      memory:     100Mi
    Liveness:     http-get http://:2021/healthz delay=120s timeout=1s period=60s #success=1 #failure=3
    Environment:  <none>
    Mounts:       <none>
  Volumes:
   varrun:
    Type:          HostPath (bare host directory volume)
    Path:          /var/run/google-fluentbit/pos-files
    HostPathType:  
   varlog:
    Type:          HostPath (bare host directory volume)
    Path:          /var/log
    HostPathType:  
   varlibkubeletpods:
    Type:          HostPath (bare host directory volume)
    Path:          /var/lib/kubelet/pods
    HostPathType:  
   varlibdockercontainers:
    Type:          HostPath (bare host directory volume)
    Path:          /var/lib/docker/containers
    HostPathType:  
   config-volume:
    Type:               ConfigMap (a volume populated by a ConfigMap)
    Name:               fluentbit-gke-config-v1.0.6
    Optional:           false
  Priority Class Name:  system-node-critical
Events:                 <none>
EN

回答 1

Stack Overflow用户

发布于 2021-06-07 21:17:29

您可能会遇到部分日志超过云日志API大小限制的情况。

Fluentbit-gke将其日志存储在每个节点上的/var/log/fluentbit.log中,这些日志不会导出到Cloud Logging。此目录是一个日志卷,它将/var/ hostPath从主机节点的文件系统挂载到Pod中。可以从主机本身访问日志文件。如果需要这些日志,请从节点获取fluentbit日志,并提供副本:

代码语言:javascript
代码运行次数:0
运行
复制
$ kubectl get nodes
$ gcloud compute scp <node_name>:/var/log/fluentbit.log* ./

与Fluentd不同,GKE 1.17中的Fluentbit目前最大单个日志条目大小为32K。这将导致fluentbit丢弃大小> 32K的用户日志,并且不会将其导出到Cloud Logging。在GKE 1.18集群上,单个日志条目的大小增加到1MB。这是将被fluentbit摄取的大小,但是,fluentbit会将其削减到200KB,以便在将条目写入Cloud Logging之前为添加到条目中的其他元数据留出一些空间。这是因为云日志接口在size of log entry上有256KB的限制。

票数 0
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/67868377

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档