部署loki采集kubernetes容器日志
部署loki采集kubernetes容器日志
锅总
发布于 2024-06-12 07:08:11
发布于 2024-06-12 07:08:11
代码可运行
运行总次数：0
代码可运行
关联问题
换一批
loki简介
Grafana Loki is a set of components that can be composed into a fully featured logging stack. Unlike other logging systems, Loki is built around the idea of only indexing metadata about your logs: labels (just like Prometheus labels). Log data itself is then compressed and stored in chunks in object stores such as Amazon Simple Storage Service (S3) or Google Cloud Storage (GCS), or even locally on the filesystem. A small index and highly compressed chunks simplifies the operation and significantly lowers the cost of Loki.
部署方式
- 所有服务部署在rancher中
- 容器日志采集端Promtail用DaemonSet部署
- 容器日志接收端Loki用Deployment部署
- 容器日志查看Grafana用Deployment部署
准备yaml文件
1. loki.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
workload.user.cattle.io/workloadselector: deployment-default-loki
name: loki
namespace: default
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
workload.user.cattle.io/workloadselector: deployment-default-loki
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
type: RollingUpdate
template:
metadata:
labels:
workload.user.cattle.io/workloadselector: deployment-default-loki
spec:
containers:
- args:
- -config.file=/mnt/config/loki-config.yaml
image: grafana/loki:2.9.7
imagePullPolicy: Always
name: loki
ports:
- containerPort: 3100
name: http-3100
protocol: TCP
resources: {}
securityContext:
allowPrivilegeEscalation: false
capabilities: {}
privileged: false
readOnlyRootFilesystem: false
runAsNonRoot: false
stdin: true
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
tty: true
volumeMounts:
- mountPath: /mnt/config
name: vol1
- mountPath: /tmp/loki/
name: vol2
dnsPolicy: ClusterFirst
restartPolicy: Always
volumes:
- configMap:
defaultMode: 511
name: loki-config
optional: false
name: vol1
- hostPath:
path: /data/local-storage/loki-data
type: ""
name: vol2
---
apiVersion: v1
kind: Service
metadata:
name: loki
namespace: default
spec:
ports:
- name: http-3100
port: 3100
protocol: TCP
targetPort: 3100
selector:
workload.user.cattle.io/workloadselector: deployment-default-loki
type: ClusterIP
---
apiVersion: v1
data:
loki-config.yaml: |-
auth_enabled: false
server:
http_listen_port: 3100
grpc_listen_port: 9096
common:
instance_addr: 0.0.0.0
path_prefix: /tmp/loki
storage:
filesystem:
chunks_directory: /tmp/loki/chunks
rules_directory: /tmp/loki/rules
replication_factor: 1
ring:
kvstore:
store: inmemory
query_range:
results_cache:
cache:
embedded_cache:
enabled: true
max_size_mb: 100
schema_config:
configs:
- from: 2020-10-24
store: boltdb-shipper
object_store: filesystem
schema: v11
index:
prefix: index_
period: 24h
table_manager:
retention_deletes_enabled: true
retention_period: 72h #日志保存多久
kind: ConfigMap
name: loki-config
namespace: default2. Promtail.yaml
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: promtail-daemonset
namespace: default
spec:
selector:
matchLabels:
name: promtail
template:
metadata:
labels:
name: promtail
spec:
containers:
- args:
- -config.file=/etc/promtail/promtail.yaml
env:
- name: HOSTNAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
image: grafana/promtail:2.9.7
imagePullPolicy: IfNotPresent
name: promtail-container
resources: {}
securityContext:
capabilities: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/log
name: logs
- mountPath: /data/docker/containers
name: varlibdockercontainers
readOnly: true
- mountPath: /etc/promtail
name: promtail-config
- mountPath: /tmp/
name: vol1
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: promtail-serviceaccount
serviceAccountName: promtail-serviceaccount
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /var/log
type: ""
name: logs
- hostPath:
path: /data/docker/containers #这里的/data/docker根据具体的docker数据目录而定，可通过 docker info | grep "Docker Root Dir" 命令查看
type: ""
name: varlibdockercontainers
- configMap:
defaultMode: 420
name: promtail-config
name: promtail-config
- hostPath:
path: /data/local-storage/promtail-data
type: ""
name: vol1
updateStrategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
---
apiVersion: v1
kind: ConfigMap
metadata:
name: promtail-config
data:
promtail.yaml: |
server:
http_listen_port: 9080
grpc_listen_port: 0
clients:
- url: https://loki.defautl.svc:3100/loki/api/v1/push
positions:
filename: /tmp/positions.yaml
target_config:
sync_period: 10s
scrape_configs:
- job_name: pod-logs
kubernetes_sd_configs:
- role: pod
pipeline_stages:
- docker: {}
relabel_configs:
- source_labels:
- __meta_kubernetes_pod_node_name
target_label: __host__
- action: labelmap
regex: __meta_kubernetes_pod_label_(.+)
- action: replace
replacement: $1
separator: /
source_labels:
- __meta_kubernetes_namespace
- __meta_kubernetes_pod_name
target_label: job
- action: replace
source_labels:
- __meta_kubernetes_namespace
target_label: namespace
- action: replace
source_labels:
- __meta_kubernetes_pod_name
target_label: pod
- action: replace
source_labels:
- __meta_kubernetes_pod_container_name
target_label: container
- replacement: /var/log/pods/*$1/*.log
separator: /
source_labels:
- __meta_kubernetes_pod_uid
- __meta_kubernetes_pod_container_name
target_label: __path__
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: promtail-clusterrole
rules:
- apiGroups: [""]
resources:
- nodes
- services
- pods
verbs:
- get
- watch
- list
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: promtail-serviceaccount
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: promtail-clusterrolebinding
subjects:
- kind: ServiceAccount
name: promtail-serviceaccount
namespace: default
roleRef:
kind: ClusterRole
name: promtail-clusterrole
apiGroup: rbac.authorization.k8s.io3. Grafana.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: grafana
name: grafana
namespace: default
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: grafana
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
labels:
app: grafana
spec:
containers:
- image: grafana/grafana:10.0.0
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 3000
timeoutSeconds: 1
name: grafana
ports:
- containerPort: 3000
name: http-grafana
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /robots.txt
port: 3000
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 2
resources:
requests:
cpu: 250m
memory: 750Mi
securityContext:
capabilities: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/lib/grafana
name: vol1
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 472
supplementalGroups:
- 0
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /data/local-storage/grafana-data
type: ""
name: vol1
---
apiVersion: v1
kind: Service
metadata:
name: grafana
namespace: default
spec:
ports:
- port: 3000
protocol: TCP
targetPort: 3000
selector:
app: "grafana"
sessionAffinity: None
type: ClusterIP
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: grafana
namespace: default
spec:
rules:
- host: grafana.demo.com
http:
paths:
- backend:
serviceName: grafana
servicePort: 3000
pathType: ImplementationSpecific
tls:
- hosts:
- grafana.demo.com
执行部署命令
kubectl apply -f loki.yaml
kubectl apply -f Promtail.yaml
kubectl apply -f Grafana.yamlgrafana启动后需要加入loki数据源
部署效果
可以愉快的查日志了
本文参与 腾讯云自媒体同步曝光计划，分享自微信公众号。
原始发表：2024-04-25，如有侵权请联系 cloudcommunity@tencent.com 删除
评论
登录后参与评论
暂无评论
推荐阅读
编辑精选文章
换一批
推荐阅读
相关推荐
kubernetes | loki 开源日志框架
更多 >腾讯云开发者
