【K8s】Kubernetes 服务发现之 Ingress

行者Sun

发布于 2024-09-02 12:44:51

1570

发布于 2024-09-02 12:44:51

文章被收录于专栏：Kubernetes

以下内容均来自个人笔记并重新梳理，如有错误欢迎指正！

如果对您有帮助，烦请点赞、关注、转发！如果您有其他想要了解的，欢迎私信联系我～

基本介绍

Ingress 是 Kubernetes 提供的一种服务发现机制，主要作用是为集群外部访问集群内部服务提供访问入口，通过制定 Ingress 策略管理 HTTP 路由，将集群外部的访问请求反向代理到集群内部不同 Service 对应的 Endpoint（即 Pod）上。

Ingress 具有以下特点：

Ingress 支持七层负载均衡，仅支持 HTTP 通信规则
Ingress 策略（rules）与 Ingress Controller 组成一个完整的 Ingress 负载均衡器
Ingress 将外部访问请求直接反向代理到 Endpoint 上，从而跳过 kube-proxy 组件的转发，kube-proxy 不再起作用
Ingress 对象与其反向代理的 Service 对象必须处于同一命名空间
Ingress 通过 path 路径访问不同服务，且 “ / ” 位于最后避免其他路径被拦截

工作原理

定义 Ingress 策略：用户在 Kubernetes 集群中创建 Ingress 资源，定义如何将外部请求路由到集群内的服务
策略监听：Ingress Controller 监听 Ingress 资源的变化，当有新的 Ingress 资源被创建或现有资源被更新时，Ingress 控制器会读取这些规则
配置负载均衡器或反向代理：Ingress Controller 根据 Ingress 策略配置内部的负载均衡器或反向代理服务器（如 Nginx、HAProxy 等），设置路由规则
路由转发：Ingress Controller 会根据配置的策略，将请求转发到正确的服务
服务响应：服务处理请求并返回响应，Ingress Controller 将响应转发回请求者

资源清单（示例）

1、Ingress Controller

# nginx-ingress-controller.yaml
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: lb-develop-controller
  namespace: kube-system
  labels:
    helm.sh/chart: ingress-nginx-4.2.5
    app.kubernetes.io/name: lb-develop
    app.kubernetes.io/instance: lb-develop
    app.kubernetes.io/version: "1.3.1"
    app.kubernetes.io/part-of: lb-develop
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
data:
  allow-snippet-annotations: "true"
  client-body-buffer-size: "10m"
  client-body-timeout: "300"
  client-header-buffer-size: "64k"
  client-header-timeout: "300"
  compute-full-forwarded-for: "true"
  enable-access-log-for-default-backend: "true"
  log-format-escape-json: "true"
  log-format-upstream: "{\"@timestamp\": \"$time_iso8601\", \"nginx.name\": \"lb-develop\", \"remote_addr\": \"$remote_addr\", \"x_forwarded_for\": \"$http_x_forwarded_for\", \"x_forwarded_proto\": \"$pass_access_scheme\", \"node-forwarded-proto\": \"$http_node_forwarded_proto\", \"request_id\": \"$req_id\", \"remote_user\": \"$remote_user\", \"bytes_sent\": $bytes_sent, \"status\": $status, \"content_length\": \"$content_length\", \"scheme\":\"$scheme\", \"vhost\": \"$host\", \"request_proto\": \"$server_protocol\", \"path\": \"$uri\", \"request_uri\": \"$request_uri\", \"request_body\": \"$request_body\", \"request_query\": \"$args\", \"request_length\": $request_length, \"duration\": $request_time, \"method\": \"$request_method\", \"http_referer\": \"$http_referer\", \"http_client_source\": \"$http_client_source\", \"http_client_version\": \"$http_client_version\", \"http_user_agent\": \"$http_user_agent\", \"http_token\": \"$http_authorization\", \"http_authorization\": \"$http_authorization\", \"http_uid\": \"$http_http_uid\", \"http_device_id\": \"$http_device_id\", \"http_x_auth_user\": \"$http_x_auth_user\", \"http_x_auth_scope\": \"$http_x_auth_scope\", \"http_x_token_type\": \"$http_x_token_type\", \"http_x_auth_client\": \"$http_x_auth_client\", \"http_origin\": \"$http_origin\", \"cookie_token\": \"$cookie_access_token\", \"cookie_uid\": \"$cookie_uid\", \"k8s_ingress_name\": \"$ingress_name\", \"k8s_namespace\": \"$namespace\", \"k8s_service_name\": \"$service_name\", \"upstream_name\": \"$proxy_upstream_name\", \"upstream_addr\": \"$upstream_addr\", \"upstream_status\": \"$upstream_status\", \"upstream_response_time\": \"$upstream_response_time\"}"
  proxy-add-original-uri-header: "true"
  proxy-body-size: "200m"
  proxy-buffer-size: "512k"
  proxy-buffering: "on"
  proxy-buffers-number: "8"
  proxy-connect-timeout: "300"
  proxy-read-timeout: "300"
  proxy-send-timeout: "300"
  upstream-keepalive-connections: "100"
  upstream-keepalive-requests: "100"
  upstream-keepalive-timeout: "30"
  use-forwarded-headers: "true"

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: lb-develop
  namespace: kube-system
  labels:
    helm.sh/chart: ingress-nginx-4.2.5
    app.kubernetes.io/name: lb-develop
    app.kubernetes.io/instance: lb-develop
    app.kubernetes.io/version: "1.3.1"
    app.kubernetes.io/part-of: lb-develop
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
automountServiceAccountToken: true

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: lb-develop
  labels:
    helm.sh/chart: ingress-nginx-4.2.5
    app.kubernetes.io/name: lb-develop
    app.kubernetes.io/instance: lb-develop
    app.kubernetes.io/version: "1.3.1"
    app.kubernetes.io/part-of: lb-develop
    app.kubernetes.io/managed-by: Helm
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
      - namespaces
    verbs:
      - list
      - watch
  - apiGroups:
      - coordination.k8s.io
    resources:
      - leases
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingressclasses
    verbs:
      - get
      - list
      - watch

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: lb-develop
  labels:
    helm.sh/chart: ingress-nginx-4.2.5
    app.kubernetes.io/name: lb-develop
    app.kubernetes.io/instance: lb-develop
    app.kubernetes.io/version: "1.3.1"
    app.kubernetes.io/part-of: lb-develop
    app.kubernetes.io/managed-by: Helm
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: lb-develop
subjects:
  - kind: ServiceAccount
    name: lb-develop
    namespace: kube-system

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: lb-develop
  namespace: kube-system
  labels:
    helm.sh/chart: ingress-nginx-4.2.5
    app.kubernetes.io/name: lb-develop
    app.kubernetes.io/instance: lb-develop
    app.kubernetes.io/version: "1.3.1"
    app.kubernetes.io/part-of: lb-develop
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
rules:
  - apiGroups:
      - ""
    resources:
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - endpoints
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingressclasses
    verbs:
      - get
      - list
      - watch
  # TODO(Jintao Zhang)
  # Once we release a new version of the controller,
  # we will be able to remove the configmap related permissions
  # We have used the Lease API for selection
  # ref: https://github.com/kubernetes/ingress-nginx/pull/8921
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      - ingress-controller-leader
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - coordination.k8s.io
    resources:
      - leases
    resourceNames:
      - ingress-controller-leader
    verbs:
      - get
      - update
  - apiGroups:
      - coordination.k8s.io
    resources:
      - leases
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: lb-develop
  namespace: kube-system
  labels:
    helm.sh/chart: ingress-nginx-4.2.5
    app.kubernetes.io/name: lb-develop
    app.kubernetes.io/instance: lb-develop
    app.kubernetes.io/version: "1.3.1"
    app.kubernetes.io/part-of: lb-develop
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: lb-develop
subjects:
  - kind: ServiceAccount
    name: lb-develop
    namespace: kube-system

---
apiVersion: v1
kind: Service
metadata:
  name: lb-develop-controller
  namespace: kube-system
  annotations:
  labels:
    helm.sh/chart: ingress-nginx-4.2.5
    app.kubernetes.io/name: lb-develop
    app.kubernetes.io/instance: lb-develop
    app.kubernetes.io/version: "1.3.1"
    app.kubernetes.io/part-of: lb-develop
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
spec:
  type: ClusterIP
  selector:
    app.kubernetes.io/name: lb-develop
    app.kubernetes.io/instance: lb-develop
    app.kubernetes.io/component: controller
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
      appProtocol: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: https
      appProtocol: https

---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: lb-develop-controller
  namespace: kube-system
  annotations: 
    reloader.stakater.com/auto: "true"
  labels:
    helm.sh/chart: ingress-nginx-4.2.5
    app.kubernetes.io/name: lb-develop
    app.kubernetes.io/instance: lb-develop
    app.kubernetes.io/version: "1.3.1"
    app.kubernetes.io/part-of: lb-develop
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: lb-develop
      app.kubernetes.io/instance: lb-develop
      app.kubernetes.io/component: controller
  template:
    metadata:
      labels:
        app.kubernetes.io/name: lb-develop
        app.kubernetes.io/instance: lb-develop
        app.kubernetes.io/component: controller
        app: lb-develop
        release: lb-develop
    spec:
      containers:
        - name: controller
          image: nginx-ingress-controller:1.2.1
          imagePullPolicy: IfNotPresent
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
          lifecycle: 
            preStop:
              exec:
                command:
                - /wait-shutdown
          args:
            - /nginx-ingress-controller
            - --publish-service=$(POD_NAMESPACE)/lb-develop-controller
            - --election-id=ingress-controller-leader
            - --controller-class=k8s.io/ingress-nginx
            - --ingress-class=lb-develop
            - --configmap=$(POD_NAMESPACE)/lb-develop-controller
            - --watch-ingress-without-class=true
          securityContext: 
            capabilities:
              drop:
              - ALL
              add:
              - NET_BIND_SERVICE
            runAsUser: 101
            allowPrivilegeEscalation: true
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: LD_PRELOAD
              value: /usr/local/lib/libmimalloc.so
            - name: TZ
              value: Asia/Shanghai
          livenessProbe: 
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 60
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe: 
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 60
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
      affinity: 
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: loadbalancer
                operator: In
                values:
                - lb-develop
      hostNetwork: true                         # 直接绑定主机的80端口、443端口
      dnsPolicy: ClusterFirstWithHostNet        # 设置对应的dns策略
      serviceAccountName: lb-develop
      terminationGracePeriodSeconds: 300

---
# We don't support namespaced ingressClass yet
# So a ClusterRole and a ClusterRoleBinding is required
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: lb-develop
  annotations:
    ingressclass.kubernetes.io/is-default-class: "true"
  labels:
    helm.sh/chart: ingress-nginx-4.2.5
    app.kubernetes.io/name: lb-develop
    app.kubernetes.io/instance: lb-develop
    app.kubernetes.io/version: "1.3.1"
    app.kubernetes.io/part-of: lb-develop
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
spec:
  controller: k8s.io/ingress-nginx

2、Ingress 对象

networking.k8s.io/v1 类型

# demo-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: "lb-develop"
spec:
  ingressClassName: lb-develop
  rules:
  - host: xx.xx.com                    # 有域名情况
    http:
      paths:
      - path: /prom
        pathType: Prefix               # 必须要指定
        backend:
          service: 
            name: prometheus
            port: 
              number: 9090             # 或者name: xxxx
      - path: /graf
        pathType: Prefix               # 必须要指定
        backend:
          service: 
            name: monitoring-grafana
            port: 
              number: 8080

  - http:                              # 无域名情况
      paths:
      - path: /nginx
        pathType: Prefix               # 必须要指定
        backend:
          service: 
            name: nginx
            port: 
              number: 8080             # 或者name: xxxx

  tls:
  - hosts:
    - xxx.xxx.com
    secretName: demo-secret

🔔 networking.k8s.io/v1 类型需要指定 pathType ，否则会创建失败
🔔 支持3种 pathType 类型：
Exact：精确匹配URL路径，区分大小写
Prefix: 匹配前缀，区分大小写，并且按照元素对路径进行匹配。🔔 前缀结尾处有无 / 均可匹配
ImplementationSpecific：匹配取决于 Ingress Controller 的实现

extensions/v1beta1 类型

# demo-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: "lb-develop"
spec:
  rules:
  - host: xx.xx.com                    # 有域名情况
    http:
      paths:
      - path: /prom
        backend:
          serviceName: prometheus
          servicePort: 9090
      - path: /graf
        backend:
          serviceName: monitoring-grafana
          servicePort: 8080
  - http:                              # 无域名情况
      paths:
      - path: /nginx
        backend:
          serviceName: nginx
          servicePort: 8080
  tls:
  - hosts:
    - xxx.xxx.com
    secretName: demo-secret

常用命令

# 创建 Ingress 对象
kubectl create -f demo-ingress.yaml
或
kubectl apply -f demo-ingress.yaml

# 查看 Ingress 对象列表
kubectl get ingress

# 查看 Ingress 对象资源清单
kubeclt get ingress -oyaml

# 查看 Ingress 对象内容
kubectl describe ingress demo-ingress.yaml

本文参与腾讯云自媒体同步曝光计划，分享自微信公众号。

原始发表：2024-08-25，如有侵权请联系 cloudcommunity@tencent.com 删除

kubernetes