ingress通过daemonSet,nodeSelector,hostNetwork方式部署

dogfei

发布于 2020-08-19 11:18:40

3.9K0

ingress通过daemonSet,nodeSelector,hostNetwork方式部署

首先我们需要在k8s集群中准备边缘节点，用来部署ingress（需要对边缘节点打污点或者使用亲和性/反亲和性），如果是私有化部署，需要对ingress做高可用，如果资源充足，还可以再做一次负载均衡，这里在本地测试的话，直接在其中一个node上启动一个就可以了

这个就是大概的流量流转图，首先经过DNS域名解析，然后到达LB，然后流量经过ingress做一次负载分发到service，最后再由service做一次负载分发到对应的pod中

安装ingress

给边缘节点打标签

给边缘节点打一个标签，用于在部署ingress时，可以将ingress通过nodeSelecor调度到该边缘节点上

1	# kubectl label nodes k8s-node01 isIngress=true

修改yaml文件

部署ingress的yaml文件可以在官网获取，我们只需要修改其中某及部分就可以了，不需要全都要修改

官方的yaml文件地址：https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.34.1/deploy/static/provider/cloud/deploy.yaml

service部分

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

# Source: ingress-nginx/templates/controller-service.yaml apiVersion: v1 kind: Service metadata: labels: helm.sh/chart: ingress-nginx-2.11.1 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.34.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: controller name: ingress-nginx-controller namespace: ingress-nginx spec: type: LoadBalancer externalTrafficPolicy: Local ports: - name: http port: 80 protocol: TCP targetPort: http - name: https port: 443 protocol: TCP targetPort: https selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/component: controller

把Deployment需改成DaemonSet

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115

# Source: ingress-nginx/templates/controller-deployment.yaml apiVersion: apps/v1 kind: DaemonSet metadata: labels: helm.sh/chart: ingress-nginx-2.11.1 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.34.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: controller name: ingress-nginx-controller namespace: ingress-nginx spec: selector: matchLabels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/component: controller revisionHistoryLimit: 10 minReadySeconds: 0 template: metadata: labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/component: controller annotations: prometheus.io/port: "10254" prometheus.io/scrape: "true" spec: dnsPolicy: ClusterFirst containers: - name: controller #image: us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller:v0.34.1@sha256:0e072dddd1f7f8fc8909a2ca6f65e76c5f0d2fcfb8be47935ae3457e8bbceb20 image: alpha-harbor.yunshicloud.com/base/ingress-controller:v0.34.1 imagePullPolicy: IfNotPresent lifecycle: preStop: exec: command: - /wait-shutdown args: - /nginx-ingress-controller - --publish-service=ingress-nginx/ingress-nginx-controller - --election-id=ingress-controller-leader - --ingress-class=nginx - --configmap=ingress-nginx/ingress-nginx-controller - --validating-webhook=:8443 - --validating-webhook-certificate=/usr/local/certificates/cert - --validating-webhook-key=/usr/local/certificates/key securityContext: capabilities: drop: - ALL add: - NET_BIND_SERVICE runAsUser: 101 allowPrivilegeEscalation: true env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace livenessProbe: httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 timeoutSeconds: 1 successThreshold: 1 failureThreshold: 5 readinessProbe: httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 timeoutSeconds: 1 successThreshold: 1 failureThreshold: 3 ports: - name: http containerPort: 80 protocol: TCP - name: https containerPort: 443 protocol: TCP - name: webhook containerPort: 8443 protocol: TCP volumeMounts: - name: webhook-cert mountPath: /usr/local/certificates/ readOnly: true resources: requests: cpu: 100m memory: 90Mi serviceAccountName: ingress-nginx terminationGracePeriodSeconds: 300 nodeSelector: isIngress: "true" hostNetwork: true volumes: - name: webhook-cert secret: secretName: ingress-nginx-admission

修改完后，直接启动就可以了

1	kubectl apply -f ingress.yaml

验证

1 2 3 4 5	# kubectl get pod -n ingress-nginx -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES ingress-nginx-admission-create-vgnzb 0/1 Completed 0 14m 172.17.135.146 k8s-node03 <none> <none> ingress-nginx-admission-patch-vp6bs 0/1 Completed 1 14m 172.17.58.223 k8s-node02 <none> <none> ingress-nginx-controller-hvgfv 1/1 Running 0 14m 192.168.0.225 k8s-node01 <none> <none>

可以看到，我们的ingress-controller已经部署到了指定的节点上了。

示例

以部署Jenkins为例，来看下如何使用ingress来对外提供服务

创建一个Jenkins

这里不再贴了

1 2 3 4 5 6 7 8 9 10 11 12 13 14

# kubectl get all -n jenkins-system NAME READY STATUS RESTARTS AGE pod/jenkins-server-848b685bfd-2rmmc 1/1 Running 1 138m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/jenkins-server ClusterIP 10.99.112.45 <none> 8080/TCP,50000/TCP 5d NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/jenkins-server 1/1 1 1 5d NAME DESIRED CURRENT READY AGE replicaset.apps/jenkins-server-848b685bfd 1 1 1 5d

这里要记住service的名称：jenkins-server，后面会用到