Gitlab CICD 与Kubernetes实践·部署GitLab

公众号: 云原生生态圈

发布于 2020-11-02 11:17:41

2.3K0

发布于 2020-11-02 11:17:41

文章被收录于专栏：云原生生态圈

上一篇，简单的从?Gitlab CI/CD方法论中探索实践中大致了解Gitlab在CI/CD功能的基本介绍，现在我们通过在K8s集群内安装Gitlab、Gitlab Runner来为深入探索Gitlab持续集成做好前期准备，首先我们要在集群内安装Gitlab.

集群环境

K8s的集群版本为1.11.5，CNI使用的是Flannel, DNS为KubeDNS

☸️  ACK? devops  ~  ? ? k cluster-info
Kubernetes master is running at https://192.168.99.128:6443
Heapster is running at https://192.168.99.128:6443/api/v1/namespaces/kube-system/services/heapster/proxy
KubeDNS is running at https://192.168.99.128:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
monitoring-influxdb is running at https://192.168.99.128:6443/api/v1/namespaces/kube-system/services/monitoring-influxdb/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
☸️  ACK? devops  ~  ? ? k get nodes
NAME                 STATUS   ROLES    AGE   VERSION
cn-beijing.i-2ze01   Ready    master   1y    v1.11.5
cn-beijing.i-2ze02   Ready    master   1y    v1.11.5
cn-beijing.i-2ze03   Ready    master   1y    v1.11.5
cn-beijing.i-2ze04   Ready    <none>   1y    v1.11.5
cn-beijing.i-2ze05   Ready    <none>   1y    v1.11.5

通过学习Github上一个开源项目Dockerized GitLab[1]在kubernetes上进行GitLab-CE的安装部署，在部署Gitlab的之前，我们需要先部署其依赖的Redis和PostgreSQL,然后才能正常的运行gitlab. 这个开源项目中资源配置清单使用的是ReplicationController,这里我将修改使用Deployment控制器：

配置Redis服务

以为redis服务是提供给Gitlab服务使用，并不需要暴露在集群外部，因此我们在gitlab<->redis之间调用的时候，采用内部通信的方式,准备redis的配置清单redis.deployment.yaml

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: gitlab-redis # deployment的名字
  namespace: devops # 部署在devops namespace里面
  labels:
    name: gitlab-redis
spec:
  replicas: 1
  template:
    metadata:
      name: gitlab-redis # 约定gitlab-redis的deployment的模板名
      labels:
        name: gitlab-redis
    spec:
      containers:
      - name: redis # pod内redis的容器名
        image: redis:5.0.9 #也可以使用sameersbn的redis镜像,此处我们使用官方的镜像
        imagePullPolicy: IfNotPresent # 镜像的拉取策略
        ports:
        - name: redis
          containerPort: 6379
        volumeMounts:
        - mountPath: /var/lib/redis # 需要持久化的数据目录
          name: data
        livenessProbe: # 进行存活性监测
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 30 # 在启动存活性探测之前等待的秒数
          timeoutSeconds: 5 # 探测的超时时长
        readinessProbe:
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 5
          timeoutSeconds: 1
      volumes:
      - name: data
        emptyDir: {}

---
apiVersion: v1
kind: Service
metadata:
  name: gitlab-redis-svc
  namespace: devops
  labels:
    name: gitlab-redis-svc
spec:
  ports:
    - name: redis
      port: 6379
      targetPort: redis
  selector:
    name: gitlab-redis

配置postgresql服务

与redis服务一样，我们准备postgresql服务的配置清单postgresql.deployment.yaml

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: postgresql
  namespace: devops
  labels:
    name: postgresql
spec:
  template:
    metadata:
      name: postgresql
      labels:
        name: postgresql
    spec:
      containers:
      - name: postgresql
        image: sameersbn/postgresql # latest
        imagePullPolicy: IfNotPresent
        env:
        - name: DB_USER
          value: gitlab
        - name: DB_PASS
          value: gitlab_postgresql
        - name: DB_NAME
          value: gitlab_production
        - name: DB_EXTENSION
          value: pg_trgm
        ports:
        - name: postgres
          containerPort: 5432
        volumeMounts:
        - mountPath: /var/lib/postgresql
          name: data
        livenessProbe:
          exec:
            command:
            - pg_isready
            - -h
            - localhost
            - -U
            - postgres
          initialDelaySeconds: 30
          timeoutSeconds: 5
        readinessProbe:
          exec:
            command:
            - pg_isready
            - -h
            - localhost
            - -U
            - postgres
          initialDelaySeconds: 5
          timeoutSeconds: 1
      volumes:
      - name: data
        emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  name: postgresql
  namespace: devops
  labels:
    name: postgresql
spec:
  ports:
    - name: postgres
      port: 5432
      targetPort: postgres
  selector:
    name: postgresql

配置Gitlab服务

gitlab服务的部署就相对复杂一些，要添加正确的redis和postgresql的链接信息，同时为了在集群外部访问gitlab,我们需要给gitlab配置一个ingress,我们使用的环境是ACK,所以最后的时候我们需要给gitlab的服务分配一个ACK上绑定的测试域名，假设我们这里ACK的测试域名为*.cloudnativecosystem.cn-beijing.alicontainer.com,准备gitlab服务的配置清单gitlab.deployment.yaml

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: gitlab
  namespace: devops
  labels:
    name: gitlab
spec:
  template:
    metadata:
      name: gitlab
      labels:
        name: gitlab
    spec:
      containers:
      - name: gitlab
        image: sameersbn/gitlab:11.8.1
        imagePullPolicy: IfNotPresent
        env:
        - name: TZ # 容器时区
          value: Asia/Shanghai
        - name: GITLAB_TIMEZONE # 配置gitlab的时区
          value: Beijing
        - name: GITLAB_SECRETS_DB_KEY_BASE # 用于加密数据库中的CI机密变量以及导入凭据。如果丢失或旋转了此机密，则将无法使用现有的CI机密
          value: PjqzXnqkv9rjKWnTqhmgKLhtbM3sCKVH9bhHrmKRpnHXttd3hRjF4zXNjxztKKsC # 可以使用`pwgen -Bsv1 64`生成随机的字符串给变量
        - name: GITLAB_SECRETS_SECRET_KEY_BASE # 用于密码重置链接和其他“标准”身份验证功能。如果丢失或旋转了此机密，电子邮件中的密码重置令牌将重置。
          value: 3Tgk4WgqcK4JFn3gwMjNcgzTwkfhpTrL4wvmwhcmTRcPwqHzT4pcmgfsTvfpzpLV
        - name: GITLAB_SECRETS_OTP_KEY_BASE # 用于加密数据库中的2FA机密。如果您丢失或旋转了此机密，则您的所有用户都将无法使用2FA登录
          value: CdM9VprWKpqsdmw4V3tmcFwkzNVmHV9Kc3pLR7WtpVgHtFKmfCkMfJMW9TNw7pf7
        - name: GITLAB_ROOT_PASSWORD
          value: P@ssw0rd
        - name: GITLAB_ROOT_EMAIL
          value: cloudnativecosystem@******.cn-beijing.alicontainer.com
        - name: GITLAB_HOST
          value: code.******.cn-beijing.alicontainer.com
        - name: GITLAB_PORT
          value: "80"
        - name: GITLAB_SSH_PORT
          value: "30003"
        - name: GITLAB_NOTIFY_ON_BROKEN_BUILDS
          value: "true"
        - name: GITLAB_NOTIFY_PUSHER
          value: "false"
        - name: GITLAB_BACKUP_SCHEDULE # 更多变量信息参考https://github.com/sameersbn/docker-gitlab#quick-start
          value: daily
        - name: GITLAB_BACKUP_TIME
          value: 01:00
        - name: DB_TYPE
          value: postgres
        - name: DB_HOST
          value: postgresql
        - name: DB_PORT
          value: "5432"
        - name: DB_USER
          value: gitlab
        - name: DB_PASS
          value: P@ssw0rd
        - name: DB_NAME
          value: gitlab_production
        - name: REDIS_HOST
          value: gitlab-redis-svc
        - name: REDIS_PORT
          value: "6379"
        ports:
        - name: http
          containerPort: 80
        - name: ssh
          containerPort: 22
        volumeMounts:
        - mountPath: /home/git/data
          name: data
        livenessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 180
          timeoutSeconds: 5
        readinessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 5
          timeoutSeconds: 1
      volumes:
          - name: data
            emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  name: gitlab
  namespace: devops
  labels:
    name: gitlab
spec:
  ports:
    - name: http
      port: 80
      targetPort: http
    - name: ssh
      port: 22
      targetPort: 30003
  selector:
    name: gitlab

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/service-weight: ''
  name: gitlab
  namespace: devops
spec:
  rules:
    - host: code.******.cn-beijing.alicontainer.com
      http:
        paths:
          - backend:
              serviceName: gitlab
              servicePort: 80
            path: /
status:
  loadBalancer: {}

在准备完成所有相关的配置清单文件之后，我们就可以直接创建服务了

☸️  ACK? devops  ~/v1.11.5/gitlab  ? ? ls -al
total 24
drwxr-xr-x   7 marionxue  staff   224 Oct 24 16:34 .
drwxr-xr-x   5 marionxue  staff   160 Oct 24 20:22 ..
-rw-r--r--   1 marionxue  staff  3631 Oct 24 21:14 gitlab.deployment.yaml
-rw-r--r--   1 marionxue  staff  1460 Oct 24 21:12 postgresql.deployment.yaml
-rw-r--r--   1 marionxue  staff  1397 Oct 24 20:47 redis.deployment.yaml
☸️  ACK? devops  ~/v1.11.5/gitlab  ? ? k apply -f .
deployment.apps/gitlab created
service/gitlab created
ingress.extensions/gitlab created
deployment.apps/postgresql created
service/postgresql created
deployment.apps/gitlab-redis created
service/gitlab-redis-svc created
☸️  ACK? devops  ~/v1.11.5/gitlab  ? ? k get pods
NAME                            READY   STATUS              RESTARTS   AGE
gitlab-7c565f7845-vxzrw         0/1     ContainerCreating   0          12s
gitlab-redis-5d86f5cf95-pr8dp   1/1     Running             0          11s
postgresql-64d79556cf-prqn5     1/1     Running             0          11s

在所有的pods的运行状态为Running的时候，就可以访问gitlab了，如果很久没有运行起来，可以通过kubectl describe/logs进行初步排查。

等待之余，gitlab已经运行起来，我们直接可以通过ingress上配置的路由进行访问了

访问Kubernetes上的GitLab(root/P@ssw0rd)

然后进行一个正常的clone上传和下载操作,首先创建一个代码仓库gitlab-ci-demo

创建一个名为gitlab-ci-demo的仓库

☸️  ACK? devops  ~/v1.11.5/gitlab  ? ? git clone http://code.c33a5017db3924e7a86deeeaca6a706b8.cn-beijing.alicontainer.com/root/gitlab-ci-demo.git
Cloning into 'gitlab-ci-demo'...
warning: You appear to have cloned an empty repository.
☸️  ACK? devops  ~/v1.11.5/gitlab  ? ? cd gitlab-ci-demo
☸️  ACK? devops  ~/v1.11.5/gitlab/gitlab-ci-demo   master  ? ? echo "云原生生态圈" > ./README.md
☸️  ACK? devops  ~/v1.11.5/gitlab/gitlab-ci-demo   master  ? ? git add .
☸️  ACK? devops  ~/v1.11.5/gitlab/gitlab-ci-demo   master ✚  ? ? git commit -am "Update README.MD"
[master (root-commit) cc89857] Update README.MD
 1 file changed, 1 insertion(+)
 create mode 100644 README.md
☸️  ACK? devops  ~/v1.11.5/gitlab/gitlab-ci-demo   master  ? ? git push origin master
remote: HTTP Basic: Access denied
fatal: Authentication failed for 'http://code.c33a5017db3924e7a86deeeaca6a706b8.cn-beijing.alicontainer.com/root/gitlab-ci-demo.git/'
☸️  ACK? devops  ~/v1.11.5/gitlab/gitlab-ci-demo   master  ? ? git push origin master
Username for 'http://code.c33a5017db3924e7a86deeeaca6a706b8.cn-beijing.alicontainer.com': root
Password for 'http://root@code.c33a5017db3924e7a86deeeaca6a706b8.cn-beijing.alicontainer.com':
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), 241 bytes | 241.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To http://code.c33a5017db3924e7a86deeeaca6a706b8.cn-beijing.alicontainer.com/root/gitlab-ci-demo.git
 * [new branch]      master -> master

至此，我们基本上完成了在k8s上运行gitlab的初步需求。下一步就可以准备Gitlab Runner在k8s上的运行和进行持续集成持续部署的实践了。