Harbor是由VMware公司开源的企业级的Docker Registry管理项目,相比docker官方拥有更丰富的权限权利和完善的架构设计,适用大规模docker集群部署提供仓库服务,并且提供UI界面。 一般harbor可以通过helm或者docker-compose安装,本文以compose安装为例,介绍harbor如何配置腾讯云对象存储COS作为私有镜像仓库存储地址。 另外,除了使用自建harbor,也可使用腾讯云镜像仓库简化维护过程: https://cloud.tencent.com/product/tcr
1 准备:
安装docker-compose
curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
2 重启docker
systemctl daemon-reload
3 下载habor装包解压 harbor.v2.1.0.tar.gz
3.1 配置项注意:
# harbor.v2.1.0.tar.gz
vi harbor.yml
-----------harbor.yml---------
# Configuration file of Harbor
# The IP address or hostname to access admin UI and registry service.
# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
hostname: harbor.yourset.com
# http related config
http:
# port for http, default is 80. If https enabled, this port will redirect to https port
port: 80
# https related config
https:
# https port for harbor, default is 443
port: 443
# The path of cert and key files for nginx
certificate: /data/key/harbor.yourset.com.crt
private_key: /data/key/harbor.yourset.xyz.key
# # Uncomment following will enable tls communication between all harbor components
# internal_tls:
# # set enabled to true means internal tls is enabled
# enabled: true
# # put your cert and key files on dir
# dir: /etc/harbor/tls/internal
# Uncomment external_url if you want to enable external proxy
# And when it enabled the hostname will no longer used
# external_url: https://reg.mydomain.com:8433
# The initial password of Harbor admin
# It only works in first time to install harbor
# Remember Change the admin password from UI after launching Harbor.
harbor_admin_password: Harborxxx
# Harbor DB configuration
database:
# The password for the root user of Harbor DB. Change this before any production use.
password: root123
# The maximum number of connections in the idle connection pool. If it <=0, no idle connections are retained.
max_idle_conns: 50
# The maximum number of open connections to the database. If it <= 0, then there is no limit on the number of open connections.
# Note: the default number of connections is 1024 for postgres of harbor.
max_open_conns: 1000
# The default data volume
data_volume: /data
# Harbor Storage settings by default is using /data dir on local filesystem
# Uncomment storage_service setting If you want to using external storage
storage_service:
s3:
accesskey: xxxxxx
secretkey: xxxxxxx
regionendpoint: cos.ap-guangzhou.myqcloud.com
region: ap-guangzhou
bucket: harborgz-xxxxx
secure: true
# ca_bundle is the path to the custom root ca certificate, which will be injected into the truststore
# of registry's and chart repository's containers. This is usually needed when the user hosts a internal storage with self signed certificate.
# ca_bundle:
# # storage backend, default is filesystem, options include filesystem, azure, gcs, s3, swift and oss
# # for more info about this configuration please refer https://docs.docker.com/registry/configuration/
# filesystem:
# maxthreads: 100
# # set disable to true when you want to disable registry redirect
# redirect:
# disabled: false
###....
4 启动,进入安装目录
$ docker-compose down -v
# 修改配置后,需要重新检查运行环境
$ ./prepare
# 启动harbor服务
$ docker-compose up -d
5 测试
5.1 网页测试,如果需要https访问,需要申请证书并配置在CLB及harbor服务器上
配置在harbor.yml的https中:
-----------------
https:
# https port for harbor, default is 443
port: 443
# The path of cert and key files for nginx
certificate: /data/key/harbor.yourset.com.crt
private_key: /data/key/harbor.yourset.xyz.key
5.2 本地测试,输入密码
5.3 测试推镜像
注意事项:需要在harbor中先建立项目,否则会推送失败
[root@centos ~/tmp]# docker tag hello-world 127.0.0.1/s3/hello-world:v1.0.0
[root@centos ~/tmp]# docker push 127.0.0.1/s3/hello-world:v1.0.0
The push refers to repository [127.0.0.1/s3/hello-world]
f22b99068db9: Preparing
unauthorized: project not found, name: s3: project not found, name: s3
[root@centos ~/tmp]# docker push 127.0.0.1/s3/hello-world:v1.0.0
The push refers to repository [127.0.0.1/s3/hello-world]
f22b99068db9: Pushed
v1.0.0: digest: sha256:1b26826f602946860c279fce65829b57792 size: 525
5.4 对象存储中也生成了相关文件:
2.1 坑一:S3配置踩坑
网上搜的文章及其他用户反馈S3部分的配置如下:
s3:
region: ap-xxx
bucket: xx-sigp-xxxxxxx
accesskey: xxxxxxx
secretkey: xxxxx
endpoint: cos.ap-singapore.myqcloud.com
secure: true
使用该配置后,启动harbor后总会有harbor-registryclt等容器不断重启,造成无法推拉镜像:
查看错误日志:
tail -f /var/log/harbor/registryctl.log
Aug 3 15:32:31 172.30.0.1 registryctl[28778]: 2021-08-03T07:32:31Z [ERROR] [/registryctl/config/config.go:63]: failed to load storage driver, err:No region parameter provided
Aug 3 15:32:31 172.30.0.1 registryctl[28778]: 2021-08-03T07:32:31Z [FATAL] [/registryctl/main.go:78]: Failed to load configurations with error: No region parameter provided
关键信息:
询问几个同事都没有结论,在google查了半天也没找到相关的文档,只好决定从源码入手,先去查看registryctl/main.go源码:
继续查看config.go:63
查看setStorageDriver()
找出storagedriver中s3的相关代码
查看s3部分:
发现一段特别的地方:
当"regionendpoint"为空时,程序会去aws的官方的 validRegins列表中查询可用区,而本次是要配置腾讯云COS地址,当然在aws的region列表里面没有,所以会提示 err:No region parameter provided。
因此需要传入"regionendpoint"的key才可避免查询aws自己的region list(网上的文章误导人啊),而不是传入"endpoint"
,所以需要在harbor.yml中把配置改为:
s3:
region: ap-xxx
bucket: xx-sigp-xxxxxxx
accesskey: xxxxxxx
secretkey: xxxxx
regionendpoint: cos.ap-singapore.myqcloud.com
secure: true
修改后重载harbor启动成功,推拉镜像正常。
2.2 坑二:COS 强一致配置踩坑
有用户根据上面部署后,发现新的报错,现象是通过docker push文件成功,但是harbor总返回500报错:
经过与COS团队沟通,主要是list强一致问题,即put 文件后直接list不一定能list到文件,因为list是最终一致性的,需要COS运维同事下发强一致的配置后,会保障能list出来。
发配置需提供客户账号appid、可用区、桶名称信息(可向腾讯云提工单),发布配置后该报错可解决。
1 部署过程及时记录自己操作过程,关注日志
2 如果网上没有现成的答案,请教身边的专家
3 实在解决不了,去查源码,所有的逻辑都已经写在代码里了
k8s学习笔记合集:
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。