kolla-ansible部署openstack多节点
kolla-ansible部署openstack多节点
cuijianzhe
发布于 2022-06-14 17:56:34
发布于 2022-06-14 17:56:34
文章被收录于专栏：cuijianzhe
环境介绍
ceph 搭建先搞定
对接 openstack 配置参考这里:
- 主机：
IP | 作用 | 主机名 |
|---|---|---|
192.168.51.204 | 存储节点 | cinder |
192.168.51.209 | 计算节点 | computer3 |
192.168.51.210 | 计算节点 | compute2 |
192.168.51.211 | 计算节点 | compute1 |
192.168.51.212 | control 节点 | control |
- 同步时间 在 ceph1 上安装 ntp 服务，是 2、3 同步 1 的时间
[root@ceph1 ~]# yum install ntp -y && systemctl enable ntpd.service && systemctl start ntpd.service
[root@ceph1 ~]# systemctl start ntpd
[root@ceph2 ~]# ntpdate 192.168.51.212
[root@ceph3 ~]# ntpdate 192.168.51.212或者所有节点配置统一互联网 ntp 服务
ntpdate ntp1.aliyun.com
timedatectl set-timezone Asia/Shanghai基础环境
- controller 配置
yum install python-devel libffi-devel gcc openssl-devel git python-pip -y
pip install -U pip
yum install -y yum-utils device-mapper-persistent-data lvm2
yum install ansiblecontroller 网卡（外网桥接网卡）配置：
注意：各个节点网卡名称必须一致
[root@controller ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth3
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=eth3
UUID=0bd8bc11-2190-42d6-bbaa-6fb963a6def0
DEVICE=eth3
ONBOOT=yes- 添加 docker-ce yum 源配置文件并安装 docker（在 controller、cinder、computer 节点配置）
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo- 更新并安装 Docker-CE 社区版本(在 controller、cinder、computer 节点都配置)
yum -y install docker-ce- 开启 Docker 服务
systemctl start docker && systemctl enable docker && systemctl status docker设置 3 台服务器的 docker volume 卷挂载方式和 docker 镜像加速器
- 卷挂载方式（在 controller、cinder、computer 节点都配置）
[root@computer ~]# mkdir /etc/systemd/system/docker.service.dtee /etc/systemd/system/docker.service.d/kolla.conf << 'EOF'
[Service]
MountFlags=shared
EOF- 设置 docker 镜像加速器（在 controller、cinder、computer 节点都配置）
curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io
systemctl daemon-reload
systemctl enable docker && systemctl restart docker && systemctl status docker- 配置 pip 软件源
mkdir ~/.pip
vim ~/.pip/pip.conf
[global]
index-url = http://mirrors.aliyun.com/pypi/simple/
[install]
trusted-host=mirrors.aliyun.com[root@localhost ~]# pvcreate /dev/sda4
WARNING: xfs signature detected on /dev/sda4 at offset 0. Wipe it? [y/n]: y
Wiping xfs signature on /dev/sda4.
Physical volume "/dev/sda4" successfully created.
[root@localhost ~]# vgcreate cinder-volumes /dev/sda4
Volume group "cinder-volumes" successfully created
[root@localhost ~]# systemctl enable lvm2-lvmetad.service
[root@localhost ~]# vgs
VG #PV #LV #SN Attr VSize VFree
cinder-volumes 1 0 0 wz--n- 461.12g 461.12g基础环境完成。
storage 配置 cinder 存储信息
- 安装 lvm 相关的软件包
yum install -y yum-utils device-mapper-persistent-data lvm2在 controller 节点安装 kolla-ansible
pip install kolla-ansible --ignore-installed PyYAML
#赋值kolla相关配置文件
cp -r /usr/share/kolla-ansible/etc_examples/kolla /etc/
cp /usr/share/kolla-ansible/ansible/inventory/* /etc/kolla/自定义 kolla-ansible 安装 openstack 的相关配置文件
- 自动生成 openstack 各服务的密码
[root@controller ~]# kolla-genpwd
[root@controller ~]# vim /etc/kolla/passwords.yml
keystone_admin_password: 598941324- 配置/etc/kolla/globals.yml 自定义部署事项
[root@controller ~]# grep -v "^#\|^$" /etc/kolla/globals.yml
---
kolla_base_distro: "centos"
kolla_install_type: "binary"
openstack_release: "stein"
kolla_internal_vip_address: "192.168.51.212"
network_interface: "enp61s0f0"
kolla_external_vip_interface: "{{ network_interface }}"
api_interface: "{{ network_interface }}"
storage_interface: "{{ network_interface }}"
cluster_interface: "{{ network_interface }}"
tunnel_interface: "{{ network_interface }}"
dns_interface: "{{ network_interface }}"
neutron_external_interface: "enp61s0f3"
enable_haproxy: "no"
enable_cinder: "yes"
enable_cinder_backend_lvm: "yes"
glance_enable_rolling_upgrade: "no"
cinder_volume_group: "cinder-volumes"
ironic_dnsmasq_dhcp_range:
tempest_image_id:
tempest_flavor_ref_id:
tempest_public_network_id:
tempest_floating_network_name:cinder_volume_group: "cinder-volumes"对应 cinder 中 lvm 挂载卷
[root@cinder ~]# vgs
VG #PV #LV #SN Attr VSize VFree
cinder-volumes 1 0 0 wz--n- 461.12g 461.12g- 配置 ssh key，授信节点
ssh-keygen
ssh-copy-id -i ~/.ssh/id_rsa.pub root@computer
ssh-copy-id -i ~/.ssh/id_rsa.pub root@computer2
ssh-copy-id -i ~/.ssh/id_rsa.pub root@cinder
ssh-copy-id -i ~/.ssh/id_rsa.pub root@controller- 配置 nultinode 多节点主机清单文件
[control]
# These hostname must be resolvable from your deployment host
controller
# The above can also be specified as follows:
#control[01:03] ansible_user=kolla
# The network nodes are where your l3-agent and loadbalancers will run
# This can be the same as a host in the control group
[network]
controller
[compute]
computer1
computer2
[monitoring]
controller
# When compute nodes and control nodes use different interfaces,
# you need to comment out "api_interface" and other interfaces from the globals.yml
# and specify like below:
#compute01 neutron_external_interface=eth0 api_interface=em1 storage_interface=em1 tunnel_interface=em1
[storage]
cinder
[deployment]
controller
[baremetal:children]
control
network
compute
storage
monitoring其他配置不用修改
- 配置完成后，我们可以进入部署阶段。 首先，我们需要设置基本的主机的软件包的依赖项，例如 docker。Kolla-Ansible 提供了一个将以正确版本安装所有必需服务的剧本 playbook。先安装具有 kolla 部署依赖关系的引导服务 bootstrap-servers
kolla-ansible -i /etc/kolla/multinode bootstrap-servers- 如果出现如下错误：
TASK [baremetal : Install docker SDK for python] *****************************************************************************************************************************************************************
ok: [controller]
fatal: [cinder]: FAILED! => {"changed": false, "cmd": ["/bin/pip2", "install", "-U", "docker"], "msg": "stdout: Collecting docker\n Using cached https://files.pythonhosted.org/packages/cc/ca/699d4754a932787ef353a157ada74efd1ceb6d1fc0bfb7989ae1e7b33111/docker-4.1.0-py2.py3-none-any.whl\nCollecting requests!=2.18.0,>=2.14.2\n Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl\nCollecting websocket-client>=0.32.0\n Using cached https://files.pythonhosted.org/packages/29/19/44753eab1fdb50770ac69605527e8859468f3c0fd7dc5a76dd9c4dbd7906/websocket_client-0.56.0-py2.py3-none-any.whl\nRequirement already satisfied, skipping upgrade: ipaddress>=1.0.16; python_version < \"3.3\" in /usr/lib/python2.7/site-packages (from docker) (1.0.16)\nRequirement already satisfied, skipping upgrade: backports.ssl-match-hostname>=3.5; python_version < \"3.5\" in /usr/lib/python2.7/site-packages (from docker) (3.5.0.1)\nRequirement already satisfied, skipping upgrade: six>=1.4.0 in /usr/lib/python2.7/site-packages (from docker) (1.9.0)\nRequirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker) (3.0.4)\nRequirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker) (2.8)\nRequirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker) (1.25.6)\nRequirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker) (2019.9.11)\nInstalling collected packages: requests, websocket-client, docker\n Found existing installation: requests 2.6.0\n\n:stderr: DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support\nERROR: Cannot uninstall 'requests'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.\n"}
解决方法：（在对应节点进行配置）
yum install -y python-pip
yum remove python-docker-py
pip install -U docker- 另一个错误：
TASK [baremetal : Install pip] ***********************************************************************************************************************************************************************************
ok: [cinder]
ok: [controller]
fatal: [computer]: FAILED! => {"changed": false, "msg": "Download error on https://pypi.python.org/simple/pip/: [Errno 101] Network is unreachable -- Some packages may not be found!\nCouldn't find index page for 'pip' (maybe misspelled?)\nDownload error on https://pypi.python.org/simple/: [Errno 101] Network is unreachable -- Some packages may not be found!\nNo local packages or download links found for pip\nerror: Could not find suitable distribution for Requirement.parse('pip')\n"}解决：（在对应节点 computer 进行配置）
pip install --upgrade pip对主机进行预部署检查
kolla-ansible -i /etc/kolla/multinode prechecks- 安装
kolla-ansible -i /etc/kolla/multinode deploy- 部署验证：
kolla-ansible -i /etc/kolla/multinode post-deploy安装 openstack client 端
pip install python-openstackclient python-glanceclient python-neutronclient如有报错：
ERROR: dogpile-cache 0.9.0 has requirement decorator>=4.0.0, but you'll have decorator 3.4.0 which is incompatible.解决：
pip install -U decorator
pip install --upgrade decorate- 报错 2：
ERROR: Package 'more-itertools' requires a different Python: 2.7.5 not in '>=3.4'解决：
pip install more-itertools==5.0.0- 再执行
pip install PyYAML --ignore-installed PyYAML
pip install python-openstackclientpip install python-neutronclient- 报错：
ImportError: No module named 'requests.packages.urllib3'解决：
pip install requests urllib3 pyOpenSSL --force --upgrade调整配置及重新部署
如果需要调整配置. 那么编辑 globals.yml 后, 然后运行 reconfigure. 使用 -t 参数可以只对变动的模块进行调整.
kolla-ansible -i /etc/kolla/multinode reconfigure -t neutron
kolla-ansible -i /etc/kolla/multinode deploy -t neutron- 编辑：/usr/share/kolla-ansible/init-runonce
EXT_NET_CIDR='192.168.50.0/24'
EXT_NET_RANGE='start=192.168.50.10,end=192.168.50.90'
EXT_NET_GATEWAY='192.168.50.1' 执行：
[root@controller ~]# cd /usr/share/kolla-ansible
[root@controller kolla-ansible]# source /etc/kolla/admin-openrc.sh
[root@controller kolla-ansible]# ./init-runonce 输出：
Done.
To deploy a demo instance, run:
openstack server create \
--image cirros \
--flavor m1.tiny \
--key-name mykey \
--network demo-net \
demo1 执行如上命令即可创建 demo1 实例
[root@controller kolla-ansible]# openstack server create \
> --image cirros \
> --flavor m1.tiny \
> --key-name mykey \
> --network demo-net \
> demo1
+-------------------------------------+-----------------------------------------------+
| Field | Value |
+-------------------------------------+-----------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | XMHWD9M9iRAE |
| config_drive | |
| created | 2019-11-03T00:25:49Z |
| flavor | m1.tiny (1) |
| hostId | |
| id | 1c4c6ba2-7a76-4207-a534-c7dbacf6a883 |
| image | cirros (4992fc52-b1e1-4a5d-88af-1c7c8ac94d9c) |
| key_name | mykey |
| name | demo1 |
| progress | 0 |
| project_id | fda196dacffd4f35b3c3118035edff0e |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2019-11-03T00:25:49Z |
| user_id | 5afd82c446e64186918b911e1f587388 |
| volumes_attached | |
+-------------------------------------+-----------------------------------------------+ 使用 openstack
- 添加公网
[root@controller kolla-ansible]# openstack network create --external --provider-physical-network physnet1 --provider-network-type flat public
+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2019-11-02T03:56:09Z |
| description | |
| dns_domain | None |
| id | 686fc9c2-4cea-4096-b98c-5f99c42cf3a9 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | False |
| is_vlan_transparent | None |
| location | cloud='', project.domain_id=, project.domain_name='Default', project.id='1cac8b32fb3047f48ced3d85811bc327', project.name='admin', region_name='RegionOne', zone= |
| mtu | 1500 |
| name | public |
| port_security_enabled | True |
| project_id | 1cac8b32fb3047f48ced3d85811bc327 |
| provider:network_type | flat |
| provider:physical_network | physnet1 |
| provider:segmentation_id | None |
| qos_policy_id | None |
| revision_number | 1 |
| router:external | External |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| updated_at | 2019-11-02T03:56:09Z |
+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+- 给 public 添加子网
[root@controller ~]# openstack subnet create --no-dhcp --allocation-pool 'start=192.168.50.10,end=192.168.50.100' --network public --subnet-range 192.168.50.0/24 --gateway 192.168.50.1 public-subnet
+-------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| allocation_pools | 192.168.50.10-192.168.50.100 |
| cidr | 192.168.50.0/24 |
| created_at | 2019-11-02T03:59:14Z |
| description | |
| dns_nameservers | |
| enable_dhcp | False |
| gateway_ip | 192.168.50.1 |
| host_routes | |
| id | c24ca524-beb7-4cb0-a8dc-01a910563fa3 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| location | cloud='', project.domain_id=, project.domain_name='Default', project.id='1cac8b32fb3047f48ced3d85811bc327', project.name='admin', region_name='RegionOne', zone= |
| name | public-subnet |
| network_id | 686fc9c2-4cea-4096-b98c-5f99c42cf3a9 |
| prefix_length | None |
| project_id | 1cac8b32fb3047f48ced3d85811bc327 |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2019-11-02T03:59:14Z |
+-------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+- 创建私有网络
[root@controller ~]# openstack network create --provider-network-type vxlan demo-net
+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2019-11-02T03:59:55Z |
| description | |
| dns_domain | None |
| id | 684dc2b5-dda3-4280-81f6-5906d6c2f48d |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | False |
| is_vlan_transparent | None |
| location | cloud='', project.domain_id=, project.domain_name='Default', project.id='1cac8b32fb3047f48ced3d85811bc327', project.name='admin', region_name='RegionOne', zone= |
| mtu | 1450 |
| name | demo-net |
| port_security_enabled | True |
| project_id | 1cac8b32fb3047f48ced3d85811bc327 |
| provider:network_type | vxlan |
| provider:physical_network | None |
| provider:segmentation_id | 48 |
| qos_policy_id | None |
| revision_number | 1 |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| updated_at | 2019-11-02T03:59:55Z |
+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+- 给私网添加子网
[root@controller ~]# openstack subnet create --subnet-range 10.0.0.0/24 --network demo-net --gateway 10.0.0.1 --dns-nameserver 8.8.8.8 demo-subnet
+-------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| allocation_pools | 10.0.0.2-10.0.0.254 |
| cidr | 10.0.0.0/24 |
| created_at | 2019-11-02T04:00:27Z |
| description | |
| dns_nameservers | 8.8.8.8 |
| enable_dhcp | True |
| gateway_ip | 10.0.0.1 |
| host_routes | |
| id | 6ddb71f1-9deb-4e55-ac7a-667d70bf21a1 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| location | cloud='', project.domain_id=, project.domain_name='Default', project.id='1cac8b32fb3047f48ced3d85811bc327', project.name='admin', region_name='RegionOne', zone= |
| name | demo-subnet |
| network_id | 684dc2b5-dda3-4280-81f6-5906d6c2f48d |
| prefix_length | None |
| project_id | 1cac8b32fb3047f48ced3d85811bc327 |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2019-11-02T04:00:27Z |
+-------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+*给外网和私网添加路由
[root@controller ~]# openstack router create demo-router
+-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2019-11-02T04:28:22Z |
| description | |
| distributed | False |
| external_gateway_info | null |
| flavor_id | None |
| ha | False |
| id | 6dc8c9c1-2391-4b95-b376-b647a3d2ff90 |
| location | cloud='', project.domain_id=, project.domain_name='Default', project.id='1cac8b32fb3047f48ced3d85811bc327', project.name='admin', region_name='RegionOne', zone= |
| name | demo-router |
| project_id | 1cac8b32fb3047f48ced3d85811bc327 |
| revision_number | 1 |
| routes | |
| status | ACTIVE |
| tags | |
| updated_at | 2019-11-02T04:28:22Z |
+-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[root@controller ~]#
[root@controller ~]# openstack router add subnet demo-router demo-subnet
[root@controller ~]# openstack router set --external-gateway public demo-router- 通过下面的命令可以查询刚刚所建的网络信息：
[root@controller ~]# neutron net-list
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+--------------------------------------+----------+----------------------------------+------------------------------------------------------+
| id | name | tenant_id | subnets |
+--------------------------------------+----------+----------------------------------+------------------------------------------------------+
| 684dc2b5-dda3-4280-81f6-5906d6c2f48d | demo-net | 1cac8b32fb3047f48ced3d85811bc327 | 6ddb71f1-9deb-4e55-ac7a-667d70bf21a1 10.0.0.0/24 |
| 686fc9c2-4cea-4096-b98c-5f99c42cf3a9 | public | 1cac8b32fb3047f48ced3d85811bc327 | c24ca524-beb7-4cb0-a8dc-01a910563fa3 192.168.50.0/24 |
+--------------------------------------+----------+----------------------------------+------------------------------------------------------+引用认证
[root@controller ~]# source /etc/kolla/admin-openrc.sh 标题：kolla-ansible部署openstack多节点
作者：cuijianzhe
本文参与 腾讯云自媒体同步曝光计划，分享自作者个人站点/博客。
如有侵权请联系 cloudcommunity@tencent.com 删除
评论
登录后参与评论
推荐阅读
目录
相关产品与服务
容器服务
腾讯云容器服务（Tencent Kubernetes Engine, TKE）基于原生 kubernetes 提供以容器为核心的、高度可扩展的高性能容器管理服务，覆盖 Serverless、边缘计算、分布式云等多种业务部署场景，业内首创单个集群兼容多种计算节点的容器资源管理模式。同时产品作为云原生 Finops 领先布道者，主导开源项目Crane，全面助力客户实现资源优化、成本控制。
腾讯云开发者
