前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >如何在OpenStack-Ansible上集成Tungsten Fabric

如何在OpenStack-Ansible上集成Tungsten Fabric

原创
作者头像
Tungsten Fabric
修改2020-11-20 17:44:14
1.4K0
修改2020-11-20 17:44:14
举报
文章被收录于专栏:Tungsten Fabric中文社区

本文作者James Denton,拥有超过15年信息技术领域行业经验,目前为知名云计算服务商Rackspace Hosting首席架构师,致力于云网络和对基于OpenStack的Rackspace私有云产品的支持。

Tungsten Fabric(前身为OpenContrail)是Linux基金会旗下的“多云、多技术栈”SDN解决方案。简而言之,Tungsten Fabric和Contrail(基于TF的商业产品)可以替代和增强标准OpenStack云的许多网络组件,并提供以下功能:

·分布式虚拟路由

·DHCP和元数据服务

·基于策略的访问控制

·与安全组的兼容性

·……等等

转发平面支持MPLS over GRE、VXLAN、L2/L3单播和L3多播,以实现虚拟网络和物理网络之间的互连。

注:关于Tungsten Fabric架构的概述可在下面的链接找到:https://tungstenfabric.org.cn/docs

我最近接受了将Tungsten Fabric集成到OpenStack-Ansible中的挑战,以简化TF和相关OpenStack bits在生产级OpenStack云中的部署。本文将在较高级别涵盖OpenStack-Ansible的主(Rocky)分支的一些补丁,以及Juniper和TF社区提供的用于部署Tungsten Fabric的contrail-ansible-deployer playbooks的一些调整。本文所描述的过程绝不意味着是最终的过程,可能会显得比较笨拙且不理想,但是,这是一个开始。

注:本文作者指明文章中的Tungsten Fabric、OpenContrail和Contrail为混用,翻译时统一为Tungsten Fabric。

集成要求

几周前,我部署了一个独立的3节点Tungsten Fabric设置,其中包含基于Queens的OpenStack-Ansible的All-in-One节点。在弄清楚使事情进入半工作状态所需的调整后,我决定亲自尝试并部署一个AIO节点,该节点包含一个单一的Tungsten Fabric服务实例以及基本的OpenStack服务。

以下为最低规格的配置建议:

·系统: Ubuntu VM

·OS: 16.04.4 LTS

·内存: 48GB

·硬盘: 300GB

·NIC: 单口网卡

如我稍后指出的,裸机节点可能复杂度较小,但是ESXi或其它管理程序上的虚拟机应该是可以尝试的。

从OpenStack-Ansible开始

首先,请克隆OpenStack-Ansible存储库。在撰写本文时,master分支是与OpenStack的第18个发行版Rocky相关联的。

代码语言:javascript
复制
# git clone https://git.openstack.org/openstack/openstack-ansible /opt/openstack-ansible
# cd /opt/openstack-ansible
# git checkout master
# export ANSIBLE_ROLE_FETCH_MODE=git-clone

接下来,运行引导程序脚本:

代码语言:javascript
复制
# scripts/bootstrap-ansible.sh
# scripts/bootstrap-aio.sh

引导程序脚本将下载playbooks以部署OpenStack,还将在服务器上准备符合OpenStack-Ansible架构的网络环境。

角色修改

对使用OpenStack-Ansible部署的OpenStack云进行更改,通常意味着对构成部署的Ansible角色进行更改。这包括对任务、模板、变量等的更改。

需要修改的角色包括:

·os_neutron

·os_nova

是否所有这些角色的更改都是必需的,还有待观察,这里只是为了更好地说明。

os_neutron

一些新文件包括:

代码语言:javascript
复制
root@aio1:/etc/ansible/roles/os_neutron# git diff --staged
diff --git a/tasks/providers/opencontrail_config.yml b/tasks/providers/opencontrail_config.yml
new file mode 100644
index 0000000..8f5fc7d
--- /dev/null
+++ b/tasks/providers/opencontrail_config.yml
@@ -0,0 +1,99 @@
+---
+# Copyright 2018, Rackspace Hosting, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+- name: Set the packages to install
+ set_fact:
+ neutron_optional_combined_pip_packages: |-
+ {% set packages = neutron_optional_opencontrail_pip_packages %}
+ {{ packages }}
+
+- name: Install OpenContrail pip packages
+ pip:
+ name: "{{ neutron_optional_combined_pip_packages }}"
+ state: "{{ neutron_pip_package_state }}"
+ virtualenv: "{{ neutron_bin | dirname }}"
+ virtualenv_site_packages: "no"
+ extra_args: >-
+ {{ neutron_developer_mode | ternary(pip_install_developer_constraints | default('--constraint /opt/developer-pip-constraints.txt'), '') }}
+ {{ (pip_install_upper_constraints is defined) | ternary('--constraint ' + pip_install_upper_constraints | default(''),'') }}
+ {{ pip_install_options | default('') }}
+ register: install_packages
+ until: install_packages|success
+ retries: 5
+ delay: 2
+ tags:
+ - opencontrail-install
+ - opencontrail-pip-packages
+
+- name: Install git
+ apt:
+ name: git
+ state: present
+ delegate_to: "{{ item }}"
+ with_items:
+ - "{{ groups['neutron_server'] }}"
+ tags:
+ - opencontrail-install
+
+- name: Clone contrail neutron plugin
+ git:
+ repo: "{{ opencontrail_plugin_git_repo }}"
+ version: "{{ opencontrail_plugin_git_install_branch }}"
+ dest: /opt/contrail-neutron-plugin
+ force: yes
+ register: contrail_plugin_git_clone
+ delegate_to: "{{ item }}"
+ with_items:
+ - "{{ groups['neutron_server'] }}"
+ until: contrail_plugin_git_clone|success
+ retries: 5
+ delay: 2
+ tags:
+ - opencontrail-install
+
+# (jamesdenton) Will need to eventually compile and/or extract from Docker container
+# The tasks marked (temp) should be reworked
+
+- name: Download Contrail python libraries (temp)
+ vars:
+ - dlpath: https://github.com/busterswt/contrail-openstack/raw/master
+ get_url:
+ url: "{{ dlpath }}/{{ item }}"
+ dest: /opt
+ mode: 0440
+ with_items:
+ - contrail-openstack-neutron-init.tar
+ tags:
+ - opencontrail-install
+
+- name: Unpack Contrail python libraries (temp)
+ unarchive:
+ remote_src: yes
+ src: /opt/contrail-openstack-neutron-init.tar
+ dest: /openstack/venvs/neutron-{{ neutron_venv_tag }}/lib/python2.7/site-packages
+ when:
+ inventory_hostname == groups['neutron_server'][0]
+ tags:
+ - opencontrail-install
+
+- name: Install contrail neutron plugin into venv
+ command: "/openstack/venvs/neutron-{{ neutron_venv_tag }}/bin/python setup.py install"
+ args:
+ chdir: /opt/contrail-neutron-plugin
+ delegate_to: "{{ item }}"
+ with_items:
+ - "{{ groups['neutron_server'] }}"
+ tags:
+ - opencontrail-install
diff --git a/templates/plugins/opencontrail/ContrailPlugin.ini.j2 b/templates/plugins/opencontrail/ContrailPlugin.ini.j2
new file mode 100644
index 0000000..9d645b0
--- /dev/null
+++ b/templates/plugins/opencontrail/ContrailPlugin.ini.j2
@@ -0,0 +1,23 @@
+# {{ ansible_managed }}
+
+{% if neutron_plugin_type == 'opencontrail' %}
+[APISERVER]
+api_server_ip = {{ opencontrail_api_vip_address }}
+api_server_port = {{ opencontrail_api_vip_port }}
+multi_tenancy = True
+contrail_extensions = ipam:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_ipam.NeutronPluginContrailIpam,policy:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_policy.NeutronPluginContrailPolicy,route-table:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_vpc.NeutronPluginContrailVpc,contrail:None,service-interface:None,vf-binding:None
+
+[COLLECTOR]
+analytics_api_ip = {{ opencontrail_collector_vip_address }}
+analytics_api_port = {{ opencontrail_collector_vip_port }}
+
+[keystone_authtoken]
+auth_host = {{ internal_lb_vip_address }}
+auth_port = {{ keystone_service_port }}
+auth_protocol = {{ keystone_service_proto }}
+admin_user = {{ keystone_admin_user_name }}
+admin_password = {{ keystone_auth_admin_password }}
+admin_tenant_name = {{ keystone_admin_tenant_name }}
+insecure = True
+region_name = {{ keystone_service_region }}
+{% endif %}

对现有文件的更改包括:

代码语言:javascript
复制
root@aio1:/etc/ansible/roles/os_neutron# git diff
diff --git a/defaults/main.yml b/defaults/main.yml
index 162e933..7054c96 100644
--- a/defaults/main.yml
+++ b/defaults/main.yml
@@ -63,6 +63,8 @@ networking_bgpvpn_git_repo: https://git.openstack.org/openstack/networking-bgpvp
 networking_bgpvpn_git_install_branch: master
 openstack_ceilometer_git_repo: https://git.openstack.org/openstack/ceilometer
 openstack_ceilometer_git_install_branch: master
+opencontrail_plugin_git_repo: https://github.com/Juniper/contrail-neutron-plugin
+opencontrail_plugin_git_install_branch: master
 # Developer mode
 neutron_developer_mode: false
@@ -164,6 +166,7 @@ neutron_sriov_nic_agent_ini_overrides: {}
 neutron_sriov_nic_agent_init_overrides: {}
 neutron_vpn_agent_init_overrides: {}
 neutron_vpnaas_agent_ini_overrides: {}
+neutron_opencontrail_conf_ini_overrides: {}
 ###
 ### Quotas
@@ -434,3 +437,12 @@ ovs_nsh_support: False
 # Set higher priority to mardim PPA when ovs_nsh_support is True
 ovs_nsh_apt_pinned_packages: [{ package: "*", release: "LP-PPA-mardim-mardim-ppa"}]
+
+###
+### Contrail/OpenContrail/Tungsten Fabric Configuration
+###
+
+opencontrail_api_vip_address: "{{ external_lb_vip_address }}"
+opencontrail_api_vip_port: "8082"
+opencontrail_collector_vip_address: "{{ external_lb_vip_address }}"
+opencontrail_collector_vip_port: "8081"
diff --git a/templates/neutron.conf.j2 b/templates/neutron.conf.j2
index 83d25a7..dd755ca 100644
--- a/templates/neutron.conf.j2
+++ b/templates/neutron.conf.j2
@@ -42,6 +42,10 @@ core_plugin = {{ neutron_plugin_core }}
 {% if neutron_plugin_type.split('.')[0] == 'ml2' %}
 service_plugins = {{ neutron_plugin_loaded_base | join(',') }}
 {% endif %}
+{% if neutron_plugin_type == 'opencontrail' %}
+service_plugins = neutron_plugin_contrail.plugins.opencontrail.loadbalancer.v2.plugin.LoadBalancerPluginV2
+api_extensions_path = /openstack/venvs/neutron-{{ neutron_venv_tag }}/lib/python2.7/site-packages/neutron_plugin_contrail/extensions:/openstack/venvs/neutron-{{ neutron_venv_tag }}/lib/python2.7/site-packages/neutron_lbaas/extensions
+{% endif %}
 # MAC address generation for VIFs
 base_mac = fa:16:3e:00:00:00
@@ -94,8 +98,9 @@ rpc_workers = {{ neutron_rpc_workers }}
 {% set dhcp_agents_max = num_agent if num_agent > 2 else 2 %}
 # DHCP
-{% if neutron_plugin_type == 'ml2.dragonflow' %}
-# In dragonflow, DHCP is fully distributed, and DHCP agents are not used
+{% if neutron_plugin_type == ('ml2.dragonflow' or 'opencontrail') %}
+# In dragonflow and opencontrail, DHCP is fully distributed and DHCP
+# agents are not used
 dhcp_agent_notification = False
 {% else %}
 dhcp_agent_notification = True
diff --git a/vars/main.yml b/vars/main.yml
index cef4ee8..2d1c2a2 100644
--- a/vars/main.yml
+++ b/vars/main.yml
@@ -121,6 +121,10 @@ neutron_plugins:
 plugin_ini: plugins/ml2/ml2_conf.ini
 driver_interface: "openvswitch"
 l3_agent_mode: "legacy"
+ opencontrail:
+ plugin_core: neutron_plugin_contrail.plugins.opencontrail.contrail_plugin.NeutronPluginContrailCoreV2
+ plugin_ini: plugins/opencontrail/ContrailPlugin.ini
+ plugin_conf_ini_overrides: "{{ neutron_opencontrail_conf_ini_overrides }}"
 ###
 ### ML2 Plugin Configuration
diff --git a/vars/source_install.yml b/vars/source_install.yml
index a246a45..24e57ea 100644
--- a/vars/source_install.yml
+++ b/vars/source_install.yml
@@ -96,6 +96,13 @@ neutron_proprietary_nuage_pip_packages:
 - nuage-openstack-neutronclient
 - nuagenetlib
+neutron_optional_opencontrail_pip_packages:
+ - bitarray
+ - bottle
+ - geventhttpclient
+ - psutil>=0.6.0
+ - requests>=1.1.0
+
 neutron_developer_constraints:
 - "git+{{ neutron_git_repo }}@{{ neutron_git_install_branch }}#egg=neutron"
 - "git+{{ neutron_fwaas_git_repo }}@{{ neutron_fwaas_git_install_branch }}#egg=neutron-fwaas"

os_nova

代码语言:javascript
复制
root@aio1:/etc/ansible/roles/os_nova# git diff
diff --git a/defaults/main.yml b/defaults/main.yml
index 67d92e9..bc44511 100644
--- a/defaults/main.yml
+++ b/defaults/main.yml
@@ -325,6 +325,9 @@ nova_network_services:
 calico:
 use_forwarded_for: True
 metadata_proxy_enabled: False
+ opencontrail:
+ use_forwarded_for: True
+ metadata_proxy_enabled: True
 # Nova quota
 nova_quota_cores: 20

HAproxy更改

为了提供单一的TF API和仪表板终端,我决定创建一个VIP,以平衡TF API和analytics服务之间的流量。这是否为最佳做法还有待观察,但是此处对group_vars的更改有助于创建VIP:

代码语言:javascript
复制
root@aio1:/opt/openstack-ansible# git diff
diff --git a/inventory/group_vars/haproxy/haproxy.yml b/inventory/group_vars/haproxy/haproxy.yml
index b837443..dc53ef4 100644
--- a/inventory/group_vars/haproxy/haproxy.yml
+++ b/inventory/group_vars/haproxy/haproxy.yml
@@ -36,6 +36,7 @@ haproxy_rabbitmq_management_whitelist_networks: "{{ haproxy_whitelist_networks }
 haproxy_repo_git_whitelist_networks: "{{ haproxy_whitelist_networks }}"
 haproxy_repo_cache_whitelist_networks: "{{ haproxy_whitelist_networks }}"
 haproxy_opendaylight_whitelist_networks: "{{ haproxy_whitelist_networks }}"
+haproxy_opencontrail_whitelist_networks: "{{ haproxy_whitelist_networks }}"
 haproxy_default_services:
 - service:
@@ -365,3 +366,23 @@ haproxy_default_services:
 haproxy_backend_httpcheck_options:
 - expect status 405
 haproxy_service_enabled: "{{ (groups['ceph-rgw'] is defined and groups['ceph-rgw'] | length > 0) or (ceph_rgws | length > 0) }}"
+ - service:
+ haproxy_service_name: opencontrail-api
+ haproxy_backend_nodes: "{{ groups['opencontrail-api_hosts'] | default([]) }}"
+ haproxy_bind: "{{ [opencontrail_api_vip_address] }}"
+ haproxy_port: 8082
+ haproxy_balance_type: tcp
+ haproxy_timeout_client: 5000s
+ haproxy_timeout_server: 5000s
+ haproxy_whitelist_networks: "{{ haproxy_opencontrail_whitelist_networks }}"
+ haproxy_service_enabled: "{{ neutron_plugin_type == 'opencontrail' }}"
+ - service:
+ haproxy_service_name: opencontrail-collector
+ haproxy_backend_nodes: "{{ groups['opencontrail-analytics_hosts'] | default([]) }}"
+ haproxy_bind: "{{ [opencontrail_collector_vip_address] }}"
+ haproxy_port: 8081
+ haproxy_balance_type: tcp
+ haproxy_timeout_client: 5000s
+ haproxy_timeout_server: 5000s
+ haproxy_whitelist_networks: "{{ haproxy_opencontrail_whitelist_networks }}"
+ haproxy_service_enabled: "{{ neutron_plugin_type == 'opencontrail' }}"

我在All-in-One节点上使用此方法遇到的一些问题,包括HAproxy将VIP绑定到端口8081的能力。后来在过程中发现,Tungsten Fabric playbooks在0.0.0.0:8081上创建了一个侦听器,从而阻止了VIP绑定在同一端口上。这里的替代方法是注释掉该服务,或在部署HAproxy后将其禁用。对于HAproxy在不同节点上的多节点安装,可以保留它。最终,这两种服务的负载平衡可能无法实现,但我暂时将其保留。

环境更改

默认的Neutron env.d框架将部署原本没必要部署的Neutron代理容器。通过覆盖默认值,我们可以删除代理容器并定义一些新的组件:

代码语言:javascript
复制
cat <<'EOF' >> /etc/openstack_deploy/env.d/neutron.yml
component_skel:
 neutron_server:
 belongs_to:
 - neutron_all
 opencontrail_vrouter:
 belongs_to:
 - neutron_all
 opencontrail_api:
 belongs_to:
 - neutron_all
 opencontrail_analytics:
 belongs_to:
 - neutron_all
container_skel:
 neutron_agents_container:
 contains: {}
 opencontail-vrouter_container:
 belongs_to:
 - compute_containers
 contains:
 - opencontrail_vrouter
 properties:
 is_metal: true
 opencontrail-api_container:
 belongs_to:
 - opencontrail-api_containers
 contains:
 - opencontrail_api
 properties:
 is_metal: true
 opencontrail-analytics_container:
 belongs_to:
 - opencontrail-analytics_containers
 contains:
 - opencontrail_analytics
 properties:
 is_metal: true
 neutron_server_container:
 belongs_to:
 - network_containers
 contains:
 - neutron_server
physical_skel:
 opencontrail-api_containers:
 belongs_to:
 - all_containers
 opencontrail-api_hosts:
 belongs_to:
 - hosts
 opencontrail-analytics_containers:
 belongs_to:
 - all_containers
 opencontrail-analytics_hosts:
 belongs_to:
 - hosts
EOF

在这一点上,我不确定所有这些更改都是必要的,以及方法是否正确,但目前暂时这样。

在 openstack_user_config.yml文件中,我定义了两个新的组,以便能够在主机之间拆分API和Analytics服务。由于这是一个AlO节点,因此IP是相同的:

代码语言:javascript
复制
opencontrail-api_hosts:
 aio1:
 ip: 172.29.236.100
opencontrail-analytics_hosts:
 aio1:
 ip: 172.29.236.100

Overrides

对os_neutron角色的更改导致添加了新的默认值,并且还需要一些overrides。在user_variables.yml中添加了以下内容:

代码语言:javascript
复制
neutron_plugin_type: opencontrail
neutron_driver_quota: neutron_plugin_contrail.plugins.opencontrail.quota.driver.QuotaDriver
opencontrail_plugin_git_install_branch: R5.0

角色中指定的一些默认值包括:

代码语言:javascript
复制
opencontrail_api_vip_address: "{{ external_lb_vip_address }}"
opencontrail_api_vip_port: "8082"
opencontrail_collector_vip_address: "{{ external_lb_vip_address }}"
opencontrail_collector_vip_port: "8081"
opencontrail_plugin_git_repo: https://github.com/Juniper/contrail-neutron-plugin
opencontrail_plugin_git_install_branch: master
neutron_opencontrail_conf_ini_overrides: {}

最终要求仍尚未确定。

运行OpenStack-Ansible playbooks

至此,已经完成了对OpenStack-Ansible playbooks的大部分更改,并且可以部署OpenStack了:

代码语言:javascript
复制
# cd /opt/openstack-ansible/playbooks
# openstack-ansible setup-hosts.yml
# openstack-ansible setup-infrastructure.yml
# openstack-ansible setup-openstack.yml

虽然Neutron会被下线,但在使用网络之前,还需要进行一些其它更改并部署Tungsten Fabric。可以使用Horizon,但可能无法完全发挥它的功能。

接下来,我们必须要执行一些操作,包括克隆Juniper ansible playbook repo,设置overrides,以及运行这些playbooks,以安装Tungsten Fabric,并为与TF相关的服务overlay一些Docker容器。

部署Tungsten Fabric

到这里,我们已经部署了OpenStack,并且知道已实现的网络后端,但是该后端还不存在。Juniper提供的playbooks可以使用Docker容器部署Tungsten Fabric。这些相同的playbooks还可以部署基于Kolla的OpenStack版本,这是另一种OpenStack部署策略。在这里,我们实际上只需要与Tungsten Fabric相关的代码。

为了实现一些更改,我克隆了repo,并进行了部署在基于OpenStack-Ansible的云上所需的必要更改。

克隆repo:

代码语言:javascript
复制
# git clone http://github.com/busterswt/contrail-ansible-deployer /opt/openstack-ansible/playbooks/contrail-ansible-deployer
# cd /opt/openstack-ansible/playbooks/contrail-ansible-deployer
# git checkout osa

Overrides

Juniper playbooks依赖于它们自己的清单和overrides,因此执行以下几步可能会感觉有些多余。我还没有重新编写playbooks以利用OpenStack-Ansible的可用清单。

可以在/etc/openstack_deploy/user_opencontrail_vars.yml的新文件中定义这些overrides:

代码语言:javascript
复制
config_file: /etc/openstack_deploy/user_opencontrail_vars.yml
opencontrail_api_vip_address: ""
opencontrail_collector_vip_address: ""
provider_config:
 bms:
 ssh_pwd:
 ssh_user: root
 ssh_public_key: /root/.ssh/id_rsa.pub
 ssh_private_key: /root/.ssh/id_rsa
 ntpserver: 129.6.15.28
instances:
 aio1:
 provider: bms
 ip: 172.29.236.100
 roles:
 config_database:
 config:
 control:
 analytics_database:
 analytics:
 webui:
 vrouter:
  VROUTER_GATEWAY: 10.50.0.1
  PHYSICAL_INTERFACE: ens160
global_configuration:
 CONTAINER_REGISTRY: opencontrailnightly
# CONTAINER_REGISTRY: hub.juniper.net/contrail
# CONTAINER_REGISTRY_USERNAME: 
# CONTAINER_REGISTRY_PASSWORD: 
contrail_configuration:
 CLOUD_ORCHESTRATOR: openstack
 CONTRAIL_VERSION: latest
# CONTRAIL_VERSION: 5.0.0-0.40
# UPGRADE_KERNEL: true
 UPGRADE_KERNEL: false
 KEYSTONE_AUTH_HOST: ""
 KEYSTONE_AUTH_PUBLIC_PORT: 5000
 KEYSTONE_AUTH_PROTO: http
 KEYSTONE_AUTH_URL_VERSION:
 KEYSTONE_AUTH_ADMIN_USER: admin
 KEYSTONE_AUTH_ADMIN_PASSWORD: ""
 KEYSTONE_AUTH_URL_VERSION: /v3
 CONTROLLER_NODES: 172.29.236.100
 CONTROL_NODES: 172.29.236.100
 ANALYTICSDB_NODES: 172.29.236.100
 WEBUI_NODES: 172.29.236.100
 ANALYTICS_NODES: 172.29.236.100
 CONFIGDB_NODES: 172.29.236.100
 CONFIG_NODES: 172.29.236.100

请特别注意以下键/值对:

代码语言:javascript
复制
VROUTER_GATEWAY: 10.50.0.1
PHYSICAL_INTERFACE: ens160
CONTAINER_REGISTRY: opencontrailnightly
# CONTAINER_REGISTRY: hub.juniper.net/contrail
# CONTAINER_REGISTRY_USERNAME: 
# CONTAINER_REGISTRY_PASSWORD: 
CONTRAIL_VERSION: latest
# CONTRAIL_VERSION: 5.0.0-0.40
CONTROLLER_NODES: 172.29.236.100
CONTROL_NODES: 172.29.236.100
ANALYTICSDB_NODES: 172.29.236.100
WEBUI_NODES: 172.29.236.100
ANALYTICS_NODES: 172.29.236.100
CONFIGDB_NODES: 172.29.236.100
CONFIG_NODES: 172.29.236.100

第一部分定义了vRouter所需的vhost0 接口将重新利用或者利用的“physical”接口。这是一个具有单个NIC的AIO节点,这意味着机会很不错,playbooks会自动确定要使用的接口。我们继续前进,同时在此处进行了标注。该主机的IP地址是10.50.0.221,网关是10.50.0.1。对于多宿主主机,有可能将接口及其自己的网关地址专用于vRouter。我尚未进行多NIC部署,但期待能实现。

第二部分定义了将从中下载容器的Docker注册表。这里opencontailnightly 注册表包含每隔一到两天生成的容器,这些容器不一定在任何给定的日期都是有效的。如果你通过Juniper访问GA注册表,还可以定义该注册表并提供访问凭据。在nightly注册表中唯一可用的版本是latest版本,而Juniper注册表可能具有已标记的发行版。记得要使用适当的注册表。

第三部分定义了各个节点类型的地址,用于在整个playbooks中用于将值插入模板。这些是当前成功完成playbooks所必需的。请注意,我在这里使用了“CONTAINER_NET”地址,是希望Tungsten Fabric和OpenStack可以在LXC使用的现有容器网络上进行通信。在此AIO节点中,172.29.236.100是br-mgmt上配置的IP,并且恰好也是内部VIP地址。部署在Docker容器中的服务会将端口绑定到其各自服务的IP上。在多节点的安装中,此更改的方式是否可行仍为待定。

OSA repo更改

OpenStack-Ansible包括用于软件包管理repo服务器,该服务器目前禁止安装部署Tungsten Fabric所需的某些版本的软件包。在这里,我通过修改/root/.pip/pip.conf禁用了内部repo服务器:

代码语言:javascript
复制
[global]
disable-pip-version-check = true
timeout = 120
#index-url = http://172.29.236.100:8181/simple
#trusted-host =
# 172.29.236.100
[install]
upgrade = true
upgrade-strategy = only-if-needed
pre = true

运行playbooks

到这里,我们应该有了一个很好的起点,可以开始运行TF playbooks了。

首先,节点必须运行引导程序:

代码语言:javascript
复制
# cd /opt/openstack-ansible/playbooks/
# openstack-ansible -e orchestrator=openstack contrail-ansible-deployer/playbooks/configure_instances.yml

当UPGRADE_KERNEL 为true时,主机可能会重新启动。如果发生这种情况,请再次重新运行该playbook。如果TF节点不是需要部署的主机,则playbooks具有一个计时器,等待主机返回。

接下来,运行install_contrail.yml playbook以部署TF:

代码语言:javascript
复制
# cd /opt/openstack-ansible/playbooks/
# openstack-ansible -e orchestrator=openstack contrail-ansible-deployer/playbooks/install_contrail.yml

安装其它组件

由于Juniper playbooks期望基于Kolla的部署,因此某些组件未安装在基于OpenStack-Ansible的云之上。我自己从Docker容器中提取了一些实用程序、模块,以及更多的内容,并编写了一个playbook来实现这些内容:

代码语言:javascript
复制
# cd /opt/openstack-ansible/playbooks/
# openstack-ansible contrail-ansible-deployer/playbooks/install_contrailtools.yml

重启

一旦TF完成部署,vRouter内核模块已编译并插入后,可能有必要重启主机以清除vRouter代理容器内遇到的一些问题,包括:

代码语言:javascript
复制
contrail-vrouter-agent: controller/src/vnsw/agent/vrouter/ksync/ksync_memory.cc:107: void KSyncMemory::Mmap(bool): Assertion `0' failed.
/entrypoint.sh: line 304: 28142 Aborted (core dumped) $@

我发现在安装后重新启动主机,足以解决这些问题。

部署后的工作

现在我们已经安装了Tungsten Fabric,可以使用 contrail-status命令检查服务的状态:

代码语言:javascript
复制
root@aio1:/opt/openstack-ansible/playbooks# contrail-status
Pod Service Original Name State Status
analytics api contrail-analytics-api running Up 21 minutes
analytics collector contrail-analytics-collector running Up 21 minutes
analytics nodemgr contrail-nodemgr running Up 21 minutes
analytics query-engine contrail-analytics-query-engine running Up 21 minutes
config api contrail-controller-config-api running Up 24 minutes
config cassandra contrail-external-cassandra running Up 27 minutes
config device-manager contrail-controller-config-devicemgr running Up 24 minutes
config nodemgr contrail-nodemgr running Up 24 minutes
config rabbitmq contrail-external-rabbitmq running Up 27 minutes
config schema contrail-controller-config-schema running Up 24 minutes
config svc-monitor contrail-controller-config-svcmonitor running Up 24 minutes
config zookeeper contrail-external-zookeeper running Up 27 minutes
control control contrail-controller-control-control running Up 23 minutes
control dns contrail-controller-control-dns running Up 23 minutes
control named contrail-controller-control-named running Up 23 minutes
control nodemgr contrail-nodemgr running Up 23 minutes
database cassandra contrail-external-cassandra running Up 22 minutes
database nodemgr contrail-nodemgr running Up 22 minutes
database zookeeper contrail-external-zookeeper running Up 22 minutes
vrouter agent contrail-vrouter-agent running Up 19 minutes
vrouter nodemgr contrail-nodemgr running Up 19 minutes
webui job contrail-controller-webui-job running Up 23 minutes
webui web contrail-controller-webui-web running Up 23 minutes
vrouter kernel module is PRESENT
== Contrail control ==
control: active
nodemgr: active
named: active
dns: active
== Contrail database ==
nodemgr: active
zookeeper: active
cassandra: active
== Contrail analytics ==
nodemgr: active
api: initializing (UvePartitions:UVE-Aggregation[Partitions:0] connection down)
collector: initializing (KafkaPub:172.29.236.100:9092 connection down)
query-engine: active
== Contrail webui ==
web: active
job: active
== Contrail vrouter ==
nodemgr: active
agent: active
== Contrail config ==
api: active
zookeeper: active
svc-monitor: active
nodemgr: active
device-manager: active
cassandra: active
rabbitmq: active
schema: active

如果看到nodemgr 服务处于Initializing 状态(NTP状态未同步),请尝试重启主机上的NTP。你可能需要定义多个NTP服务器。我尚未解决Analytics的问题,希望能尽快解决。

一些问题

我想说的是,到这里一切都应该可以正常进行了,但事实并非如此!使用OpenStack或Neutron客户端可能无法正常工作,因为neutron-server 服务可能因以下错误而运行失败:

代码语言:javascript
复制
Unrecoverable error: please check log for details.: ExtensionsNotFound: Extensions not found: ['route-table']. Related to WARNING neutron.api.extensions [-] Extension file vpcroutetable.py wasn't loaded due to cannot import name attributes.

在Rocky中,一个TF插件所依赖而被OpenStack弃用的模块被删除了。不要担心,它只是移动到了另一个模块/文件中。

在 neutron_server容器中,我修改了/openstack/venvs/neutron-18.0.0.0b3/lib/python2.7/site-packages/neutron_plugin_contrail/extensions/vpcroutetable.py文件,如下所示:

代码语言:javascript
复制
-from neutron.api.v2 import attributes as attr
+from neutron_lib.api import attributes as attr

需要重新启动neutron-server服务:

代码语言:javascript
复制
# systemctl restart neutron-server

测试验证

Tungsten Fabric已配置为使用Keystone身份验证,要验证这一点,可以使用外部VIP地址和端口8143在浏览器中打开TF的UI:

输入用户名admin,以及在openrc 文件中定义的密码。域的地方填default。 如果身份验证成功,则登录面板应如下显示:

回到OpenStack,我们可以执行一个openstack network list 的命令来查看网络:

代码语言:javascript
复制
root@aio1-utility-container-ee37a935:~# openstack network list
+--------------------------------------+-------------------------+---------+
| ID | Name | Subnets |
+--------------------------------------+-------------------------+---------+
| 723e67c1-8ccd-43ba-a6f4-8b2399c1b8d2 | __link_local__ | |
| 5a2947ce-0030-4d2a-a06a-76b0d6934102 | ip-fabric | |
| 5f4b0153-8146-4e9c-91d4-c60364ece6bc | default-virtual-network | |
+--------------------------------------+-------------------------+---------+

这些网络都是由TF插件/驱动程序创建的,不应删除。

创建一个测试网络:

代码语言:javascript
复制
root@aio1-utility-container-ee37a935:~# openstack network create test_network_green
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | None |
| availability_zones | None |
| created_at | None |
| description | None |
| dns_domain | None |
| id | d9e0507f-5ef4-4b62-bf69-176340095053 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | None |
| is_vlan_transparent | None |
| mtu | None |
| name | test_network_green |
| port_security_enabled | True |
| project_id | e565909917a5463b867c5a7594a7612f |
| provider:network_type | None |
| provider:physical_network | None |
| provider:segmentation_id | None |
| qos_policy_id | None |
| revision_number | None |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| updated_at | None |
+---------------------------+--------------------------------------+

注意 provider属性是未指定的,但这些对我们不再重要。TF插件可能不支持其它的属性。

创建子网:

代码语言:javascript
复制
root@aio1-utility-container-ee37a935:~# openstack subnet create --subnet-range 172.23.0.0/24 --network test_network_green test_subnet_green
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| allocation_pools | 172.23.0.2-172.23.0.254 |
| cidr | 172.23.0.0/24 |
| created_at | None |
| description | None |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 172.23.0.1 |
| host_routes | |
| id | cc2d2f56-5c87-49fb-afd5-14e32feccd6a |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | test_subnet_green |
| network_id | d9e0507f-5ef4-4b62-bf69-176340095053 |
| project_id | e565909917a5463b867c5a7594a7612f |
| revision_number | None |
| segment_id | None |
| service_types | None |
| subnetpool_id | None |
| tags | |
| updated_at | None |
+-------------------+--------------------------------------+

IPv6应该是支持的,但是在尝试创建IPv6子网时遇到了问题。这里我们的网络已准备好用于VM。为了达到良好的效果,我创建了一个安全组,该安全组可以应用于允许SSH的实例:

代码语言:javascript
复制
root@aio1-utility-container-ee37a935:~# openstack security group create allow_ssh
+-----------------+--------------------------------------+
| Field | Value |
+-----------------+--------------------------------------+
| created_at | None |
| description | allow_ssh |
| id | 39a9e241-27c3-452a-b37a-80b6dcbbf783 |
| name | allow_ssh |
| project_id | e565909917a5463b867c5a7594a7612f |
| revision_number | None |
| rules | |
| tags | [] |
| updated_at | None |
+-----------------+--------------------------------------+
root@aio1-utility-container-ee37a935:~# openstack security group rule create --dst-port 22 allow_ssh
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| created_at | None |
| description | None |
| direction | ingress |
| ether_type | IPv4 |
| id | b8393e4d-1d9d-47e9-877e-86374f38dca1 |
| name | None |
| port_range_max | 22 |
| port_range_min | 22 |
| project_id | e565909917a5463b867c5a7594a7612f |
| protocol | tcp |
| remote_group_id | None |
| remote_ip_prefix | 0.0.0.0/0 |
| revision_number | None |
| security_group_id | 39a9e241-27c3-452a-b37a-80b6dcbbf783 |
| updated_at | None |
+-------------------+--------------------------------------+

随后,我使用tiny flavor和CirrOS镜像启动了实例:

代码语言:javascript
复制
root@aio1-utility-container-ee37a935:~# openstack server create --image cirros --flavor test_flavor --nic net-id=test_network_green --security-group allow_ssh test1
+-------------------------------------+----------------------------------------------------+
| Field | Value |
+-------------------------------------+----------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | a8tghwSoTWZP |
| config_drive | |
| created | 2018-06-18T14:34:49Z |
| flavor | test_flavor (5c0600b7-f9fe-46f3-8af5-f8390ee5c6f3) |
| hostId | |
| id | b14d1861-8855-4d17-a2d3-87eb67a3d81c |
| image | cirros (4006fd58-cdc5-4bd8-bc25-ef73be1cd429) |
| key_name | None |
| name | test1 |
| progress | 0 |
| project_id | e565909917a5463b867c5a7594a7612f |
| properties | |
| security_groups | name='39a9e241-27c3-452a-b37a-80b6dcbbf783' |
| status | BUILD |
| updated | 2018-06-18T14:34:49Z |
| user_id | f6aac1aa53294659998aa71838133a1d |
| volumes_attached | |
+-------------------------------------+----------------------------------------------------+
root@aio1-utility-container-ee37a935:~# openstack server list
+--------------------------------------+-------+--------+-------------------------------+--------+-------------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+-------+--------+-------------------------------+--------+-------------+
| b14d1861-8855-4d17-a2d3-87eb67a3d81c | test1 | ACTIVE | test_network_green=172.23.0.3 | cirros | test_flavor |
+--------------------------------------+-------+--------+-------------------------------+--------+-------------+

现在,我可以连接到实例的控制台,并尝试出站连接:

在Tungsten Fabric UI中,我能够在网络上启用snat ,以允许vRouter对来自VM的出站连接进行snat:

快速测试显示ping正常工作:

到VM的入站连接也是可行的,但需要Tungsten Fabric进行一些额外的工作才能通告VM地址。在我的实验室中有一个Cisco ASA 1001,已配置为与TF控制器建立对等关系,但我们下一次再展示它是如何配置的吧。

总结

对于学习了解Tungsten Fabric的运行方式,以及围绕如何在基于OpenStack-Ansible的云中部署构建最佳实践,还有很多工作要做。用于安装过程的某些组件,被大量包装在Docker容器中,并且必须先提取才能在LXC容器和/或主机中进行部署。这是不可扩展的,但目前来说已经足够了。

最近,我遇到了与opencontrailnightly 版本有关的问题,vRouter丢弃来自VM的出站或响应流量。借助Juniper repo中的GA版本,该问题已经解决了,但并非每个人都可以使用该访问权限。

我遇到的另一个问题是,在往返于VM的ping工作正常(在中间使用ASR)的同时,SSH却连接失败。实际上,任何TCP连接都失败了。在该实例中看到了SYN,并且观察到发送了SYN/ACK。但是,SYN/ACK从未通过vRouter。抓包信息表明,SYN/ACK的校验和无效。当在主机的“physical”接口上禁用通用IP校验和,这种情况下为ens160 ,可以使一切恢复正常。下面这篇文章超级有帮助:

https://kb.juniper.net/InfoCenter/index?page=content&id=KB30500

随着我获得更多的reps,我希望简化流程,并且能有一天将其移到上游以包含在OpenStack-Ansible中。在那之前,祝我们都好运!


作者:James Denton 译者:TF编译组

原文链接:https://www.jimmdenton.com/contrail-osa/

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • 集成要求
  • 从OpenStack-Ansible开始
  • 角色修改
  • os_neutron
  • os_nova
  • HAproxy更改
  • 环境更改
  • Overrides
  • 运行OpenStack-Ansible playbooks
  • 部署Tungsten Fabric
  • Overrides
  • OSA repo更改
  • 运行playbooks
  • 安装其它组件
  • 重启
  • 部署后的工作
  • 一些问题
  • 测试验证
  • 总结
相关产品与服务
容器服务
腾讯云容器服务(Tencent Kubernetes Engine, TKE)基于原生 kubernetes 提供以容器为核心的、高度可扩展的高性能容器管理服务,覆盖 Serverless、边缘计算、分布式云等多种业务部署场景,业内首创单个集群兼容多种计算节点的容器资源管理模式。同时产品作为云原生 Finops 领先布道者,主导开源项目Crane,全面助力客户实现资源优化、成本控制。
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档