本文作者James Denton,拥有超过15年信息技术领域行业经验,目前为知名云计算服务商Rackspace Hosting首席架构师,致力于云网络和对基于OpenStack的Rackspace私有云产品的支持。
Tungsten Fabric(前身为OpenContrail)是Linux基金会旗下的“多云、多技术栈”SDN解决方案。简而言之,Tungsten Fabric和Contrail(基于TF的商业产品)可以替代和增强标准OpenStack云的许多网络组件,并提供以下功能:
·分布式虚拟路由
·DHCP和元数据服务
·基于策略的访问控制
·与安全组的兼容性
·……等等
转发平面支持MPLS over GRE、VXLAN、L2/L3单播和L3多播,以实现虚拟网络和物理网络之间的互连。
注:关于Tungsten Fabric架构的概述可在下面的链接找到:https://tungstenfabric.org.cn/docs
我最近接受了将Tungsten Fabric集成到OpenStack-Ansible中的挑战,以简化TF和相关OpenStack bits在生产级OpenStack云中的部署。本文将在较高级别涵盖OpenStack-Ansible的主(Rocky)分支的一些补丁,以及Juniper和TF社区提供的用于部署Tungsten Fabric的contrail-ansible-deployer playbooks的一些调整。本文所描述的过程绝不意味着是最终的过程,可能会显得比较笨拙且不理想,但是,这是一个开始。
注:本文作者指明文章中的Tungsten Fabric、OpenContrail和Contrail为混用,翻译时统一为Tungsten Fabric。
几周前,我部署了一个独立的3节点Tungsten Fabric设置,其中包含基于Queens的OpenStack-Ansible的All-in-One节点。在弄清楚使事情进入半工作状态所需的调整后,我决定亲自尝试并部署一个AIO节点,该节点包含一个单一的Tungsten Fabric服务实例以及基本的OpenStack服务。
以下为最低规格的配置建议:
·系统: Ubuntu VM
·OS: 16.04.4 LTS
·内存: 48GB
·硬盘: 300GB
·NIC: 单口网卡
如我稍后指出的,裸机节点可能复杂度较小,但是ESXi或其它管理程序上的虚拟机应该是可以尝试的。
首先,请克隆OpenStack-Ansible存储库。在撰写本文时,master分支是与OpenStack的第18个发行版Rocky相关联的。
# git clone https://git.openstack.org/openstack/openstack-ansible /opt/openstack-ansible
# cd /opt/openstack-ansible
# git checkout master
# export ANSIBLE_ROLE_FETCH_MODE=git-clone
接下来,运行引导程序脚本:
# scripts/bootstrap-ansible.sh
# scripts/bootstrap-aio.sh
引导程序脚本将下载playbooks以部署OpenStack,还将在服务器上准备符合OpenStack-Ansible架构的网络环境。
对使用OpenStack-Ansible部署的OpenStack云进行更改,通常意味着对构成部署的Ansible角色进行更改。这包括对任务、模板、变量等的更改。
需要修改的角色包括:
·os_neutron
·os_nova
是否所有这些角色的更改都是必需的,还有待观察,这里只是为了更好地说明。
一些新文件包括:
root@aio1:/etc/ansible/roles/os_neutron# git diff --staged
diff --git a/tasks/providers/opencontrail_config.yml b/tasks/providers/opencontrail_config.yml
new file mode 100644
index 0000000..8f5fc7d
--- /dev/null
+++ b/tasks/providers/opencontrail_config.yml
@@ -0,0 +1,99 @@
+---
+# Copyright 2018, Rackspace Hosting, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+- name: Set the packages to install
+ set_fact:
+ neutron_optional_combined_pip_packages: |-
+ {% set packages = neutron_optional_opencontrail_pip_packages %}
+ {{ packages }}
+
+- name: Install OpenContrail pip packages
+ pip:
+ name: "{{ neutron_optional_combined_pip_packages }}"
+ state: "{{ neutron_pip_package_state }}"
+ virtualenv: "{{ neutron_bin | dirname }}"
+ virtualenv_site_packages: "no"
+ extra_args: >-
+ {{ neutron_developer_mode | ternary(pip_install_developer_constraints | default('--constraint /opt/developer-pip-constraints.txt'), '') }}
+ {{ (pip_install_upper_constraints is defined) | ternary('--constraint ' + pip_install_upper_constraints | default(''),'') }}
+ {{ pip_install_options | default('') }}
+ register: install_packages
+ until: install_packages|success
+ retries: 5
+ delay: 2
+ tags:
+ - opencontrail-install
+ - opencontrail-pip-packages
+
+- name: Install git
+ apt:
+ name: git
+ state: present
+ delegate_to: "{{ item }}"
+ with_items:
+ - "{{ groups['neutron_server'] }}"
+ tags:
+ - opencontrail-install
+
+- name: Clone contrail neutron plugin
+ git:
+ repo: "{{ opencontrail_plugin_git_repo }}"
+ version: "{{ opencontrail_plugin_git_install_branch }}"
+ dest: /opt/contrail-neutron-plugin
+ force: yes
+ register: contrail_plugin_git_clone
+ delegate_to: "{{ item }}"
+ with_items:
+ - "{{ groups['neutron_server'] }}"
+ until: contrail_plugin_git_clone|success
+ retries: 5
+ delay: 2
+ tags:
+ - opencontrail-install
+
+# (jamesdenton) Will need to eventually compile and/or extract from Docker container
+# The tasks marked (temp) should be reworked
+
+- name: Download Contrail python libraries (temp)
+ vars:
+ - dlpath: https://github.com/busterswt/contrail-openstack/raw/master
+ get_url:
+ url: "{{ dlpath }}/{{ item }}"
+ dest: /opt
+ mode: 0440
+ with_items:
+ - contrail-openstack-neutron-init.tar
+ tags:
+ - opencontrail-install
+
+- name: Unpack Contrail python libraries (temp)
+ unarchive:
+ remote_src: yes
+ src: /opt/contrail-openstack-neutron-init.tar
+ dest: /openstack/venvs/neutron-{{ neutron_venv_tag }}/lib/python2.7/site-packages
+ when:
+ inventory_hostname == groups['neutron_server'][0]
+ tags:
+ - opencontrail-install
+
+- name: Install contrail neutron plugin into venv
+ command: "/openstack/venvs/neutron-{{ neutron_venv_tag }}/bin/python setup.py install"
+ args:
+ chdir: /opt/contrail-neutron-plugin
+ delegate_to: "{{ item }}"
+ with_items:
+ - "{{ groups['neutron_server'] }}"
+ tags:
+ - opencontrail-install
diff --git a/templates/plugins/opencontrail/ContrailPlugin.ini.j2 b/templates/plugins/opencontrail/ContrailPlugin.ini.j2
new file mode 100644
index 0000000..9d645b0
--- /dev/null
+++ b/templates/plugins/opencontrail/ContrailPlugin.ini.j2
@@ -0,0 +1,23 @@
+# {{ ansible_managed }}
+
+{% if neutron_plugin_type == 'opencontrail' %}
+[APISERVER]
+api_server_ip = {{ opencontrail_api_vip_address }}
+api_server_port = {{ opencontrail_api_vip_port }}
+multi_tenancy = True
+contrail_extensions = ipam:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_ipam.NeutronPluginContrailIpam,policy:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_policy.NeutronPluginContrailPolicy,route-table:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_vpc.NeutronPluginContrailVpc,contrail:None,service-interface:None,vf-binding:None
+
+[COLLECTOR]
+analytics_api_ip = {{ opencontrail_collector_vip_address }}
+analytics_api_port = {{ opencontrail_collector_vip_port }}
+
+[keystone_authtoken]
+auth_host = {{ internal_lb_vip_address }}
+auth_port = {{ keystone_service_port }}
+auth_protocol = {{ keystone_service_proto }}
+admin_user = {{ keystone_admin_user_name }}
+admin_password = {{ keystone_auth_admin_password }}
+admin_tenant_name = {{ keystone_admin_tenant_name }}
+insecure = True
+region_name = {{ keystone_service_region }}
+{% endif %}
对现有文件的更改包括:
root@aio1:/etc/ansible/roles/os_neutron# git diff
diff --git a/defaults/main.yml b/defaults/main.yml
index 162e933..7054c96 100644
--- a/defaults/main.yml
+++ b/defaults/main.yml
@@ -63,6 +63,8 @@ networking_bgpvpn_git_repo: https://git.openstack.org/openstack/networking-bgpvp
networking_bgpvpn_git_install_branch: master
openstack_ceilometer_git_repo: https://git.openstack.org/openstack/ceilometer
openstack_ceilometer_git_install_branch: master
+opencontrail_plugin_git_repo: https://github.com/Juniper/contrail-neutron-plugin
+opencontrail_plugin_git_install_branch: master
# Developer mode
neutron_developer_mode: false
@@ -164,6 +166,7 @@ neutron_sriov_nic_agent_ini_overrides: {}
neutron_sriov_nic_agent_init_overrides: {}
neutron_vpn_agent_init_overrides: {}
neutron_vpnaas_agent_ini_overrides: {}
+neutron_opencontrail_conf_ini_overrides: {}
###
### Quotas
@@ -434,3 +437,12 @@ ovs_nsh_support: False
# Set higher priority to mardim PPA when ovs_nsh_support is True
ovs_nsh_apt_pinned_packages: [{ package: "*", release: "LP-PPA-mardim-mardim-ppa"}]
+
+###
+### Contrail/OpenContrail/Tungsten Fabric Configuration
+###
+
+opencontrail_api_vip_address: "{{ external_lb_vip_address }}"
+opencontrail_api_vip_port: "8082"
+opencontrail_collector_vip_address: "{{ external_lb_vip_address }}"
+opencontrail_collector_vip_port: "8081"
diff --git a/templates/neutron.conf.j2 b/templates/neutron.conf.j2
index 83d25a7..dd755ca 100644
--- a/templates/neutron.conf.j2
+++ b/templates/neutron.conf.j2
@@ -42,6 +42,10 @@ core_plugin = {{ neutron_plugin_core }}
{% if neutron_plugin_type.split('.')[0] == 'ml2' %}
service_plugins = {{ neutron_plugin_loaded_base | join(',') }}
{% endif %}
+{% if neutron_plugin_type == 'opencontrail' %}
+service_plugins = neutron_plugin_contrail.plugins.opencontrail.loadbalancer.v2.plugin.LoadBalancerPluginV2
+api_extensions_path = /openstack/venvs/neutron-{{ neutron_venv_tag }}/lib/python2.7/site-packages/neutron_plugin_contrail/extensions:/openstack/venvs/neutron-{{ neutron_venv_tag }}/lib/python2.7/site-packages/neutron_lbaas/extensions
+{% endif %}
# MAC address generation for VIFs
base_mac = fa:16:3e:00:00:00
@@ -94,8 +98,9 @@ rpc_workers = {{ neutron_rpc_workers }}
{% set dhcp_agents_max = num_agent if num_agent > 2 else 2 %}
# DHCP
-{% if neutron_plugin_type == 'ml2.dragonflow' %}
-# In dragonflow, DHCP is fully distributed, and DHCP agents are not used
+{% if neutron_plugin_type == ('ml2.dragonflow' or 'opencontrail') %}
+# In dragonflow and opencontrail, DHCP is fully distributed and DHCP
+# agents are not used
dhcp_agent_notification = False
{% else %}
dhcp_agent_notification = True
diff --git a/vars/main.yml b/vars/main.yml
index cef4ee8..2d1c2a2 100644
--- a/vars/main.yml
+++ b/vars/main.yml
@@ -121,6 +121,10 @@ neutron_plugins:
plugin_ini: plugins/ml2/ml2_conf.ini
driver_interface: "openvswitch"
l3_agent_mode: "legacy"
+ opencontrail:
+ plugin_core: neutron_plugin_contrail.plugins.opencontrail.contrail_plugin.NeutronPluginContrailCoreV2
+ plugin_ini: plugins/opencontrail/ContrailPlugin.ini
+ plugin_conf_ini_overrides: "{{ neutron_opencontrail_conf_ini_overrides }}"
###
### ML2 Plugin Configuration
diff --git a/vars/source_install.yml b/vars/source_install.yml
index a246a45..24e57ea 100644
--- a/vars/source_install.yml
+++ b/vars/source_install.yml
@@ -96,6 +96,13 @@ neutron_proprietary_nuage_pip_packages:
- nuage-openstack-neutronclient
- nuagenetlib
+neutron_optional_opencontrail_pip_packages:
+ - bitarray
+ - bottle
+ - geventhttpclient
+ - psutil>=0.6.0
+ - requests>=1.1.0
+
neutron_developer_constraints:
- "git+{{ neutron_git_repo }}@{{ neutron_git_install_branch }}#egg=neutron"
- "git+{{ neutron_fwaas_git_repo }}@{{ neutron_fwaas_git_install_branch }}#egg=neutron-fwaas"
root@aio1:/etc/ansible/roles/os_nova# git diff
diff --git a/defaults/main.yml b/defaults/main.yml
index 67d92e9..bc44511 100644
--- a/defaults/main.yml
+++ b/defaults/main.yml
@@ -325,6 +325,9 @@ nova_network_services:
calico:
use_forwarded_for: True
metadata_proxy_enabled: False
+ opencontrail:
+ use_forwarded_for: True
+ metadata_proxy_enabled: True
# Nova quota
nova_quota_cores: 20
为了提供单一的TF API和仪表板终端,我决定创建一个VIP,以平衡TF API和analytics服务之间的流量。这是否为最佳做法还有待观察,但是此处对group_vars的更改有助于创建VIP:
root@aio1:/opt/openstack-ansible# git diff
diff --git a/inventory/group_vars/haproxy/haproxy.yml b/inventory/group_vars/haproxy/haproxy.yml
index b837443..dc53ef4 100644
--- a/inventory/group_vars/haproxy/haproxy.yml
+++ b/inventory/group_vars/haproxy/haproxy.yml
@@ -36,6 +36,7 @@ haproxy_rabbitmq_management_whitelist_networks: "{{ haproxy_whitelist_networks }
haproxy_repo_git_whitelist_networks: "{{ haproxy_whitelist_networks }}"
haproxy_repo_cache_whitelist_networks: "{{ haproxy_whitelist_networks }}"
haproxy_opendaylight_whitelist_networks: "{{ haproxy_whitelist_networks }}"
+haproxy_opencontrail_whitelist_networks: "{{ haproxy_whitelist_networks }}"
haproxy_default_services:
- service:
@@ -365,3 +366,23 @@ haproxy_default_services:
haproxy_backend_httpcheck_options:
- expect status 405
haproxy_service_enabled: "{{ (groups['ceph-rgw'] is defined and groups['ceph-rgw'] | length > 0) or (ceph_rgws | length > 0) }}"
+ - service:
+ haproxy_service_name: opencontrail-api
+ haproxy_backend_nodes: "{{ groups['opencontrail-api_hosts'] | default([]) }}"
+ haproxy_bind: "{{ [opencontrail_api_vip_address] }}"
+ haproxy_port: 8082
+ haproxy_balance_type: tcp
+ haproxy_timeout_client: 5000s
+ haproxy_timeout_server: 5000s
+ haproxy_whitelist_networks: "{{ haproxy_opencontrail_whitelist_networks }}"
+ haproxy_service_enabled: "{{ neutron_plugin_type == 'opencontrail' }}"
+ - service:
+ haproxy_service_name: opencontrail-collector
+ haproxy_backend_nodes: "{{ groups['opencontrail-analytics_hosts'] | default([]) }}"
+ haproxy_bind: "{{ [opencontrail_collector_vip_address] }}"
+ haproxy_port: 8081
+ haproxy_balance_type: tcp
+ haproxy_timeout_client: 5000s
+ haproxy_timeout_server: 5000s
+ haproxy_whitelist_networks: "{{ haproxy_opencontrail_whitelist_networks }}"
+ haproxy_service_enabled: "{{ neutron_plugin_type == 'opencontrail' }}"
我在All-in-One节点上使用此方法遇到的一些问题,包括HAproxy将VIP绑定到端口8081的能力。后来在过程中发现,Tungsten Fabric playbooks在0.0.0.0:8081上创建了一个侦听器,从而阻止了VIP绑定在同一端口上。这里的替代方法是注释掉该服务,或在部署HAproxy后将其禁用。对于HAproxy在不同节点上的多节点安装,可以保留它。最终,这两种服务的负载平衡可能无法实现,但我暂时将其保留。
默认的Neutron env.d框架将部署原本没必要部署的Neutron代理容器。通过覆盖默认值,我们可以删除代理容器并定义一些新的组件:
cat <<'EOF' >> /etc/openstack_deploy/env.d/neutron.yml
component_skel:
neutron_server:
belongs_to:
- neutron_all
opencontrail_vrouter:
belongs_to:
- neutron_all
opencontrail_api:
belongs_to:
- neutron_all
opencontrail_analytics:
belongs_to:
- neutron_all
container_skel:
neutron_agents_container:
contains: {}
opencontail-vrouter_container:
belongs_to:
- compute_containers
contains:
- opencontrail_vrouter
properties:
is_metal: true
opencontrail-api_container:
belongs_to:
- opencontrail-api_containers
contains:
- opencontrail_api
properties:
is_metal: true
opencontrail-analytics_container:
belongs_to:
- opencontrail-analytics_containers
contains:
- opencontrail_analytics
properties:
is_metal: true
neutron_server_container:
belongs_to:
- network_containers
contains:
- neutron_server
physical_skel:
opencontrail-api_containers:
belongs_to:
- all_containers
opencontrail-api_hosts:
belongs_to:
- hosts
opencontrail-analytics_containers:
belongs_to:
- all_containers
opencontrail-analytics_hosts:
belongs_to:
- hosts
EOF
在这一点上,我不确定所有这些更改都是必要的,以及方法是否正确,但目前暂时这样。
在 openstack_user_config.yml文件中,我定义了两个新的组,以便能够在主机之间拆分API和Analytics服务。由于这是一个AlO节点,因此IP是相同的:
opencontrail-api_hosts:
aio1:
ip: 172.29.236.100
opencontrail-analytics_hosts:
aio1:
ip: 172.29.236.100
对os_neutron角色的更改导致添加了新的默认值,并且还需要一些overrides。在user_variables.yml中添加了以下内容:
neutron_plugin_type: opencontrail
neutron_driver_quota: neutron_plugin_contrail.plugins.opencontrail.quota.driver.QuotaDriver
opencontrail_plugin_git_install_branch: R5.0
角色中指定的一些默认值包括:
opencontrail_api_vip_address: "{{ external_lb_vip_address }}"
opencontrail_api_vip_port: "8082"
opencontrail_collector_vip_address: "{{ external_lb_vip_address }}"
opencontrail_collector_vip_port: "8081"
opencontrail_plugin_git_repo: https://github.com/Juniper/contrail-neutron-plugin
opencontrail_plugin_git_install_branch: master
neutron_opencontrail_conf_ini_overrides: {}
最终要求仍尚未确定。
至此,已经完成了对OpenStack-Ansible playbooks的大部分更改,并且可以部署OpenStack了:
# cd /opt/openstack-ansible/playbooks
# openstack-ansible setup-hosts.yml
# openstack-ansible setup-infrastructure.yml
# openstack-ansible setup-openstack.yml
虽然Neutron会被下线,但在使用网络之前,还需要进行一些其它更改并部署Tungsten Fabric。可以使用Horizon,但可能无法完全发挥它的功能。
接下来,我们必须要执行一些操作,包括克隆Juniper ansible playbook repo,设置overrides,以及运行这些playbooks,以安装Tungsten Fabric,并为与TF相关的服务overlay一些Docker容器。
到这里,我们已经部署了OpenStack,并且知道已实现的网络后端,但是该后端还不存在。Juniper提供的playbooks可以使用Docker容器部署Tungsten Fabric。这些相同的playbooks还可以部署基于Kolla的OpenStack版本,这是另一种OpenStack部署策略。在这里,我们实际上只需要与Tungsten Fabric相关的代码。
为了实现一些更改,我克隆了repo,并进行了部署在基于OpenStack-Ansible的云上所需的必要更改。
克隆repo:
# git clone http://github.com/busterswt/contrail-ansible-deployer /opt/openstack-ansible/playbooks/contrail-ansible-deployer
# cd /opt/openstack-ansible/playbooks/contrail-ansible-deployer
# git checkout osa
Juniper playbooks依赖于它们自己的清单和overrides,因此执行以下几步可能会感觉有些多余。我还没有重新编写playbooks以利用OpenStack-Ansible的可用清单。
可以在/etc/openstack_deploy/user_opencontrail_vars.yml的新文件中定义这些overrides:
config_file: /etc/openstack_deploy/user_opencontrail_vars.yml
opencontrail_api_vip_address: ""
opencontrail_collector_vip_address: ""
provider_config:
bms:
ssh_pwd:
ssh_user: root
ssh_public_key: /root/.ssh/id_rsa.pub
ssh_private_key: /root/.ssh/id_rsa
ntpserver: 129.6.15.28
instances:
aio1:
provider: bms
ip: 172.29.236.100
roles:
config_database:
config:
control:
analytics_database:
analytics:
webui:
vrouter:
VROUTER_GATEWAY: 10.50.0.1
PHYSICAL_INTERFACE: ens160
global_configuration:
CONTAINER_REGISTRY: opencontrailnightly
# CONTAINER_REGISTRY: hub.juniper.net/contrail
# CONTAINER_REGISTRY_USERNAME:
# CONTAINER_REGISTRY_PASSWORD:
contrail_configuration:
CLOUD_ORCHESTRATOR: openstack
CONTRAIL_VERSION: latest
# CONTRAIL_VERSION: 5.0.0-0.40
# UPGRADE_KERNEL: true
UPGRADE_KERNEL: false
KEYSTONE_AUTH_HOST: ""
KEYSTONE_AUTH_PUBLIC_PORT: 5000
KEYSTONE_AUTH_PROTO: http
KEYSTONE_AUTH_URL_VERSION:
KEYSTONE_AUTH_ADMIN_USER: admin
KEYSTONE_AUTH_ADMIN_PASSWORD: ""
KEYSTONE_AUTH_URL_VERSION: /v3
CONTROLLER_NODES: 172.29.236.100
CONTROL_NODES: 172.29.236.100
ANALYTICSDB_NODES: 172.29.236.100
WEBUI_NODES: 172.29.236.100
ANALYTICS_NODES: 172.29.236.100
CONFIGDB_NODES: 172.29.236.100
CONFIG_NODES: 172.29.236.100
请特别注意以下键/值对:
VROUTER_GATEWAY: 10.50.0.1
PHYSICAL_INTERFACE: ens160
CONTAINER_REGISTRY: opencontrailnightly
# CONTAINER_REGISTRY: hub.juniper.net/contrail
# CONTAINER_REGISTRY_USERNAME:
# CONTAINER_REGISTRY_PASSWORD:
CONTRAIL_VERSION: latest
# CONTRAIL_VERSION: 5.0.0-0.40
CONTROLLER_NODES: 172.29.236.100
CONTROL_NODES: 172.29.236.100
ANALYTICSDB_NODES: 172.29.236.100
WEBUI_NODES: 172.29.236.100
ANALYTICS_NODES: 172.29.236.100
CONFIGDB_NODES: 172.29.236.100
CONFIG_NODES: 172.29.236.100
第一部分定义了vRouter所需的vhost0 接口将重新利用或者利用的“physical”接口。这是一个具有单个NIC的AIO节点,这意味着机会很不错,playbooks会自动确定要使用的接口。我们继续前进,同时在此处进行了标注。该主机的IP地址是10.50.0.221,网关是10.50.0.1。对于多宿主主机,有可能将接口及其自己的网关地址专用于vRouter。我尚未进行多NIC部署,但期待能实现。
第二部分定义了将从中下载容器的Docker注册表。这里opencontailnightly 注册表包含每隔一到两天生成的容器,这些容器不一定在任何给定的日期都是有效的。如果你通过Juniper访问GA注册表,还可以定义该注册表并提供访问凭据。在nightly注册表中唯一可用的版本是latest版本,而Juniper注册表可能具有已标记的发行版。记得要使用适当的注册表。
第三部分定义了各个节点类型的地址,用于在整个playbooks中用于将值插入模板。这些是当前成功完成playbooks所必需的。请注意,我在这里使用了“CONTAINER_NET”地址,是希望Tungsten Fabric和OpenStack可以在LXC使用的现有容器网络上进行通信。在此AIO节点中,172.29.236.100是br-mgmt上配置的IP,并且恰好也是内部VIP地址。部署在Docker容器中的服务会将端口绑定到其各自服务的IP上。在多节点的安装中,此更改的方式是否可行仍为待定。
OpenStack-Ansible包括用于软件包管理repo服务器,该服务器目前禁止安装部署Tungsten Fabric所需的某些版本的软件包。在这里,我通过修改/root/.pip/pip.conf禁用了内部repo服务器:
[global]
disable-pip-version-check = true
timeout = 120
#index-url = http://172.29.236.100:8181/simple
#trusted-host =
# 172.29.236.100
[install]
upgrade = true
upgrade-strategy = only-if-needed
pre = true
到这里,我们应该有了一个很好的起点,可以开始运行TF playbooks了。
首先,节点必须运行引导程序:
# cd /opt/openstack-ansible/playbooks/
# openstack-ansible -e orchestrator=openstack contrail-ansible-deployer/playbooks/configure_instances.yml
当UPGRADE_KERNEL 为true时,主机可能会重新启动。如果发生这种情况,请再次重新运行该playbook。如果TF节点不是需要部署的主机,则playbooks具有一个计时器,等待主机返回。
接下来,运行install_contrail.yml playbook以部署TF:
# cd /opt/openstack-ansible/playbooks/
# openstack-ansible -e orchestrator=openstack contrail-ansible-deployer/playbooks/install_contrail.yml
由于Juniper playbooks期望基于Kolla的部署,因此某些组件未安装在基于OpenStack-Ansible的云之上。我自己从Docker容器中提取了一些实用程序、模块,以及更多的内容,并编写了一个playbook来实现这些内容:
# cd /opt/openstack-ansible/playbooks/
# openstack-ansible contrail-ansible-deployer/playbooks/install_contrailtools.yml
一旦TF完成部署,vRouter内核模块已编译并插入后,可能有必要重启主机以清除vRouter代理容器内遇到的一些问题,包括:
contrail-vrouter-agent: controller/src/vnsw/agent/vrouter/ksync/ksync_memory.cc:107: void KSyncMemory::Mmap(bool): Assertion `0' failed.
/entrypoint.sh: line 304: 28142 Aborted (core dumped) $@
我发现在安装后重新启动主机,足以解决这些问题。
现在我们已经安装了Tungsten Fabric,可以使用 contrail-status命令检查服务的状态:
root@aio1:/opt/openstack-ansible/playbooks# contrail-status
Pod Service Original Name State Status
analytics api contrail-analytics-api running Up 21 minutes
analytics collector contrail-analytics-collector running Up 21 minutes
analytics nodemgr contrail-nodemgr running Up 21 minutes
analytics query-engine contrail-analytics-query-engine running Up 21 minutes
config api contrail-controller-config-api running Up 24 minutes
config cassandra contrail-external-cassandra running Up 27 minutes
config device-manager contrail-controller-config-devicemgr running Up 24 minutes
config nodemgr contrail-nodemgr running Up 24 minutes
config rabbitmq contrail-external-rabbitmq running Up 27 minutes
config schema contrail-controller-config-schema running Up 24 minutes
config svc-monitor contrail-controller-config-svcmonitor running Up 24 minutes
config zookeeper contrail-external-zookeeper running Up 27 minutes
control control contrail-controller-control-control running Up 23 minutes
control dns contrail-controller-control-dns running Up 23 minutes
control named contrail-controller-control-named running Up 23 minutes
control nodemgr contrail-nodemgr running Up 23 minutes
database cassandra contrail-external-cassandra running Up 22 minutes
database nodemgr contrail-nodemgr running Up 22 minutes
database zookeeper contrail-external-zookeeper running Up 22 minutes
vrouter agent contrail-vrouter-agent running Up 19 minutes
vrouter nodemgr contrail-nodemgr running Up 19 minutes
webui job contrail-controller-webui-job running Up 23 minutes
webui web contrail-controller-webui-web running Up 23 minutes
vrouter kernel module is PRESENT
== Contrail control ==
control: active
nodemgr: active
named: active
dns: active
== Contrail database ==
nodemgr: active
zookeeper: active
cassandra: active
== Contrail analytics ==
nodemgr: active
api: initializing (UvePartitions:UVE-Aggregation[Partitions:0] connection down)
collector: initializing (KafkaPub:172.29.236.100:9092 connection down)
query-engine: active
== Contrail webui ==
web: active
job: active
== Contrail vrouter ==
nodemgr: active
agent: active
== Contrail config ==
api: active
zookeeper: active
svc-monitor: active
nodemgr: active
device-manager: active
cassandra: active
rabbitmq: active
schema: active
如果看到nodemgr 服务处于Initializing 状态(NTP状态未同步),请尝试重启主机上的NTP。你可能需要定义多个NTP服务器。我尚未解决Analytics的问题,希望能尽快解决。
我想说的是,到这里一切都应该可以正常进行了,但事实并非如此!使用OpenStack或Neutron客户端可能无法正常工作,因为neutron-server 服务可能因以下错误而运行失败:
Unrecoverable error: please check log for details.: ExtensionsNotFound: Extensions not found: ['route-table']. Related to WARNING neutron.api.extensions [-] Extension file vpcroutetable.py wasn't loaded due to cannot import name attributes.
在Rocky中,一个TF插件所依赖而被OpenStack弃用的模块被删除了。不要担心,它只是移动到了另一个模块/文件中。
在 neutron_server容器中,我修改了/openstack/venvs/neutron-18.0.0.0b3/lib/python2.7/site-packages/neutron_plugin_contrail/extensions/vpcroutetable.py文件,如下所示:
-from neutron.api.v2 import attributes as attr
+from neutron_lib.api import attributes as attr
需要重新启动neutron-server服务:
# systemctl restart neutron-server
Tungsten Fabric已配置为使用Keystone身份验证,要验证这一点,可以使用外部VIP地址和端口8143在浏览器中打开TF的UI:
输入用户名admin,以及在openrc 文件中定义的密码。域的地方填default。 如果身份验证成功,则登录面板应如下显示:
回到OpenStack,我们可以执行一个openstack network list 的命令来查看网络:
root@aio1-utility-container-ee37a935:~# openstack network list
+--------------------------------------+-------------------------+---------+
| ID | Name | Subnets |
+--------------------------------------+-------------------------+---------+
| 723e67c1-8ccd-43ba-a6f4-8b2399c1b8d2 | __link_local__ | |
| 5a2947ce-0030-4d2a-a06a-76b0d6934102 | ip-fabric | |
| 5f4b0153-8146-4e9c-91d4-c60364ece6bc | default-virtual-network | |
+--------------------------------------+-------------------------+---------+
这些网络都是由TF插件/驱动程序创建的,不应删除。
创建一个测试网络:
root@aio1-utility-container-ee37a935:~# openstack network create test_network_green
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | None |
| availability_zones | None |
| created_at | None |
| description | None |
| dns_domain | None |
| id | d9e0507f-5ef4-4b62-bf69-176340095053 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | None |
| is_vlan_transparent | None |
| mtu | None |
| name | test_network_green |
| port_security_enabled | True |
| project_id | e565909917a5463b867c5a7594a7612f |
| provider:network_type | None |
| provider:physical_network | None |
| provider:segmentation_id | None |
| qos_policy_id | None |
| revision_number | None |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| updated_at | None |
+---------------------------+--------------------------------------+
注意 provider属性是未指定的,但这些对我们不再重要。TF插件可能不支持其它的属性。
创建子网:
root@aio1-utility-container-ee37a935:~# openstack subnet create --subnet-range 172.23.0.0/24 --network test_network_green test_subnet_green
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| allocation_pools | 172.23.0.2-172.23.0.254 |
| cidr | 172.23.0.0/24 |
| created_at | None |
| description | None |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 172.23.0.1 |
| host_routes | |
| id | cc2d2f56-5c87-49fb-afd5-14e32feccd6a |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | test_subnet_green |
| network_id | d9e0507f-5ef4-4b62-bf69-176340095053 |
| project_id | e565909917a5463b867c5a7594a7612f |
| revision_number | None |
| segment_id | None |
| service_types | None |
| subnetpool_id | None |
| tags | |
| updated_at | None |
+-------------------+--------------------------------------+
IPv6应该是支持的,但是在尝试创建IPv6子网时遇到了问题。这里我们的网络已准备好用于VM。为了达到良好的效果,我创建了一个安全组,该安全组可以应用于允许SSH的实例:
root@aio1-utility-container-ee37a935:~# openstack security group create allow_ssh
+-----------------+--------------------------------------+
| Field | Value |
+-----------------+--------------------------------------+
| created_at | None |
| description | allow_ssh |
| id | 39a9e241-27c3-452a-b37a-80b6dcbbf783 |
| name | allow_ssh |
| project_id | e565909917a5463b867c5a7594a7612f |
| revision_number | None |
| rules | |
| tags | [] |
| updated_at | None |
+-----------------+--------------------------------------+
root@aio1-utility-container-ee37a935:~# openstack security group rule create --dst-port 22 allow_ssh
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| created_at | None |
| description | None |
| direction | ingress |
| ether_type | IPv4 |
| id | b8393e4d-1d9d-47e9-877e-86374f38dca1 |
| name | None |
| port_range_max | 22 |
| port_range_min | 22 |
| project_id | e565909917a5463b867c5a7594a7612f |
| protocol | tcp |
| remote_group_id | None |
| remote_ip_prefix | 0.0.0.0/0 |
| revision_number | None |
| security_group_id | 39a9e241-27c3-452a-b37a-80b6dcbbf783 |
| updated_at | None |
+-------------------+--------------------------------------+
随后,我使用tiny flavor和CirrOS镜像启动了实例:
root@aio1-utility-container-ee37a935:~# openstack server create --image cirros --flavor test_flavor --nic net-id=test_network_green --security-group allow_ssh test1
+-------------------------------------+----------------------------------------------------+
| Field | Value |
+-------------------------------------+----------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | a8tghwSoTWZP |
| config_drive | |
| created | 2018-06-18T14:34:49Z |
| flavor | test_flavor (5c0600b7-f9fe-46f3-8af5-f8390ee5c6f3) |
| hostId | |
| id | b14d1861-8855-4d17-a2d3-87eb67a3d81c |
| image | cirros (4006fd58-cdc5-4bd8-bc25-ef73be1cd429) |
| key_name | None |
| name | test1 |
| progress | 0 |
| project_id | e565909917a5463b867c5a7594a7612f |
| properties | |
| security_groups | name='39a9e241-27c3-452a-b37a-80b6dcbbf783' |
| status | BUILD |
| updated | 2018-06-18T14:34:49Z |
| user_id | f6aac1aa53294659998aa71838133a1d |
| volumes_attached | |
+-------------------------------------+----------------------------------------------------+
root@aio1-utility-container-ee37a935:~# openstack server list
+--------------------------------------+-------+--------+-------------------------------+--------+-------------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+-------+--------+-------------------------------+--------+-------------+
| b14d1861-8855-4d17-a2d3-87eb67a3d81c | test1 | ACTIVE | test_network_green=172.23.0.3 | cirros | test_flavor |
+--------------------------------------+-------+--------+-------------------------------+--------+-------------+
现在,我可以连接到实例的控制台,并尝试出站连接:
在Tungsten Fabric UI中,我能够在网络上启用snat ,以允许vRouter对来自VM的出站连接进行snat:
快速测试显示ping正常工作:
到VM的入站连接也是可行的,但需要Tungsten Fabric进行一些额外的工作才能通告VM地址。在我的实验室中有一个Cisco ASA 1001,已配置为与TF控制器建立对等关系,但我们下一次再展示它是如何配置的吧。
对于学习了解Tungsten Fabric的运行方式,以及围绕如何在基于OpenStack-Ansible的云中部署构建最佳实践,还有很多工作要做。用于安装过程的某些组件,被大量包装在Docker容器中,并且必须先提取才能在LXC容器和/或主机中进行部署。这是不可扩展的,但目前来说已经足够了。
最近,我遇到了与opencontrailnightly 版本有关的问题,vRouter丢弃来自VM的出站或响应流量。借助Juniper repo中的GA版本,该问题已经解决了,但并非每个人都可以使用该访问权限。
我遇到的另一个问题是,在往返于VM的ping工作正常(在中间使用ASR)的同时,SSH却连接失败。实际上,任何TCP连接都失败了。在该实例中看到了SYN,并且观察到发送了SYN/ACK。但是,SYN/ACK从未通过vRouter。抓包信息表明,SYN/ACK的校验和无效。当在主机的“physical”接口上禁用通用IP校验和,这种情况下为ens160 ,可以使一切恢复正常。下面这篇文章超级有帮助:
https://kb.juniper.net/InfoCenter/index?page=content&id=KB30500
随着我获得更多的reps,我希望简化流程,并且能有一天将其移到上游以包含在OpenStack-Ansible中。在那之前,祝我们都好运!
作者:James Denton 译者:TF编译组
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。