DevStack 部署 OpenStack 多节点
DevStack 支持OpenStack多节点部署，下面以一个控制节点和一个计算节点为例，介绍多节点多网卡部署流程。
官方文档：
https://docs.openstack.org/devstack/latest/guides/multinode-lab.html
https://docs.openstack.org/neutron/latest/contributor/testing/ml2_ovn_devstack.html
节点规划
基于Ubuntu 22.04 操作系统，使用 Devstack 部署工具搭建多节点的 OpenStack 开发环境。
节点规划，devstack21 为控制节点，devstack22为计算节点。
| 主机名 | 节点IP | CPU | 内存 | 磁盘 | 网卡 | OS | 角色 |
|---|---|---|---|---|---|---|---|
| devstack21 | 192.168.93.21 | 4C | 8G | 100G | 2 | Ubuntu22.04 | 控制节点 |
| devstack22 | 192.168.93.22 | 4C | 8G | 100G | 2 | Ubuntu22.04 | 计算节点 |
网段规划，规划两个网络平面，对应管理网络和外部网络。租户网络（geneve隧道网络）与管理网络合并。
| 网段 | 网关 | 网卡模式 | 网络类型 |
|---|---|---|---|
| 192.168.93.0/24 | 192.168.93.2 | NAT | 管理网络 |
| 192.168.72.0/24 | 192.168.72.8 | 桥接 | 外部网络 |
网卡规划，每个节点两块网卡，分别对应管理网络和外部网络。
| 主机名 | 网卡 | IP地址 | 网卡模式 | 网络类型 |
|---|---|---|---|---|
| devstack01 | ens33 | 192.168.93.21/24 | NAT | 管理网络 |
| ens37 | NA | 桥接 | 外部网络 | |
| devstack02 | ens33 | 192.168.93.22/24 | NAT | 管理网络 |
| ens37 | NA | 桥接 | 外部网络 |
网络架构图如下：
说明：
- 管理网络需要能够访问外网，部署时需要通过该网络下载依赖，所以选择NAT模式
- 外部网络也可以选择hostonly模式，外部网络不一定是公网
测试环境： VMware Workstation 17
说明：
- 准备两块网卡，注意配置网卡模式
- 也可以勾选处理器–>虚拟化引擎选项，提高openstack创建的实例性能
前置准备
以下操作在所有节点执行。
配置http代理（可选，需要科技）
cat >/etc/profile.d/proxy.sh<<EOF
export http_proxy="http://192.168.93.1:7890"
export https_proxy="http://192.168.93.1:7890"
export no_proxy="192.168.93.0/24,10.0.0.0/8,192.168.72.0/24,localhost,127.0.0.0/8,.localdomain"
EOF
source /etc/profile
说明：否则你可能需要自行配置国内的apt源、pip源，例如阿里云国内源，切换git仓库源码地址为trystack源，另外devstack直接下载二进制文件时你可能只能使用export http_proxy=xxx这种方式，例如stack.sh脚本会执行以下操作。
wget --progress=dot:giga -t 2 -c https://github.com/etcd-io/etcd/releases/download/v3.4.27/etcd-v3.4.27-linux-amd64.tar.gz -O /opt/stack/devstack/files/etcd-v3.4.27-linux-amd64.tar.gz
系统更新
apt update -y && apt upgrade -y
配置主机名
hostnamectl set-hostname devstack21
hostnamectl set-hostname devstack22
配置时间同步
apt install -y chrony
timedatectl set-timezone Asia/Shanghai
devstack21节点网卡配置示例:
root@devstack21:~# cat /etc/netplan/00-installer-config.yaml
network:ethernets:ens33:addresses:- 192.168.93.21/24nameservers:addresses:- 223.5.5.5- 223.6.6.6search: []routes:- to: defaultvia: 192.168.93.2ens37:dhcp4: falseversion: 2
devstack21网卡信息示例如下:
root@devstack21:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000link/ether 00:0c:29:b0:da:db brd ff:ff:ff:ff:ff:ffaltname enp2s1inet 192.168.93.21/24 brd 192.168.93.255 scope global ens33valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:feb0:dadb/64 scope link valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000link/ether 00:0c:29:b0:da:e5 brd ff:ff:ff:ff:ff:ffaltname enp2s5inet6 fe80::20c:29ff:feb0:dae5/64 scope link valid_lft forever preferred_lft forever
说明：devstack22节点配置类似，仅IP不同。
添加 Stack 用户
以下操作在所有节点执行。
sudo useradd -s /bin/bash -d /opt/stack -m stack
确保 stack 用户的主目录对所有人都具有可执行权限，因为基于 RHEL 的发行版使用 700 创建它，而 Ubuntu 21.04+ 使用 750 创建它，这可能会在运行期间导致问题部署。
sudo chmod +x /opt/stack
由于该用户将对您的系统进行许多更改，因此它应该具有 sudo 权限：
echo "stack ALL=(ALL) NOPASSWD: ALL" | sudo tee /etc/sudoers.d/stack
sudo -u stack -i
此时会切换到stack用户视图下。
下载 DevStack
以下操作在所有节点执行。
下载devstack源码
stack@devstack21:~$ pwd
/opt/stack
stack@devstack21:~$ git clone https://opendev.org/openstack/devstack
下载neutron源码
stack@devstack21:~$ pwd
/opt/stack
stack@devstack21:~$ git clone https://opendev.org/openstack/neutron.git
安装控制节点
以下操作在控制节点执行。
配置 DevStack 以使用 OVN 驱动程序。OVN 驱动程序附带了一个示例 DevStack 配置文件，位于neutron项目代码下。
复制neutron示例文件到devstack目录下：
stack@devstack21:~$ cd devstack
stack@devstack21:~/devstack$ pwd
/opt/stack/devstack
stack@devstack21:~$ egrep -v "^#|^$" ../neutron/devstack/ovn-local.conf.sample > ./local.conf
配置文件修改后如下
cat >local.conf<<'EOF'
[[local|localrc]]
# git_base
GIT_BASE="https://github.com"
#GIT_BASE=https://opendev.org
#GIT_BASE=http://git.trystack.cn# password
ADMIN_PASSWORD=secret
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
SERVICE_TOKEN=$ADMIN_PASSWORDHOST_IP=192.168.93.21
SERVICE_HOST=192.168.93.21
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292Q_AGENT=ovn
Q_ML2_PLUGIN_MECHANISM_DRIVERS=ovn,logger
Q_ML2_PLUGIN_TYPE_DRIVERS=local,flat,vlan,geneve
Q_ML2_TENANT_NETWORK_TYPE="geneve"
LOGFILE=$DEST/logs/stack.sh.log
enable_service ovn-northd
enable_service ovn-controller
enable_service q-ovn-metadata-agent
enable_service q-svc
disable_service q-agt
disable_service q-l3
disable_service q-dhcp
disable_service q-meta
enable_plugin neutron https://github.com/openstack/neutron
enable_service q-trunk
enable_service q-dns
enable_service q-port-forwarding
enable_service q-qos
enable_service neutron-segments
enable_service q-log
enable_plugin neutron-tempest-plugin https://github.com/openstack/neutron-tempest-plugin
enable_service horizon
disable_service cinder c-sch c-api c-vol
OVN_BUILD_MODULES=True
ENABLE_CHASSIS_AS_GW=True## Neutron options
FLOATING_RANGE="192.168.72.0/24"
Q_FLOATING_ALLOCATION_POOL=start=192.168.72.110,end=192.168.72.120
PUBLIC_NETWORK_GATEWAY="192.168.72.8"
PUBLIC_INTERFACE=ens37# Open vSwitch provider networking configuration
Q_USE_PROVIDERNET_FOR_PUBLIC=True
OVS_PHYSICAL_BRIDGE=br-ex
PUBLIC_BRIDGE=br-ex
OVS_BRIDGE_MAPPINGS=public:br-ex[[post-config|$NOVA_CONF]]
[scheduler]
discover_hosts_in_cells_interval = 2
EOF
主要配置参数说明：
GIT_BASE: 网络不好时可尝试使用GIT_BASE=http://git.trystack.cnHOST_IP: 本机IP地址FLOATING_RANGE：用于 GUEST VM 访问外部网络，以通过向GUEST VM分配FLOATING IP来从外部网络访问GUEST VM。Q_FLOATING_ALLOCATION_POOL：为devstack分配指定的外部IP地址范围，因为本地其他机器也在使用该物理网段。PUBLIC_INTERFACE: 外部网络网卡名称PUBLIC_NETWORK_GATEWAY：外部网络网关地址Q_USE_PROVIDERNET_FOR_PUBLIC：允许配置Neutron L3 Agent的外部接口使用provider网络，从而实现对OpenStack云实例的公共网络访问。OVS_PHYSICAL_BRIDGE：指定了一个OVS网桥br-ex作为物理网桥，外部网络的流量将通过这个OVS网桥br-ex，底层物理网卡将连接到br-ex网桥上PUBLIC_BRIDGE：直接将之前定义的br-ex网桥指定为公共网络的网桥，所有外部网络的流量都将通过br-ex网桥OVS_BRIDGE_MAPPINGS：映射外部网络到之前定义的br-ex网桥
安装控制节点
$ ./stack.sh
确认安装成功
......
=========================
DevStack Component Timing(times are in seconds)
=========================
wait_for_service 12
async_wait 102
osc 180
apt-get 71
test_with_retry 5
dbsync 7
pip_install 128
apt-get-update 1
run_process 27
git_timed 85
-------------------------
Unaccounted time 184
=========================
Total runtime 802=================Async summary
=================Time spent in the background minus waits: 322 secElapsed time: 802 secTime if we did everything serially: 1124 secSpeedup: 1.4015Post-stack database query stats:
+------------+-----------+-------+
| db | op | count |
+------------+-----------+-------+
| keystone | SELECT | 31269 |
| keystone | INSERT | 82 |
| neutron | SELECT | 4482 |
| neutron | CREATE | 1 |
| neutron | SHOW | 4 |
| neutron | INSERT | 4113 |
| neutron | DELETE | 27 |
| neutron | UPDATE | 116 |
| placement | SELECT | 38 |
| placement | INSERT | 55 |
| placement | SET | 1 |
| nova_api | SELECT | 305 |
| nova_cell0 | SELECT | 65 |
| nova_cell1 | SELECT | 317 |
| nova_cell0 | INSERT | 5 |
| nova_cell0 | UPDATE | 3 |
| nova_cell1 | INSERT | 4 |
| nova_cell1 | UPDATE | 20 |
| placement | UPDATE | 3 |
| glance | SELECT | 57 |
| glance | INSERT | 14 |
| glance | UPDATE | 2 |
| nova_api | INSERT | 23 |
| nova_api | SAVEPOINT | 11 |
| nova_api | RELEASE | 11 |
+------------+-----------+-------+This is your host IP address: 192.168.93.21
This is your host IPv6 address: ::1
Horizon is now available at http://192.168.93.21/dashboard
Keystone is serving at http://192.168.93.21/identity/
The default users are: admin and demo
The password: secretServices are running under systemd unit files.
For more information see:
https://docs.openstack.org/devstack/latest/systemd.htmlDevStack Version: 2024.1
Change: f4f09416ca8de09820b52599c7bcd2911b407466 Merge "Drop unused environments for TripleO and heat agents" 2024-03-19 18:55:14 +0000
OS Version: Ubuntu 22.04 jammy
修改网卡配置，增加bridges部分，使网桥开机自动启动
stack@devstack21:~/devstack$ sudo vim /etc/netplan/00-installer-config.yaml
# This is the network config written by 'subiquity'
network:ethernets:ens33:addresses:- 192.168.93.21/24nameservers:addresses:- 223.5.5.5- 223.6.6.6search: []routes:- to: defaultvia: 192.168.93.2ens37:dhcp4: falsebridges:br-ex:openvswitch: {}dhcp4: falseversion: 2
应用网卡配置
stack@devstack21:~/devstack$ sudo netplan apply
stack@devstack21:~/devstack$
查看网卡信息
stack@devstack21:~/devstack$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000link/ether 00:0c:29:b0:da:db brd ff:ff:ff:ff:ff:ffaltname enp2s1inet 192.168.93.21/24 brd 192.168.93.255 scope global ens33valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:feb0:dadb/64 scope link valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP group default qlen 1000link/ether 00:0c:29:b0:da:e5 brd ff:ff:ff:ff:ff:ffaltname enp2s5inet6 fe80::20c:29ff:feb0:dae5/64 scope link valid_lft forever preferred_lft forever
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000link/ether 22:0d:02:9f:90:1d brd ff:ff:ff:ff:ff:ff
5: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000link/ether f2:ad:f1:af:54:9a brd ff:ff:ff:ff:ff:ff
6: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000link/ether 52:54:00:f2:bf:a4 brd ff:ff:ff:ff:ff:ffinet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0valid_lft forever preferred_lft forever
7: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 00:0c:29:b0:da:e5 brd ff:ff:ff:ff:ff:ffinet6 fe80::20c:29ff:feb0:dae5/64 scope link valid_lft forever preferred_lft forever
查看网桥信息
stack@devstack21:~/devstack$ sudo ovs-vsctl show
8fc32620-e49f-4e8a-87a1-675d4185e1aaManager "ptcp:6640:127.0.0.1"is_connected: trueBridge br-intfail_mode: securedatapath_type: systemPort br-intInterface br-inttype: internalPort patch-br-int-to-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575Interface patch-br-int-to-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575type: patchoptions: {peer=patch-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575-to-br-int}Bridge br-exfail_mode: standalonePort br-exInterface br-extype: internalPort ens37Interface ens37Port patch-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575-to-br-intInterface patch-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575-to-br-inttype: patchoptions: {peer=patch-br-int-to-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575}ovs_version: "2.17.9"
此时已经可以登录dashboard，并创建虚拟机，默认账号为admin/secret。
http://192.168.93.21/
安装计算节点
以下操作在计算节点执行。
从neutron文件夹下复制计算节点示例配置文件
stack@devstack22:~$ pwd
/opt/stack
stack@devstack22:~$ cd devstack
stack@devstack22:~$ egrep -v "^#|^$" ../neutron/devstack/ovn-compute-local.conf.sample > ./local.conf
配置文件修改后如下：
cat >local.conf<<'EOF'
[[local|localrc]]
# git_base
GIT_BASE="https://github.com"
#GIT_BASE=https://opendev.org
#GIT_BASE=http://git.trystack.cn# password
ADMIN_PASSWORD=secret
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
SERVICE_TOKEN=$ADMIN_PASSWORD# OVN
Q_AGENT=ovn
NEUTRON_AGENT=$Q_AGENT
Q_ML2_PLUGIN_MECHANISM_DRIVERS=ovn,logger
Q_ML2_PLUGIN_TYPE_DRIVERS=local,flat,vlan,geneve
Q_ML2_TENANT_NETWORK_TYPE="geneve"
LOGFILE=$DEST/logs/stack.sh.log
enable_plugin neutron https://opendev.org/openstack/neutron
disable_all_services
enable_service n-cpu
enable_service placement-client
enable_service ovn-controller
enable_service q-ovn-metadata-agent
SERVICE_HOST=192.168.93.21
RABBIT_HOST=$SERVICE_HOST
Q_HOST=$SERVICE_HOST
OVN_SB_REMOTE=tcp:$SERVICE_HOST:6642
OVN_BUILD_MODULES=True
HOST_IP=192.168.93.22
NOVA_VNC_ENABLED=True
NOVNCPROXY_URL=http://$SERVICE_HOST:6080/vnc_lite.html
VNCSERVER_LISTEN=$HOST_IP
VNCSERVER_PROXYCLIENT_ADDRESS=$VNCSERVER_LISTEN## Neutron options
Q_USE_SECGROUP=True
FLOATING_RANGE="192.168.72.0/24"
Q_FLOATING_ALLOCATION_POOL=start=192.168.72.110,end=192.168.72.120
PUBLIC_NETWORK_GATEWAY="192.168.72.8"
PUBLIC_INTERFACE=ens37# Open vSwitch provider networking configuration
Q_USE_PROVIDERNET_FOR_PUBLIC=True
OVS_PHYSICAL_BRIDGE=br-ex
PUBLIC_BRIDGE=br-ex
OVS_BRIDGE_MAPPINGS=public:br-ex
EOF
主要配置参数说明：
HOST_IP: 本机IP地址SERVICE_HOST：必须是控制节点IP地址
安装计算节点
$ ./stack.sh
确认安装完成
......
=========================
DevStack Component Timing(times are in seconds)
=========================
wait_for_service 7
async_wait 0
osc 0
apt-get 264
pip_install 103
apt-get-update 1
run_process 4
git_timed 45
-------------------------
Unaccounted time 100
=========================
Total runtime 524=================Async summary
=================Time spent in the background minus waits: 1 secElapsed time: 524 secTime if we did everything serially: 525 secSpeedup: 1.00191This is your host IP address: 192.168.93.22
This is your host IPv6 address: ::1Services are running under systemd unit files.
For more information see:
https://docs.openstack.org/devstack/latest/systemd.htmlDevStack Version: 2024.1
Change: f4f09416ca8de09820b52599c7bcd2911b407466 Merge "Drop unused environments for TripleO and heat agents" 2024-03-19 18:55:14 +0000
OS Version: Ubuntu 22.04 jammy
修改网卡配置，增加bridges部分，使网桥开机自动启动
stack@devstack22:~/devstack$ sudo vim /etc/netplan/00-installer-config.yaml
# This is the network config written by 'subiquity'
network:ethernets:ens33:addresses:- 192.168.93.22/24nameservers:addresses:- 223.5.5.5- 223.6.6.6search: []routes:- to: defaultvia: 192.168.93.2ens37:dhcp4: falsebridges:br-ex:openvswitch: {}dhcp4: falseversion: 2
应用网卡配置
stack@devstack22:~/devstack$ sudo netplan apply
stack@devstack22:~/devstack$
查看网卡信息
stack@devstack22:~/devstack$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000link/ether 00:0c:29:31:a7:2c brd ff:ff:ff:ff:ff:ffaltname enp2s1inet 192.168.93.22/24 brd 192.168.93.255 scope global ens33valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:fe31:a72c/64 scope link valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP group default qlen 1000link/ether 00:0c:29:31:a7:36 brd ff:ff:ff:ff:ff:ffaltname enp2s5inet6 fe80::20c:29ff:fe31:a736/64 scope link valid_lft forever preferred_lft forever
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000link/ether 22:0d:02:9f:90:1d brd ff:ff:ff:ff:ff:ff
5: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000link/ether 06:0d:1b:71:58:0e brd ff:ff:ff:ff:ff:ff
6: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000link/ether 52:54:00:42:16:35 brd ff:ff:ff:ff:ff:ffinet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0valid_lft forever preferred_lft forever
7: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 00:0c:29:31:a7:36 brd ff:ff:ff:ff:ff:ffinet6 fe80::20c:29ff:fe31:a736/64 scope link valid_lft forever preferred_lft forever
8: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000link/ether 1e:dd:eb:ff:7a:66 brd ff:ff:ff:ff:ff:ffinet6 fe80::f4be:3fff:fe9b:6d89/64 scope link valid_lft forever preferred_lft forever
查看网桥信息
stack@devstack22:~/devstack$ sudo ovs-vsctl show
15b21aff-829b-4c9f-8ef6-1fa372eddb3aManager "ptcp:6640:127.0.0.1"is_connected: trueBridge br-exfail_mode: standalonePort br-exInterface br-extype: internalPort ens37Interface ens37Port patch-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575-to-br-intInterface patch-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575-to-br-inttype: patchoptions: {peer=patch-br-int-to-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575}Bridge br-intfail_mode: securedatapath_type: systemPort patch-br-int-to-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575Interface patch-br-int-to-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575type: patchoptions: {peer=patch-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575-to-br-int}Port ovn-2018dc-0Interface ovn-2018dc-0type: geneveoptions: {csum="true", key=flow, remote_ip="192.168.93.21"}bfd_status: {diagnostic="No Diagnostic", flap_count="1", forwarding="true", remote_diagnostic="Neighbor Signaled Session Down", remote_state=up, state=up}Port br-intInterface br-inttype: internalovs_version: "2.17.9"
节点发现
以下操作在控制节点执行。
切换到管理员用户
stack@devstack21:~/devstack$ . openrc admin admin
WARNING: setting legacy OS_TENANT_NAME to support cli tools.
stack@devstack21:~/devstack$
执行节点发现脚本
stack@devstack21:~/devstack$ pwd
/opt/stack/devstack
stack@devstack21:~/devstack$ ./tools/discover_hosts.sh
执行结果如下
stack@devstack21:~/devstack$ ./tools/discover_hosts.sh
INFO dbcounter [None req-1bb45b78-fd14-4fda-bd70-8cb20326bb8e None None] Registered counter for database nova_api
DEBUG dbcounter [-] [131423] Writer thread running {{(pid=131423) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}}
Found 2 cell mappings.
Skipping cell0 since it does not contain hosts.
Getting computes from cell 'cell1': ac941876-1395-4846-befe-ee823eb14d5e
INFO dbcounter [None req-1bb45b78-fd14-4fda-bd70-8cb20326bb8e None None] Registered counter for database nova_cell1
DEBUG dbcounter [-] [131423] Writer thread running {{(pid=131423) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}}
Found 0 unmapped computes in cell: ac941876-1395-4846-befe-ee823eb14d5e
stack@devstack21:~/devstack$
查看新发现的hypervisor
stack@devstack21:~/devstack$ openstack hypervisor list
+--------------------------------------+---------------------+-----------------+---------------+-------+
| ID | Hypervisor Hostname | Hypervisor Type | Host IP | State |
+--------------------------------------+---------------------+-----------------+---------------+-------+
| be439833-6735-4f3e-9ddd-3e37f768e18d | devstack21 | QEMU | 192.168.93.21 | up |
| a2b491c5-8124-4c49-b279-e327d98bb412 | devstack22 | QEMU | 192.168.93.22 | up |
+--------------------------------------+---------------------+-----------------+---------------+-------+
stack@devstack21:~/devstack$
还可以查看 OVN 和 OVS 以查看第二个主机已出现。例如，OVN_Southbound 数据库的 Chassis 表中将有第二个条目。可以使用 ovn-sbctl 实用程序列出chassis、其配置以及绑定到每个chassis的端口：
stack@devstack21:~/devstack$ sudo ovn-sbctl show
Chassis "2018dc69-862f-45d9-ad7d-0fc4dc5d08c4"hostname: devstack21Encap geneveip: "192.168.93.21"options: {csum="true"}Port_Binding cr-lrp-7ff9bc36-6877-44c2-aee9-d9d8eb22c8e0
Chassis "1e889c0c-2509-4cc5-ad6e-f9d3bdfb5c40"hostname: devstack22Encap geneveip: "192.168.93.22"options: {csum="true"}
还可以看到创建到另一个计算节点的隧道：
stack@devstack21:~/devstack$ sudo ovs-vsctl show
......Bridge br-intfail_mode: securedatapath_type: systemPort br-intInterface br-inttype: internalPort ovn-1e889c-0Interface ovn-1e889c-0type: geneveoptions: {csum="true", key=flow, remote_ip="192.168.93.22"}
......
两节点openstack已经部署完成，你可以通过openstack CLI或登录dashboard创建虚拟机实例进一步测试。
创建虚拟机
在本节中，我们将使用openstack CLI逐步完成创建两个虚拟机的步骤，这些虚拟机具有连接到 private Neutron 网络的虚拟 NIC。
DevStack 默认使用 libvirt 作为 Nova 后端。如果 KVM 可用，则会使用它。否则，它只会运行 qemu 模拟来宾。这对于我们的测试来说非常合适，因为我们只需要这些虚拟机能够发送和接收少量流量，因此性能并不是很重要。
虚拟机创建流程
1、切换到demo用户
stack@devstack21:~/devstack$ pwd
/opt/stack/devstack
stack@devstack21:~/devstack$ . openrc demo demo
WARNING: setting legacy OS_TENANT_NAME to support cli tools.
stack@devstack21:~/devstack$
2、获取网络 UUID。
查看可用的网络，devstack默认为我们创建了三个网络，我们将使用 private 租户网络创建虚拟机
stack@devstack21:~/devstack$ openstack network list
+--------------------------------------+---------+----------------------------------------------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+---------+----------------------------------------------------------------------------+
| 6211312f-ab8e-4346-bc58-aed8a63d4f62 | public | 0cf53da9-2c68-4b7d-8043-8d682fe34b23, 8a1dbf88-3caa-4a88-afa4-6fb6823cb224 |
| 8b8cdb32-c009-49e1-88d4-3d58198e6d6c | private | 24bfad1c-46ab-45a8-b607-da0ed8ea0b3c, 912dae4c-691a-44be-afb1-252c19ebfe24 |
| 98a54015-c7ac-4825-80a8-b3364d41db43 | shared | 0e754263-61ee-496a-8187-3103af900846 |
+--------------------------------------+---------+----------------------------------------------------------------------------+
首先从之前的 openstack network list 输出中获取 private 网络的 UUID 并将其保存：
PRIVATE_NET_ID=$(openstack network show private -c id -f value)
3、创建 SSH 密钥对。
接下来在 Nova 中创建 SSH 密钥对。稍后，当我们启动虚拟机时，我们会要求将公钥放入虚拟机中，以便我们可以通过 SSH 访问它。
openstack keypair create demo > id_rsa_demo
chmod 600 id_rsa_demo
查看我们创建的keypair
stack@devstack21:~/devstack$ openstack keypair list
+------+-------------------------------------------------+------+
| Name | Fingerprint | Type |
+------+-------------------------------------------------+------+
| demo | dc:8b:e9:ce:37:c9:bd:16:57:df:c0:20:dc:8c:64:d0 | ssh |
+------+-------------------------------------------------+------+
stack@devstack21:~/devstack$
3、选择一种flavor，devstack默认为我们创建了多种flavor。
我们需要为这些测试虚拟机提供最少的资源，因此 m1.nano 风格就足够了。
stack@devstack21:~/devstack$ openstack flavor list
+----+-----------+-------+------+-----------+-------+-----------+
| ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+----+-----------+-------+------+-----------+-------+-----------+
| 1 | m1.tiny | 512 | 1 | 0 | 1 | True |
| 2 | m1.small | 2048 | 20 | 0 | 1 | True |
| 3 | m1.medium | 4096 | 40 | 0 | 2 | True |
| 4 | m1.large | 8192 | 80 | 0 | 4 | True |
| 42 | m1.nano | 128 | 1 | 0 | 1 | True |
| 5 | m1.xlarge | 16384 | 160 | 0 | 8 | True |
| 84 | m1.micro | 192 | 1 | 0 | 1 | True |
| c1 | cirros256 | 256 | 1 | 0 | 1 | True |
| d1 | ds512M | 512 | 5 | 0 | 1 | True |
| d2 | ds1G | 1024 | 10 | 0 | 1 | True |
| d3 | ds2G | 2048 | 10 | 0 | 2 | True |
| d4 | ds4G | 4096 | 20 | 0 | 4 | True |
+----+-----------+-------+------+-----------+-------+-----------+$ FLAVOR_ID=$(openstack flavor show m1.nano -c id -f value)
4、 选择一个镜像，devstack 默认已经为我们创建了一个cirros镜像
DevStack 默认导入 CirrOS 镜像，这非常适合我们的测试。这是一个非常小的测试图像。
stack@devstack21:~/devstack$ openstack image list
+--------------------------------------+--------------------------+--------+
| ID | Name | Status |
+--------------------------------------+--------------------------+--------+
| df855e1f-4752-4d3c-a81b-6dc5ebfa96ab | cirros-0.6.2-x86_64-disk | active |
+--------------------------------------+--------------------------+--------+$ IMAGE_ID=$(openstack image list -c ID -f value)
5、设置安全规则，以便我们可以访问接下来要启动的虚拟机。
默认情况下，DevStack 不允许用户访问虚拟机，要启用它，我们需要添加一条规则。我们将允许 ICMP 和 SSH。
openstack security group rule create --ingress --ethertype IPv4 --dst-port 22 --protocol tcp default
openstack security group rule create --ingress --ethertype IPv4 --protocol ICMP default
查看创建的安全组规则
$ openstack security group rule list
+--------------------------------------+-------------+-----------+------------+--------------------------------------+--------------------------------------+
| ID | IP Protocol | IP Range | Port Range | Remote Security Group | Security Group |
+--------------------------------------+-------------+-----------+------------+--------------------------------------+--------------------------------------+
...
| ade97198-db44-429e-9b30-24693d86d9b1 | tcp | 0.0.0.0/0 | 22:22 | None | a47b14da-5607-404a-8de4-3a0f1ad3649c |
| d0861a98-f90e-4d1a-abfb-827b416bc2f6 | icmp | 0.0.0.0/0 | | None | a47b14da-5607-404a-8de4-3a0f1ad3649c |
...
+--------------------------------------+-------------+-----------+------------+--------------------------------------+--------------------------------------+
6、启动一些虚拟机。
现在我们将启动两个虚拟机。我们将它们命名为 test1 和 test2 。
stack@devstack21:~/devstack$ openstack server create --nic net-id=$PRIVATE_NET_ID --flavor $FLAVOR_ID --image $IMAGE_ID --key-name demo test1
+--------------------------------------+-----------------------------------------------------------------+
| Field | Value |
+--------------------------------------+-----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | Y5j83PELiYKs |
| config_drive | |
| created | 2024-03-24T04:42:57Z |
| flavor | m1.nano (42) |
| hostId | |
| id | 88ea2c47-7fe9-460b-b98a-0e4ebf773c98 |
| image | cirros-0.6.2-x86_64-disk (df855e1f-4752-4d3c-a81b-6dc5ebfa96ab) |
| key_name | demo |
| name | test1 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| project_id | 6525123a5d8d4f048d51edf1ed18f06b |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2024-03-24T04:42:57Z |
| user_id | 55e4cf4eec0f48efa3ffa182046af847 |
+--------------------------------------+-----------------------------------------------------------------+
stack@devstack21:~/devstack$ stack@devstack21:~/devstack$ openstack server create --nic net-id=$PRIVATE_NET_ID --flavor $FLAVOR_ID --image $IMAGE_ID --key-name demo test2
+--------------------------------------+-----------------------------------------------------------------+
| Field | Value |
+--------------------------------------+-----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | m3dhpx8meFHh |
| config_drive | |
| created | 2024-03-24T04:43:35Z |
| flavor | m1.nano (42) |
| hostId | |
| id | 8a32e126-7f97-4e1a-9a06-059867643785 |
| image | cirros-0.6.2-x86_64-disk (df855e1f-4752-4d3c-a81b-6dc5ebfa96ab) |
| key_name | demo |
| name | test2 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| project_id | 6525123a5d8d4f048d51edf1ed18f06b |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2024-03-24T04:43:34Z |
| user_id | 55e4cf4eec0f48efa3ffa182046af847 |
+--------------------------------------+-----------------------------------------------------------------+
两个虚拟机启动后，它们的状态将为 ACTIVE ：
stack@devstack22:~/devstack$ openstack server list
+--------------------------------------+-------+--------+--------------------------------------------------------+--------------------------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+-------+--------+--------------------------------------------------------+--------------------------+---------+
| 8a32e126-7f97-4e1a-9a06-059867643785 | test2 | ACTIVE | private=10.0.0.54, fd66:a9a:3259:0:f816:3eff:fe5e:4f23 | cirros-0.6.2-x86_64-disk | m1.nano |
| 88ea2c47-7fe9-460b-b98a-0e4ebf773c98 | test1 | ACTIVE | private=10.0.0.36, fd66:a9a:3259:0:f816:3eff:fe6f:4b92 | cirros-0.6.2-x86_64-disk | m1.nano |
+--------------------------------------+-------+--------+--------------------------------------------------------+--------------------------+---------+
stack@devstack22:~/devstack$
虚拟机连接
我们可以通过关联公共网络中的浮动 IP 地址来连接到我们的虚拟机。
stack@devstack22:~/devstack$ TEST1_PORT_ID=$(openstack port list --server test1 -c id -f value)
stack@devstack22:~/devstack$ openstack floating ip create --port $TEST1_PORT_ID public
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at | 2024-03-24T05:26:11Z |
| description | |
| dns_domain | |
| dns_name | |
| fixed_ip_address | 10.0.0.36 |
| floating_ip_address | 192.168.72.119 |
| floating_network_id | 6211312f-ab8e-4346-bc58-aed8a63d4f62 |
| id | 8b830932-5b79-4445-8730-2143e1779e54 |
| name | 192.168.72.119 |
| port_details | {'name': '', 'network_id': '8b8cdb32-c009-49e1-88d4-3d58198e6d6c', 'mac_address': 'fa:16:3e:6f:4b:92', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': |
| | '88ea2c47-7fe9-460b-b98a-0e4ebf773c98', 'device_owner': 'compute:nova'} |
| port_forwardings | [] |
| port_id | 5fc6a820-8b4e-404b-899d-b57090708efd |
| project_id | 6525123a5d8d4f048d51edf1ed18f06b |
| qos_policy_id | None |
| revision_number | 0 |
| router_id | c0e52c09-8791-4a4e-9c82-9cd40d27a65e |
| status | DOWN |
| subnet_id | None |
| tags | [] |
| updated_at | 2024-03-24T05:26:11Z |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
stack@devstack22:~/devstack$
现在应该能够通过其浮动 IP 地址连接到虚拟机。首先，ping 地址。
stack@devstack21:~/devstack$ ping 192.168.72.119 -c 3
PING 192.168.72.119 (192.168.72.119) 56(84) bytes of data.
64 bytes from 192.168.72.119: icmp_seq=1 ttl=128 time=2.43 ms
64 bytes from 192.168.72.119: icmp_seq=2 ttl=128 time=1.75 ms
64 bytes from 192.168.72.119: icmp_seq=3 ttl=128 time=1.03 ms--- 192.168.72.119 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 1.031/1.735/2.425/0.569 ms
stack@devstack21:~/devstack$
现在通过 SSH 连接到虚拟机：
stack@devstack21:~/devstack$ ssh -i id_rsa_demo cirros@192.168.72.119 hostname
test1
stack@devstack21:~/devstack$
登录dashboard
切换到demo项目，查看虚拟机管理器
查看test1实例位于devstack22节点上
查看test2实例位于devstack21节点上
查看网络拓扑
