openEuler 22.03 LTS SP3源码编译部署OpenStack-Caracal遇到的问题解决

问题一给路由设置外部网关后Status为DOWN（使用的是OVN）
- 问题描述
- 临时的解决办法
- 永久解决办法（修改源代码）
问题二分离卷一直显示分离中
- 问题描述
- 解决办法（不太严谨）

问题一给路由设置外部网关后Status为DOWN（使用的是OVN）

问题描述

问题如下图
在这里插入图片描述
经过排查后发现是
Logical_Router_Port的gateway_chassis : []为空导致openstack给路由设置外部网关后Status为DOWN

临时的解决办法

临时的解决办法是手动去添加gateway_chassis
首先通过如下命令查看需要添加的gateway_chassis的_uuid

ovn-nbctl list Logical_Router_Port

在这里插入图片描述再通过如下命令去查看需要添加的chassis值

ovn-sbctl show

在这里插入图片描述
在通过如下的命令去手动的添加即可

ovn-nbctl lrp-set-gateway-chassis _uuid值 Chassis值

在这里插入图片描述

永久解决办法（修改源代码）

只所以会出现这个状况是因为在代码中添加chassis无法匹配到合适的
具体如下
有关代码的路径如下
/usr/local/lib/python3.9/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb
代码文件名叫ovn_client.py
未修改源代码如下

def get_candidates_for_scheduling(self, physnet, cms=None,chassis_physnets=None,availability_zone_hints=None):"""Return chassis for scheduling gateway router.Criteria for selecting chassis as candidates1) Chassis from cms with proper bridge mappings only (that means thesegateway chassis with the requested physical network).2) Filter the available chassis accordingly to the routersavailability zone hints (if present)If the logical router port belongs to a tunnelled network, there won'tbe any candidate."""# TODO(lucasagomes): Simplify the logic here, the CMS option has# been introduced long ago and by now all gateway chassis should# include it. This will match the logic in the is_gateway_chassis()# (utils.py)cms = cms or self._sb_idl.get_gateway_chassis_from_cms_options()chassis_physnets = (chassis_physnets orself._sb_idl.get_chassis_and_physnets())candidates = set()for chassis, physnets in chassis_physnets.items():if (physnet andphysnet in physnets andchassis in cms):candidates.add(chassis)candidates = list(candidates)# Filter for availability zonesif availability_zone_hints:LOG.debug('Filtering Chassis candidates by availability zone ''hints: %s', ', '.join(availability_zone_hints))candidates = [ch for ch in candidatesfor az in availability_zone_hintsif az in utils.get_chassis_availability_zones(self._sb_idl.lookup('Chassis', ch, None))]LOG.debug('Chassis candidates for scheduling gateway router ports ''for "%s" physical network: %s', physnet, candidates)return candidates

主要的原因是获取到cms列表未空导致无法匹配到合适gateway_chassis将其添加
在这里插入图片描述解决办法有两个

不需要修改源代码，去添加一个cms标记，让代码那个获取到，但是由于我是第一次遇到这个情况，我不知道要设置什么样的cms标记。所以无法从这方面入手。
修该源代码去添加一个处理cms列表为空的逻辑
添加一个处理cms列表为空的逻辑，代码如下：

def get_candidates_for_scheduling(self, physnet, cms=None, chassis_physnets=None, availability_zone_hints=None):"""Return chassis for scheduling gateway router.Criteria for selecting chassis as candidates:1) Chassis from cms with proper bridge mappings only (that means thesegateway chassis with the requested physical network).2) Filter the available chassis accordingly to the routersavailability zone hints (if present)If the logical router port belongs to a tunnelled network, there won'tbe any candidate."""cms = cms or self._sb_idl.get_gateway_chassis_from_cms_options()chassis_physnets = chassis_physnets or self._sb_idl.get_chassis_and_physnets()candidates = set()# If CMS is empty, we may assume all chassis are managedmanaged_chassis = cms if cms else [chassis for chassis in chassis_physnets]for chassis in managed_chassis:physnets = chassis_physnets.get(chassis, [])if physnet in physnets:candidates.add(chassis)# Convert candidates set to listcandidates = list(candidates)# Filter for availability zonesif availability_zone_hints:LOG.debug('Filtering Chassis candidates by availability zone hints: %s', ', '.join(availability_zone_hints))filtered_candidates = []for ch in candidates:azs = utils.get_chassis_availability_zones(self._sb_idl.lookup('Chassis', ch, None))if any(az in azs for az in availability_zone_hints):filtered_candidates.append(ch)candidates = filtered_candidatesLOG.debug('Chassis candidates for scheduling gateway router ports for "%s" physical network: %s', physnet, candidates)return candidates

修改完成后重启一下neutron服务即可。

问题二分离卷一直显示分离中

问题描述

创建了一个10G的卷
在这里插入图片描述连接到实例

进行分离卷

解决办法（不太严谨）

根据日志提示

2024-05-27 20:08:29.656 59902 ERROR cinder.volume.api [req-e5dea5da-9565-4e0e-b05b-9dbc58f7deef req-87c34a80-75b3-4bdd-87bf-1136cbb88b96 950f09833b4e4dffaf4acf3ac9f1d4bd 840a040edf78409f92d7c7a128652718 - - default default] Detected user call to delete in-use attachment. Call must come from the nova service and nova must be configured to send the service token. Bug #2004555
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault [req-e5dea5da-9565-4e0e-b05b-9dbc58f7deef req-87c34a80-75b3-4bdd-87bf-1136cbb88b96 950f09833b4e4dffaf4acf3ac9f1d4bd 840a040edf78409f92d7c7a128652718 - - default default] Caught error: <class 'cinder.exception.ConflictNovaUsingAttachment'> Detach volume from instance 4164eba5-b07e-4546-8d41-c4cdc030c396 using the Compute API: cinder.exception.ConflictNovaUsingAttachment: Detach volume from instance 4164eba5-b07e-4546-8d41-c4cdc030c396 using the Compute API
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault Traceback (most recent call last):
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/cinder/api/middleware/fault.py", line 84, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     return req.get_response(self.application)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     status, headers, app_iter = self.call_application(
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     app_iter = application(self.environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 143, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     return resp(environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     resp = self.call_func(req, *args, **kw)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     return self.func(req, *args, **kwargs)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/osprofiler/web.py", line 111, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     return request.get_response(self.application)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     status, headers, app_iter = self.call_application(
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     app_iter = application(self.environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     resp = self.call_func(req, *args, **kw)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     return self.func(req, *args, **kwargs)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/keystonemiddleware/auth_token/__init__.py", line 340, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     response = req.get_response(self._app)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     status, headers, app_iter = self.call_application(
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     app_iter = application(self.environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 143, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     return resp(environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 143, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     return resp(environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/routes/middleware.py", line 153, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     response = self.app(environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 143, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     return resp(environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     resp = self.call_func(req, *args, **kw)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     return self.func(req, *args, **kwargs)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/cinder/api/openstack/wsgi.py", line 839, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     return self._process_stack(request, action, action_args,
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/cinder/api/openstack/wsgi.py", line 900, in _process_stack
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     action_result = self.dispatch(meth, request, action_args)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/cinder/api/openstack/wsgi.py", line 995, in dispatch
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     return method(req=request, **action_args)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/cinder/api/openstack/wsgi.py", line 1160, in version_select
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     return func.func(self, *args, **kwargs)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/cinder/api/v3/attachments.py", line 282, in delete
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     attachments = self.volume_api.attachment_delete(context, attachment)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/cinder/volume/api.py", line 2668, in attachment_delete
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     self.attachment_deletion_allowed(ctxt, attachment)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault   File "/usr/local/lib/python3.9/site-packages/cinder/volume/api.py", line 2659, in attachment_deletion_allowed
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault     raise exception.ConflictNovaUsingAttachment(instance_id=server_id)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault cinder.exception.ConflictNovaUsingAttachment: Detach volume from instance 4164eba5-b07e-4546-8d41-c4cdc030c396 using the Compute API
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault

提示我using the Compute API
于是我去设置如下的内容
controller节点

vim /etc/nova/nova.conf

[cinder]
send_service_user_token = True #添加

compute节点

vim /etc/nova/nova.conf

[cinder]
send_service_user_token = True #添加

但是没有解决，还是提示同样的问题
于是去查看源代码
在/usr/local/lib/python3.9/site-packages/cinder/volume/api.py
源代码如下

def attachment_deletion_allowed(self,ctxt: context.RequestContext,attachment_or_attachment_id,volume=None):"""Check if deleting an attachment is allowed (Bug #2004555)Allowed is based on the REST API policy, the status of the attachment,where it is used, and who is making the request.Deleting an attachment on the Cinder side while leaving the volumeconnected to the nova host results in leftover devices that can lead todata leaks/corruption.OS-Brick may have code to detect it, but in some cases it is detectedafter it has already been exposed, so it's better to prevent users frombeing able to intentionally triggering the issue."""# It's ok to delete an attachment if the request comes from a serviceif self.is_service_request(ctxt):returnif not attachment_or_attachment_id and volume:if not volume.volume_attachment:returnif len(volume.volume_attachment) == 1:attachment_or_attachment_id = volume.volume_attachment[0]if isinstance(attachment_or_attachment_id, str):try:attachment = objects.VolumeAttachment.get_by_id(ctxt, attachment_or_attachment_id)except exception.VolumeAttachmentNotFound:attachment = Noneelse:attachment = attachment_or_attachment_idif attachment:if volume:if volume.id != attachment.volume_id:raise exception.InvalidInput(reason='Mismatched volume and attachment')server_id = attachment.instance_uuid# It's ok to delete if it's not connected to a vm.if not server_id or not attachment.connection_info:returnvolume = volume or attachment.volumenova = compute.API()LOG.info('Attachment connected to vm %s, checking data on nova',server_id)# If nova is down the client raises 503 and we report thattry:nova_volume = nova.get_server_volume(ctxt, server_id,volume.id)except nova.NotFound:LOG.warning('Instance or volume not found on Nova, deleting ''attachment locally, which may leave leftover ''devices on Nova compute')returnif nova_volume.attachment_id != attachment.id:LOG.warning('Mismatch! Nova has different attachment id (%s) ''for the volume, deleting attachment locally. ''May leave leftover devices in a compute node',nova_volume.attachment_id)returnelse:server_id = ''LOG.error('Detected user call to delete in-use attachment. Call must ''come from the nova service and nova must be configured to ''send the service token. Bug #2004555')raise exception.ConflictNovaUsingAttachment(instance_id=server_id)

添加一个逻辑处理，通过实例的uuid再一次确认它是否有附加卷

instance_uuid = attachment.instance_uuid
try:attachments = objects.VolumeAttachmentList.get_all_by_instance_uuid(ctxt, instance_uuid)if attachments:LOG.info("Instance {} has other attachments: {}".format(instance_uuid, [att.id for att in attachments]))else:LOG.info("No other attachments found for instance {}".format(instance_uuid))return
except Exception as e:LOG.error("Failed to retrieve attachments for instance {}: {}".format(instance_uuid, str(e)))raise exception.VolumeBackendAPIException(reason=str(e))

修改后的完整的代码如下

def attachment_deletion_allowed(self,ctxt: context.RequestContext,attachment_or_attachment_id,volume=None):"""Check if deleting an attachment is allowed (Bug #2004555)Allowed is based on the REST API policy, the status of the attachment,where it is used, and who is making the request.Deleting an attachment on the Cinder side while leaving the volumeconnected to the nova host results in leftover devices that can lead todata leaks/corruption.OS-Brick may have code to detect it, but in some cases it is detectedafter it has already been exposed, so it's better to prevent users frombeing able to intentionally triggering the issue."""# It's ok to delete an attachment if the request comes from a serviceif self.is_service_request(ctxt):returnif not attachment_or_attachment_id and volume:if not volume.volume_attachment:returnif len(volume.volume_attachment) == 1:attachment_or_attachment_id = volume.volume_attachment[0]if isinstance(attachment_or_attachment_id, str):try:attachment = objects.VolumeAttachment.get_by_id(ctxt, attachment_or_attachment_id)except exception.VolumeAttachmentNotFound:attachment = Noneelse:attachment = attachment_or_attachment_idif attachment:if volume:if volume.id != attachment.volume_id:raise exception.InvalidInput(reason='Mismatched volume and attachment')server_id = attachment.instance_uuid# It's ok to delete if it's not connected to a vm.if not server_id or not attachment.connection_info:returnvolume = volume or attachment.volumenova = compute.API()LOG.info('Attachment connected to vm %s, checking data on nova',server_id)# If nova is down the client raises 503 and we report thattry:nova_volume = nova.get_server_volume(ctxt, server_id,volume.id)except nova.NotFound:LOG.warning('Instance or volume not found on Nova, deleting ''attachment locally, which may leave leftover ''devices on Nova compute')returnif nova_volume.attachment_id != attachment.id:LOG.warning('Mismatch! Nova has different attachment id (%s) ''for the volume, deleting attachment locally. ''May leave leftover devices in a compute node',nova_volume.attachment_id)returnelse:server_id = ''instance_uuid = attachment.instance_uuidtry:attachments = objects.VolumeAttachmentList.get_all_by_instance_uuid(ctxt, instance_uuid)if attachments:LOG.info("Instance {} has other attachments: {}".format(instance_uuid, [att.id for att in attachments]))else:LOG.info("No other attachments found for instance {}".format(instance_uuid))returnexcept Exception as e:LOG.error("Failed to retrieve attachments for instance {}: {}".format(instance_uuid, str(e)))raise exception.VolumeBackendAPIException(reason=str(e))