OceanBase 升级过程研究(4.2.1.6-4.2.1.8)

模拟业务

使用benchmark加载10仓数据模拟业务场景

升级方法

使用滚动升级方式来进行OB升级。该方法前提是OB集群必须满足官方规定的高可用架构(如果 Zone 个数小于 3，滚动升级时则无法构成多数派), 滚动升级的原理就是轮流完成每个ZONE的升级工作，由于OB属于分布式集群，当一个ZONE处于升级状态中，其余的ZONE提供服务，不影响业务

任务类型

升级时会产生一个主任务及多个子任务。主任务根据每个需要替换 binary 版本的升级节点，来生成数个子任务，来逐步替换掉需要升级的 binary 文件。

  • 主任务：将升级路径中每个需要替换 binary 文件的升级节点都生成为一个子任务，用于逐步替换掉需要升级的 binary 文件。 如下图所示，Submit cluster upgrade task 即为生成一个子任务。Wait dag success 为等待子任务执行完成。Submit cluster upgrade task 和 Wait dag success 这个组合可能会执行多次，执行次数取决于升级路径中有多少个替换 binary 的节点。

  • 子任务：升级所有 OceanBase 集群版本。主要是执行升级脚本: 预检查脚本、升级 Pre 脚本、替换 OBServer 节点、升级 Post 脚本、版本检查等。

升级任务(子任务)

由于是滚动升级，所以子任务中的升级任务数量取决于ZONE的数量

升级前检查

在 set zone contaxt 任务模块前的所有任务均是升级前对集群的检查，检查租户的凭据以及运行安装目录etc下的升级检查脚本，查看当前集群是否满足升级。由于是检查操作，所以这些任务模块如果运行失败可以进行回滚。这些检查模块运行成功后开始执行ZONE的升级

upgrade_checker.py 脚本进行升级前置检查，脚本执行成功表示可以继续升级

upgrade_pre.py 该脚本主要用于执行升级前的一系列准备工作，确保集群处于适合升级的状态

upgrade_health_checker.py该脚本主要用于执行集群级别的健康检查，确保集群在升级前处于健康状态

upgrade_checker.py脚本内容

[root@sdw1 etc]# cat upgrade_checker.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-import sys
import os
import mysql.connector
from mysql.connector import errorcode
import logging
import getopt
import timeclass UpgradeParams:log_filename = 'upgrade_checker.log'old_version = '4.0.0.0'
#### --------------start : my_error.py --------------
class MyError(Exception):def __init__(self, value):self.value = valuedef __str__(self):return repr(self.value)
#### --------------start : actions.py------------
class Cursor:__cursor = Nonedef __init__(self, cursor):self.__cursor = cursordef exec_sql(self, sql, print_when_succ = True):try:self.__cursor.execute(sql)rowcount = self.__cursor.rowcountif True == print_when_succ:logging.info('succeed to execute sql: %s, rowcount = %d', sql, rowcount)return rowcountexcept mysql.connector.Error, e:logging.exception('mysql connector error, fail to execute sql: %s', sql)raise eexcept Exception, e:logging.exception('normal error, fail to execute sql: %s', sql)raise edef exec_query(self, sql, print_when_succ = True):try:self.__cursor.execute(sql)results = self.__cursor.fetchall()rowcount = self.__cursor.rowcountif True == print_when_succ:logging.info('succeed to execute query: %s, rowcount = %d', sql, rowcount)return (self.__cursor.description, results)except mysql.connector.Error, e:logging.exception('mysql connector error, fail to execute sql: %s', sql)raise eexcept Exception, e:logging.exception('normal error, fail to execute sql: %s', sql)raise edef set_parameter(cur, parameter, value):sql = """alter system set {0} = '{1}'""".format(parameter, value)logging.info(sql)cur.execute(sql)wait_parameter_sync(cur, parameter, value)def wait_parameter_sync(cur, key, value):sql = """select count(*) as cnt from oceanbase.__all_virtual_sys_parameter_statwhere name = '{0}' and value != '{1}'""".format(key, value)times = 10while times > 0:logging.info(sql)cur.execute(sql)result = cur.fetchall()if len(result) != 1 or len(result[0]) != 1:logging.exception('result cnt not match')raise eelif result[0][0] == 0:logging.info("""{0} is sync, value is {1}""".format(key, value))breakelse:logging.info("""{0} is not sync, value should be {1}""".format(key, value))times -= 1if times == 0:logging.exception("""check {0}:{1} sync timeout""".format(key, value))raise etime.sleep(5)#### --------------start :  opt.py --------------
help_str = \
"""
Help:
""" +\
sys.argv[0] + """ [OPTIONS]""" +\
'\n\n' +\
'-I, --help          Display this help and exit.\n' +\
'-V, --version       Output version information and exit.\n' +\
'-h, --host=name     Connect to host.\n' +\
'-P, --port=name     Port number to use for connection.\n' +\
'-u, --user=name     User for login.\n' +\
'-t, --timeout=name  Cmd/Query/Inspection execute timeout(s).\n' +\
'-p, --password=name Password to use when connecting to server. If password is\n' +\
'                    not given it\'s empty string "".\n' +\
'-m, --module=name   Modules to run. Modules should be a string combined by some of\n' +\
'                    the following strings: ddl, normal_dml, each_tenant_dml,\n' +\
'                    system_variable_dml, special_action, all. "all" represents\n' +\
'                    that all modules should be run. They are splitted by ",".\n' +\
'                    For example: -m all, or --module=ddl,normal_dml,special_action\n' +\
'-l, --log-file=name Log file path. If log file path is not given it\'s ' + os.path.splitext(sys.argv[0])[0] + '.log\n' +\
'\n\n' +\
'Maybe you want to run cmd like that:\n' +\
sys.argv[0] + ' -h 127.0.0.1 -P 3306 -u admin -p admin\n'version_str = """version 1.0.0"""class Option:__g_short_name_set = set([])__g_long_name_set = set([])__short_name = None__long_name = None__is_with_param = None__is_local_opt = None__has_value = None__value = Nonedef __init__(self, short_name, long_name, is_with_param, is_local_opt, default_value = None):if short_name in Option.__g_short_name_set:raise MyError('duplicate option short name: {0}'.format(short_name))elif long_name in Option.__g_long_name_set:raise MyError('duplicate option long name: {0}'.format(long_name))Option.__g_short_name_set.add(short_name)Option.__g_long_name_set.add(long_name)self.__short_name = short_nameself.__long_name = long_nameself.__is_with_param = is_with_paramself.__is_local_opt = is_local_optself.__has_value = Falseif None != default_value:self.set_value(default_value)def is_with_param(self):return self.__is_with_paramdef get_short_name(self):return self.__short_namedef get_long_name(self):return self.__long_namedef has_value(self):return self.__has_valuedef get_value(self):return self.__valuedef set_value(self, value):self.__value = valueself.__has_value = Truedef is_local_opt(self):return self.__is_local_optdef is_valid(self):return None != self.__short_name and None != self.__long_name and True == self.__has_value and None != self.__valueg_opts =\
[\
Option('I', 'help', False, True),\
Option('V', 'version', False, True),\
Option('h', 'host', True, False),\
Option('P', 'port', True, False),\
Option('u', 'user', True, False),\
Option('t', 'timeout', True, False, 0),\
Option('p', 'password', True, False, ''),\
# 要跑哪个模块，默认全跑
Option('m', 'module', True, False, 'all'),\
# 日志文件路径，不同脚本的main函数中中会改成不同的默认值
Option('l', 'log-file', True, False)
]\def change_opt_defult_value(opt_long_name, opt_default_val):global g_optsfor opt in g_opts:if opt.get_long_name() == opt_long_name:opt.set_value(opt_default_val)returndef has_no_local_opts():global g_optsno_local_opts = Truefor opt in g_opts:if opt.is_local_opt() and opt.has_value():no_local_opts = Falsereturn no_local_optsdef check_db_client_opts():global g_optsfor opt in g_opts:if not opt.is_local_opt() and not opt.has_value():raise MyError('option "-{0}" has not been specified, maybe you should run "{1} --help" for help'\.format(opt.get_short_name(), sys.argv[0]))def parse_option(opt_name, opt_val):global g_optsfor opt in g_opts:if opt_name in (('-' + opt.get_short_name()), ('--' + opt.get_long_name())):opt.set_value(opt_val)def parse_options(argv):global g_optsshort_opt_str = ''long_opt_list = []for opt in g_opts:if opt.is_with_param():short_opt_str += opt.get_short_name() + ':'else:short_opt_str += opt.get_short_name()for opt in g_opts:if opt.is_with_param():long_opt_list.append(opt.get_long_name() + '=')else:long_opt_list.append(opt.get_long_name())(opts, args) = getopt.getopt(argv, short_opt_str, long_opt_list)for (opt_name, opt_val) in opts:parse_option(opt_name, opt_val)if has_no_local_opts():check_db_client_opts()def deal_with_local_opt(opt):if 'help' == opt.get_long_name():global help_strprint help_strelif 'version' == opt.get_long_name():global version_strprint version_strdef deal_with_local_opts():global g_optsif has_no_local_opts():raise MyError('no local options, can not deal with local options')else:for opt in g_opts:if opt.is_local_opt() and opt.has_value():deal_with_local_opt(opt)# 只处理一个returndef get_opt_host():global g_optsfor opt in g_opts:if 'host' == opt.get_long_name():return opt.get_value()def get_opt_port():global g_optsfor opt in g_opts:if 'port' == opt.get_long_name():return opt.get_value()def get_opt_user():global g_optsfor opt in g_opts:if 'user' == opt.get_long_name():return opt.get_value()def get_opt_password():global g_optsfor opt in g_opts:if 'password' == opt.get_long_name():return opt.get_value()def get_opt_timeout():global g_optsfor opt in g_opts:if 'timeout' == opt.get_long_name():return opt.get_value()def get_opt_module():global g_optsfor opt in g_opts:if 'module' == opt.get_long_name():return opt.get_value()def get_opt_log_file():global g_optsfor opt in g_opts:if 'log-file' == opt.get_long_name():return opt.get_value()
#### ---------------end----------------------#### --------------start :  do_upgrade_pre.py--------------
def config_logging_module(log_filenamme):logging.basicConfig(level=logging.INFO,\format='[%(asctime)s] %(levelname)s %(filename)s:%(lineno)d %(message)s',\datefmt='%Y-%m-%d %H:%M:%S',\filename=log_filenamme,\filemode='w')# 定义日志打印格式formatter = logging.Formatter('[%(asctime)s] %(levelname)s %(filename)s:%(lineno)d %(message)s', '%Y-%m-%d %H:%M:%S')######################################## 定义一个Handler打印INFO及以上级别的日志到sys.stdoutstdout_handler = logging.StreamHandler(sys.stdout)stdout_handler.setLevel(logging.INFO)# 设置日志打印格式stdout_handler.setFormatter(formatter)# 将定义好的stdout_handler日志handler添加到root loggerlogging.getLogger('').addHandler(stdout_handler)
#### ---------------end----------------------fail_list=[]def get_version(version_str):versions = version_str.split(".")if len(versions) != 4:logging.exception("""version:{0} is invalid""".format(version_str))raise emajor = int(versions[0])minor = int(versions[1])major_patch = int(versions[2])minor_patch = int(versions[3])if major > 0xffffffff or minor > 0xffff or major_patch > 0xff or minor_patch > 0xff:logging.exception("""version:{0} is invalid""".format(version_str))raise eversion = (major << 32) | (minor << 16) | (major_patch << 8) | (minor_patch)return version#### START ####
# 1. 检查前置版本
def check_observer_version(query_cur, upgrade_params):(desc, results) = query_cur.exec_query("""select distinct value from GV$OB_PARAMETERS  where name='min_observer_version'""")if len(results) != 1:fail_list.append('min_observer_version is not sync')elif cmp(results[0][0], upgrade_params.old_version) < 0 :fail_list.append('old observer version is expected equal or higher then: {0}, actual version:{1}'.format(upgrade_params.old_version, results[0][0]))logging.info('check observer version success, version = {0}'.format(results[0][0]))def check_data_version(query_cur, input_min_cluster_version):min_cluster_version = 0sql = """select distinct value from GV$OB_PARAMETERS  where name='min_observer_version'"""(desc, results) = query_cur.exec_query(sql)if len(results) != 1:fail_list.append('min_observer_version is not sync')elif len(results[0]) != 1:fail_list.append('column cnt not match')else:min_cluster_version = get_version(results[0][0])input_min_cluster_version[0] = min_cluster_version# check data versionif min_cluster_version < get_version("4.1.0.0"):# last barrier cluster version should be 4.1.0.0fail_list.append('last barrier cluster version is 4.1.0.0. prohibit cluster upgrade from cluster version less than 4.1.0.0')else:data_version_str = ''data_version = 0# check compatible is samesql = """select distinct value from oceanbase.__all_virtual_tenant_parameter_info where name='compatible'"""(desc, results) = query_cur.exec_query(sql)if len(results) != 1:fail_list.append('compatible is not sync')elif len(results[0]) != 1:fail_list.append('column cnt not match')else:data_version_str = results[0][0]data_version = get_version(results[0][0])if data_version < get_version("4.1.0.0"):# last barrier data version should be 4.1.0.0fail_list.append('last barrier data version is 4.1.0.0. prohibit cluster upgrade from data version less than 4.1.0.0')else:# check target_data_version/current_data_versionsql = "select count(*) from oceanbase.__all_tenant"(desc, results) = query_cur.exec_query(sql)if len(results) != 1 or len(results[0]) != 1:fail_list.append('result cnt not match')else:tenant_count = results[0][0]sql = "select count(*) from __all_virtual_core_table where column_name in ('target_data_version', 'current_data_version') and column_value = {0}".format(data_version)(desc, results) = query_cur.exec_query(sql)if len(results) != 1 or len(results[0]) != 1:fail_list.append('result cnt not match')elif 2 * tenant_count != results[0][0]:fail_list.append('target_data_version/current_data_version not match with {0}, tenant_cnt:{1}, result_cnt:{2}'.format(data_version_str, tenant_count, results[0][0]))else:logging.info("check data version success, all tenant's compatible/target_data_version/current_data_version is {0}".format(data_version_str))# 2. 检查paxos副本是否同步, paxos副本是否缺失
def check_paxos_replica(query_cur):# 2.1 检查paxos副本是否同步(desc, results) = query_cur.exec_query("""select count(1) as unsync_cnt from GV$OB_LOG_STAT where in_sync = 'NO'""")if results[0][0] > 0 :fail_list.append('{0} replicas unsync, please check'.format(results[0][0]))# 2.2 检查paxos副本是否有缺失 TODOlogging.info('check paxos replica success')# 3. 检查是否有做balance, locality变更
def check_rebalance_task(query_cur):# 3.1 检查是否有做locality变更(desc, results) = query_cur.exec_query("""select count(1) as cnt from DBA_OB_TENANT_JOBS where job_status='INPROGRESS' and result_code is null""")if results[0][0] > 0 :fail_list.append('{0} locality tasks is doing, please check'.format(results[0][0]))# 3.2 检查是否有做balance(desc, results) = query_cur.exec_query("""select count(1) as rebalance_task_cnt from CDB_OB_LS_REPLICA_TASKS""")if results[0][0] > 0 :fail_list.append('{0} rebalance tasks is doing, please check'.format(results[0][0]))logging.info('check rebalance task success')# 4. 检查集群状态
def check_cluster_status(query_cur, min_cluster_version):# 4.1 检查是否非合并状态(desc, results) = query_cur.exec_query("""select count(1) from CDB_OB_MAJOR_COMPACTION where (GLOBAL_BROADCAST_SCN > LAST_SCN or STATUS != 'IDLE')""")if results[0][0] > 0 :fail_list.append('{0} tenant is merging, please check'.format(results[0][0]))if len(min_cluster_version) > 0 and min_cluster_version[0] < get_version("4.2.0.0"):(desc, results) = query_cur.exec_query("""select /*+ query_timeout(1000000000) */ count(1) from __all_virtual_tablet_compaction_info where max_received_scn > finished_scn and max_received_scn > 0""")if results[0][0] > 0 :fail_list.append('{0} tablet is merging, please check'.format(results[0][0]))logging.info('check cluster tablet major status success, cluster_version={0}'.format(min_cluster_version))else:logging.info('skip check cluster tablet major status, cluster_version={0}'.format(min_cluster_version))# 5. 检查是否有异常租户(creating，延迟删除，恢复中)
def check_tenant_status(query_cur):# check tenant schema(desc, results) = query_cur.exec_query("""select count(*) as count from DBA_OB_TENANTS where status != 'NORMAL'""")if len(results) != 1 or len(results[0]) != 1:fail_list.append('results len not match')elif 0 != results[0][0]:fail_list.append('has abnormal tenant, should stop')else:logging.info('check tenant status success')# check tenant info# don't support restore tenant upgrade(desc, results) = query_cur.exec_query("""select count(*) as count from oceanbase.__all_virtual_tenant_info where tenant_role != 'PRIMARY' and tenant_role != 'STANDBY'""")if len(results) != 1 or len(results[0]) != 1:fail_list.append('results len not match')elif 0 != results[0][0]:fail_list.append('has abnormal tenant info, should stop')else:logging.info('check tenant info success')# 6. 检查无恢复任务
def check_restore_job_exist(query_cur):(desc, results) = query_cur.exec_query("""select count(1) from CDB_OB_RESTORE_PROGRESS""")if len(results) != 1 or len(results[0]) != 1:fail_list.append('failed to restore job cnt')elif results[0][0] != 0:fail_list.append("""still has restore job, upgrade is not allowed temporarily""")logging.info('check restore job success')def check_is_primary_zone_distributed(primary_zone_str):semicolon_pos = len(primary_zone_str)for i in range(len(primary_zone_str)):if primary_zone_str[i] == ';':semicolon_pos = ibreakcomma_pos = len(primary_zone_str)for j in range(len(primary_zone_str)):if primary_zone_str[j] == ',':comma_pos = jbreakif comma_pos < semicolon_pos:return Trueelse:return False# 7. 升级前需要primary zone只有一个
def check_tenant_primary_zone(query_cur):sql = """select distinct value from GV$OB_PARAMETERS  where name='min_observer_version'"""(desc, results) = query_cur.exec_query(sql)if len(results) != 1:fail_list.append('min_observer_version is not sync')elif len(results[0]) != 1:fail_list.append('column cnt not match')else:min_cluster_version = get_version(results[0][0])if min_cluster_version < get_version("4.1.0.0"):(desc, results) = query_cur.exec_query("""select tenant_name,primary_zone from DBA_OB_TENANTS where  tenant_id != 1""");for item in results:if cmp(item[1], "RANDOM") == 0:fail_list.append('{0} tenant primary zone random before update not allowed'.format(item[0]))elif check_is_primary_zone_distributed(item[1]):fail_list.append('{0} tenant primary zone distributed before update not allowed'.format(item[0]))logging.info('check tenant primary zone success')# 8. 修改永久下线的时间，避免升级过程中缺副本
def modify_server_permanent_offline_time(cur):set_parameter(cur, 'server_permanent_offline_time', '72h')# 9. 检查是否有DDL任务在执行
def check_ddl_task_execute(query_cur):(desc, results) = query_cur.exec_query("""select count(1) from __all_virtual_ddl_task_status""")if 0 != results[0][0]:fail_list.append("There are DDL task in progress")logging.info('check ddl task execut status success')# 10. 检查无备份任务
def check_backup_job_exist(query_cur):# Backup jobs cannot be in-progress during upgrade.(desc, results) = query_cur.exec_query("""select count(1) from CDB_OB_BACKUP_JOBS""")if len(results) != 1 or len(results[0]) != 1:fail_list.append('failed to backup job cnt')elif results[0][0] != 0:fail_list.append("""still has backup job, upgrade is not allowed temporarily""")else:logging.info('check backup job success')# 11. 检查无归档任务
def check_archive_job_exist(query_cur):min_cluster_version = 0sql = """select distinct value from GV$OB_PARAMETERS  where name='min_observer_version'"""(desc, results) = query_cur.exec_query(sql)if len(results) != 1:fail_list.append('min_observer_version is not sync')elif len(results[0]) != 1:fail_list.append('column cnt not match')else:min_cluster_version = get_version(results[0][0])# Archive jobs cannot be in-progress before upgrade from 4.0.if min_cluster_version < get_version("4.1.0.0"):(desc, results) = query_cur.exec_query("""select count(1) from CDB_OB_ARCHIVELOG where status!='STOP'""")if len(results) != 1 or len(results[0]) != 1:fail_list.append('failed to archive job cnt')elif results[0][0] != 0:fail_list.append("""still has archive job, upgrade is not allowed temporarily""")else:logging.info('check archive job success')# 12. 检查归档路径是否清空
def check_archive_dest_exist(query_cur):min_cluster_version = 0sql = """select distinct value from GV$OB_PARAMETERS  where name='min_observer_version'"""(desc, results) = query_cur.exec_query(sql)if len(results) != 1:fail_list.append('min_observer_version is not sync')elif len(results[0]) != 1:fail_list.append('column cnt not match')else:min_cluster_version = get_version(results[0][0])# archive dest need to be cleaned before upgrade from 4.0.if min_cluster_version < get_version("4.1.0.0"):(desc, results) = query_cur.exec_query("""select count(1) from CDB_OB_ARCHIVE_DEST""")if len(results) != 1 or len(results[0]) != 1:fail_list.append('failed to archive dest cnt')elif results[0][0] != 0:fail_list.append("""still has archive destination, upgrade is not allowed temporarily""")else:logging.info('check archive destination success')# 13. 检查备份路径是否清空
def check_backup_dest_exist(query_cur):min_cluster_version = 0sql = """select distinct value from GV$OB_PARAMETERS  where name='min_observer_version'"""(desc, results) = query_cur.exec_query(sql)if len(results) != 1:fail_list.append('min_observer_version is not sync')elif len(results[0]) != 1:fail_list.append('column cnt not match')else:min_cluster_version = get_version(results[0][0])# backup dest need to be cleaned before upgrade from 4.0.if min_cluster_version < get_version("4.1.0.0"):(desc, results) = query_cur.exec_query("""select count(1) from CDB_OB_BACKUP_PARAMETER where name='data_backup_dest' and (value!=NULL or value!='')""")if len(results) != 1 or len(results[0]) != 1:fail_list.append('failed to data backup dest cnt')elif results[0][0] != 0:fail_list.append("""still has backup destination, upgrade is not allowed temporarily""")else:logging.info('check backup destination success')def check_server_version(query_cur):sql = """select distinct(substring_index(build_version, '_', 1)) from __all_server""";(desc, results) = query_cur.exec_query(sql);if len(results) != 1:fail_list.append("servers build_version not match")else:logging.info("check server version success")# 14. 检查server是否可服务
def check_observer_status(query_cur):(desc, results) = query_cur.exec_query("""select count(*) from oceanbase.__all_server where (start_service_time <= 0 or status != "active")""")if results[0][0] > 0 :fail_list.append('{0} observer not available , please check'.format(results[0][0]))logging.info('check observer status success')# 15  检查schema是否刷新成功
def check_schema_status(query_cur):(desc, results) = query_cur.exec_query("""select if (a.cnt = b.cnt, 1, 0) as passed from (select count(*) as cnt from oceanbase.__all_virtual_server_schema_info where refreshed_schema_version > 1 and refreshed_schema_version % 8 = 0) as a join (select count(*) as cnt from oceanbase.__all_server join oceanbase.__all_tenant) as b""")if results[0][0] != 1 :fail_list.append('{0} schema not available, please check'.format(results[0][0]))logging.info('check schema status success')# 16. 检查是否存在名为all/all_user/all_meta的租户
def check_not_supported_tenant_name(query_cur):names = ["all", "all_user", "all_meta"](desc, results) = query_cur.exec_query("""select tenant_name from oceanbase.DBA_OB_TENANTS""")for i in range(len(results)):if results[i][0].lower() in names:fail_list.append('a tenant named all/all_user/all_meta (case insensitive) cannot exist in the cluster, please rename the tenant')breaklogging.info('check special tenant name success')# last check of do_check, make sure no function execute after check_fail_list
def check_fail_list():if len(fail_list) != 0 :error_msg ="upgrade checker failed with " + str(len(fail_list)) + " reasons: " + ", ".join(['['+x+"] " for x in fail_list])raise MyError(error_msg)def set_query_timeout(query_cur, timeout):if timeout != 0:sql = """set @@session.ob_query_timeout = {0}""".format(timeout * 1000 * 1000)query_cur.exec_sql(sql)# 开始升级前的检查
def do_check(my_host, my_port, my_user, my_passwd, timeout, upgrade_params):try:conn = mysql.connector.connect(user = my_user,password = my_passwd,host = my_host,port = my_port,database = 'oceanbase',raise_on_warnings = True)conn.autocommit = Truecur = conn.cursor(buffered=True)min_cluster_version = [0]try:query_cur = Cursor(cur)set_query_timeout(query_cur, timeout)check_observer_version(query_cur, upgrade_params)check_data_version(query_cur, min_cluster_version)check_paxos_replica(query_cur)check_rebalance_task(query_cur)check_cluster_status(query_cur, min_cluster_version)check_tenant_status(query_cur)check_restore_job_exist(query_cur)check_tenant_primary_zone(query_cur)check_ddl_task_execute(query_cur)check_backup_job_exist(query_cur)check_archive_job_exist(query_cur)check_archive_dest_exist(query_cur)check_backup_dest_exist(query_cur)check_observer_status(query_cur)check_schema_status(query_cur)check_server_version(query_cur)check_not_supported_tenant_name(query_cur)# all check func should execute before check_fail_listcheck_fail_list()modify_server_permanent_offline_time(cur)except Exception, e:logging.exception('run error')raise efinally:cur.close()conn.close()except mysql.connector.Error, e:logging.exception('connection error')raise eexcept Exception, e:logging.exception('normal error')raise eif __name__ == '__main__':upgrade_params = UpgradeParams()change_opt_defult_value('log-file', upgrade_params.log_filename)parse_options(sys.argv[1:])if not has_no_local_opts():deal_with_local_opts()else:check_db_client_opts()log_filename = get_opt_log_file()upgrade_params.log_filename = log_filename# 日志配置放在这里是为了前面的操作不要覆盖掉日志文件config_logging_module(upgrade_params.log_filename)try:host = get_opt_host()port = int(get_opt_port())user = get_opt_user()password = get_opt_password()timeout = int(get_opt_timeout())logging.info('parameters from cmd: host=\"%s\", port=%s, user=\"%s\", password=\"%s\", timeout=\"%s\", log-file=\"%s\"',\host, port, user, password, timeout, log_filename)do_check(host, port, user, password, timeout, upgrade_params)except mysql.connector.Error, e:logging.exception('mysql connctor error')raise eexcept Exception, e:logging.exception('normal error')raise e
[root@sdw1 etc]#

ZONE升级过程

由于采用的是滚动升级的方法。升级任务为了保障升级接断业务的稳定性，先进行一个ZONE升级，当这个ZONE升级完成后，再进行另外一个ZONE升级。即使一个ZONE正在进行升级或者升级失败，其它ZONE依然可以提供服务，保持业务的可持续性

每个ZONE升级任务阶段均是以下过程:

Set zone context (开始) -> Check zone(结束)

ZONE升级任务

ZONE升级前会关闭该ZONE的运行状态，并且停掉observer的进程，之后下载 升级目标版本的依赖(Install dependencies) 以及 OB安装包(Install OB rpm)

依赖和安装包下载完成后，开启observer进程(Wait observer version refreshed) 以及 observer版本的刷新(Wait observer version refreshed)

observer启动成功之后，会检查 clog的状态(Check clog stat) 和 observer版本更新的情况(Check observer refresh schema) 和 集群的检查(Execute upgrade health checker script)，之后开启ZONE

注意:当前zone开启成功后代表该zone已经升级成功，之后会重复以上步骤升级剩余的zone，所以升级任务的多少取决于zone以及租户的数量

查看业务情况

初始化数据完成，证明滚动升级不会对业务运行产生影响

总结

1.集群升级前会进行相应的集群检查工作，确保当前OB是否满足升级条件

2.升级任务量的多少取决于zone的多少

3.升级过程中会停止zone 与 observer ，该操作不会进行回滚，所以之前的检查任务如果出错必须查看原因，不可以进行跳过

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.mzph.cn/pingmian/61108.shtml

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈email:809451989@qq.com，一经查实，立即删除！

相关文章

微知-DOCA ARGP参数模块的相关接口和用法（config单元、params单元，argp pipe line，回调）

微知-DOCA ARGP参数模块的相关接口和用法（config单元、params单元，argp pipe line，回调）

文章目录 1. 背景2. 设置参数的主要流程2.1 初始化2.2 注册某个params的处理方式以及回调函数2.4 定义好前面的params以及init指定config地点后start处理argv 3. 其他4. DOCA ARGP包相关4.1 主要接口4.2 DOCA ARGP的2个rpm包4.2.1 doca-sdk-argp-2.9.0072-1.el8.x86_64.rpm4.2.…
阅读更多...
C#.Net筑基-字符串超全总结

C#.Net筑基-字符串超全总结

字符串是日常编码中最常用的引用类型了&#xff0c;可能没有之一&#xff0c;加上字符串的不可变性、驻留性&#xff0c;很容易产生性能问题&#xff0c;因此必须全面了解一下。 01、字符与字符编码 1.1、字符Char 字符 char 表示为 Unicode字符&#xff0c;在C#中用 UTF-16 …
阅读更多...
苍穹外卖-后端部分

苍穹外卖-后端部分

软件开发整体介绍 前端搭建 在非中文目录中双击nginx.exe然后浏览器访问localhost即可 后端搭建 基础准备 导入初始文件 使用git进行版本控制 创建本地仓库和远程仓库,提交Git 连接数据库 连接数据库把资料中的文件放入运行即可 前后端联调测试 苍穹外卖项目接口文档…
阅读更多...
剧本杀门店预约小程序，解锁沉浸式推理体验

剧本杀门店预约小程序，解锁沉浸式推理体验

一、开发背景 剧本杀作为一种热门娱乐游戏&#xff0c;深受大众的欢迎&#xff0c;但随着市场的快速发展&#xff0c;竞争也在不断加大&#xff0c;对于剧本杀线下商家来说面临着发展创新。 剧本杀线下门店数量目前正在逐渐增加&#xff0c;竞争激烈&#xff0c;而门店的获客…
阅读更多...
【WPF】Prism学习（二）

【WPF】Prism学习（二）

Prism Commands 1.命令&#xff08;Commanding&#xff09; 1.1. ViewModel的作用&#xff1a; ViewModel不仅提供在视图中显示或编辑的数据&#xff0c;还可能定义一个或多个用户可以执行的动作或操作。这些用户可以通过用户界面&#xff08;UI&#xff09;执行的动作或操作…
阅读更多...
学者观察 | 元计算、人工智能和Web 3.0——山东大学教授成秀珍

学者观察 | 元计算、人工智能和Web 3.0——山东大学教授成秀珍

导语 成秀珍教授提出元计算是在开放的零信任环境下整合算力资源打通数据壁垒构建自进化智能的新质生产力技术&#xff0c;是一种新计算范式&#xff1b;区块链是Web3.0的核心技术之一&#xff0c;有助于保障开放零信任环境下&#xff0c;用户、设备和服务间去中心化数据流通的…
阅读更多...
学习笔记022——Ubuntu 安装 MySQL8.0版本踩坑记录

学习笔记022——Ubuntu 安装 MySQL8.0版本踩坑记录

目录 1、查看可安装 MySQL 版本 2、Ubuntu安装 MySQL8.0 3、MySQL8.0 区分大小写问题 4、MySQL8.0 设置sql_mode 5、MySQL8.0 改端口33060&#xff08;个人遇到问题&#xff09; 1、查看可安装 MySQL 版本 ## 列出可用的MySQL版本&#xff08;列出所有可用的MySQL版本以…
阅读更多...
「AI Infra 软件开源不是一个选项，而是必然」丨云边端架构和 AI Infra专场回顾@RTE2024

「AI Infra 软件开源不是一个选项，而是必然」丨云边端架构和 AI Infra专场回顾@RTE2024

在人工智能和开源技术蓬勃发展的当下&#xff0c;AI Infra 项目正经历着日新月异的变革。从跨平台运行时到云边端 AI 基础设施&#xff0c;再到多模态知识助手&#xff0c;创新浪潮席卷而来。这些进步不仅显著提升了技术指标&#xff0c;也为实时音视频处理、边缘计算、大模型应…
阅读更多...
《Python制作动态爱心粒子特效》

《Python制作动态爱心粒子特效》

一、实现思路 粒子效果&#xff1a; – 使用Pygame模拟粒子运动&#xff0c;粒子会以爱心的轨迹分布并运动。爱心公式&#xff1a; 爱心的数学公式&#xff1a; x16sin 3 (t),y13cos(t)−5cos(2t)−2cos(3t)−cos(4t) 参数 t t 的范围决定爱心形状。 动态效果&#xff1a; 粒子…
阅读更多...
免费实时图片编辑工具：MagicQuill

免费实时图片编辑工具：MagicQuill

参看&#xff1a; https://huggingface.co/spaces/AI4Editing/MagicQuill 人工智能交互式图像编辑&#xff1a;可以制定涂改增加删除
阅读更多...
web——upload-labs——第九关——特殊字符::$DATA绕过

web——upload-labs——第九关——特殊字符::$DATA绕过

特殊字符::$DATA绕过 典型绕过场景 在一些系统中&#xff0c;::$DATA 被用于绕过文件路径的限制。比如&#xff1a; 路径过滤绕过&#xff1a;如果系统有某种机制来检查和限制文件路径&#xff08;例如&#xff0c;禁止访问某些系统目录或敏感文件&#xff09;&#xff0c;通…
阅读更多...
本地部署 excalidraw

本地部署 excalidraw

本地部署 excalidraw 0. 引言1. 本地部署 excalidraw2. 访问 excalidraw 0. 引言 Excalidraw 编辑器是一款开源虚拟手绘白板&#xff0c;支持协作且端到端加密。 1. 本地部署 excalidraw git clone https://github.com/excalidraw/excalidraw.git; cd excalidrawvi docker-c…
阅读更多...
《Java核心技术 卷I》用户界面AWT事件继承层次

《Java核心技术 卷I》用户界面AWT事件继承层次

AWT事件继承层次 EventObject类有一个子类AWTEvent&#xff0c;它是所有AWT事件类的父类。 Swing组件会生成更多其他事件对象&#xff0c;都直接拓展自EventObject而不是AWTEvent。 AWT将事件分为底层(low-level)事件和语义事件。 语义事件&#xff1a;表示用户的动作事件&…
阅读更多...
三周精通FastAPI：42 手动运行服务器 - Uvicorn Gunicorn with Uvicorn

三周精通FastAPI：42 手动运行服务器 - Uvicorn Gunicorn with Uvicorn

官方文档&#xff1a;Server Workers - Gunicorn with Uvicorn - FastAPI 使用 fastapi 运行命令 可以直接使用fastapi run命令来启动FastAPI应用&#xff1a; fastapi run main.py如创建openapi.py文件&#xff1a; from fastapi import FastAPIapp FastAPI(openapi_url&…
阅读更多...
整理iPhone空间：iphone怎么删除相簿

整理iPhone空间：iphone怎么删除相簿

随着时间的积累&#xff0c;我们的iPhone中不仅会堆积大量照片&#xff0c;还可能会有多个不再需要的相簿。这些相簿不仅占用存储空间&#xff0c;还可能使相册应用变得杂乱无章。本文将探讨iphone怎么删除相簿&#xff0c;并介绍精简iPhone相册的技巧&#xff0c;使你的相册管…
阅读更多...
路漫漫其修远兮，吾将上下而求索---第一次使用github的过程记录和个人感受

路漫漫其修远兮，吾将上下而求索---第一次使用github的过程记录和个人感受

文章目录 1.仓库位置2.新建仓库3.配置仓库4.克隆和上传5.推荐文章和我的感受 1.仓库位置 这个仓库的位置就是在我们的这个个人主页的右上角&#xff1b;如果是第一次注册账号的话&#xff0c;这个主页里面肯定是不存在仓库的&#xff0c;需要我们自己手动的进行创建&#xff1…
阅读更多...
ICML24最新开源时序基础模型MOMENT

ICML24最新开源时序基础模型MOMENT

论文标题&#xff1a;MOMENT: A Family of Open Time-series Foundation Models 论文链接&#xff1a;https://arxiv.org/pdf/2402.03885 前言 当前时间序列数据上预训练大型模型面临以下挑战&#xff1a;(1) 缺乏大型且统一的公共时间序列数据集&#xff0c;(2) 时间序列特…
阅读更多...
SpringBoot Data Redis连接Redis-Cluster集群

SpringBoot Data Redis连接Redis-Cluster集群

使用SpringBoot Data Redis无法连接Redis-Cluster集群 最近在研究系统高并发下的缓存架构&#xff0c;因此自己在自己买的云服务器上搭建好Redis 5.0 版本的集群后&#xff0c;使用springboot的 RedisTemplate连接是发现总是访问不到集群节点。上网百度了发现没有好的解决办法&…
阅读更多...
鸿蒙中服务卡片数据的获取和渲染

鸿蒙中服务卡片数据的获取和渲染

1. 2.在卡片中使用LocalStorageProp接受传递的数据 LocalStorageProp("configNewsHead") configNewsHeadLocal: ConfigNewsHeadInfoItem[] [] 注意&#xff1a;LocalStorageProp括号中的为第一步图片2中的键 3.第一次在服务卡片的第一个卡片中可能会获取不到数据…
阅读更多...
ARM64环境部署EFK8.15.3收集K8S集群容器日志

ARM64环境部署EFK8.15.3收集K8S集群容器日志

环境规划 主机IP系统部署方式ES版本CPU架构用户名密码192.168.1.225Ubuntu 22.04.4 LTSdockerelasticsearch:8.15.3ARM64elasticllodyi4TMmZD ES集群部署 创建持久化目录(所有节点) mkdir -p /data/es/{data,certs,logs,plugins} mkdir -p /data/es/certs/{ca,es01}服务器…
阅读更多...
推荐文章
最新文章

Copyright @ 2022~2023