测试环境
Python 3.9.17
系统：macOS Ventura 13.0.1
测试1 多进程采用test.run方式调用函数。
发现，调用的test.run 函数没运行
def multiprocessing_get_slow_log():pool = Pool(3)sp = Spider()conn = sp.mysql_conncur = conn.cursor()sql = """select inst_id,inst_name from rds_info where status = 1 and inst_name like '%_publish%' limit 5"""cur.execute(sql)info_list = cur.fetchall()conn.commit()for inst_id in info_list:logger.info('{} {} '.format(inst_id[0], inst_id[1]))print('{} {} '.format(inst_id[0], inst_id[1]))test = GetSlowlogRecords(inst_id[0], inst_id[1])pool.apply_async(test.run, args=())# pool.apply_async(test_fun, args=(inst_id[0], inst_id[1],))pool.close()pool.join()# time.sleep(100)print("结束")测试2 采用test.run()方式调用函数。
python 多进程调用 类的自定义函数run() 加括号，会使多进程功能失效，变成穿行执行的程序。
test = GetSlowlogRecords(inst_id[0], inst_id[1])
pool.apply_async(test.run(), args=())
def multiprocessing_get_slow_log():pool = Pool(3)sp = Spider()conn = sp.mysql_conncur = conn.cursor()sql = """select inst_id,inst_name from rds_info where status = 1 and inst_name like '%_publish%' limit 5"""cur.execute(sql)info_list = cur.fetchall()conn.commit()for inst_id in info_list:logger.info('{} {} '.format(inst_id[0], inst_id[1]))print('{} {} '.format(inst_id[0], inst_id[1]))test = GetSlowlogRecords(inst_id[0], inst_id[1])pool.apply_async(test.run(), args=())pool.close()pool.join()# time.sleep(100)print("结束")测试3 把创建对象单独放到一个函数中。
可以实现python 调用class类的多进程功能。
def test_fun(inst_id, inst_name):test = GetSlowlogRecords(inst_id, inst_name)test.run()def multiprocessing_get_slow_log():pool = Pool(3)sp = Spider()conn = sp.mysql_conncur = conn.cursor()sql = """select inst_id,inst_name from rds_info where status = 1 and inst_name like '%_publish%' limit 5"""cur.execute(sql)info_list = cur.fetchall()conn.commit()for inst_id in info_list:logger.info('{} {} '.format(inst_id[0], inst_id[1]))print('{} {} '.format(inst_id[0], inst_id[1]))pool.apply_async(test_fun, args=(inst_id[0], inst_id[1],))pool.close()pool.join()# time.sleep(100)print("结束")
结论
测试3 用的方法满足需求
)
)
