目录
1、创建并运行并行进程
2、使用队列（Queue）来共享数据
3、进程池
4、进程锁
5、比较使用多进程和使用单进程执行一段代码的时间消耗
6、共享变量
多进程是计算机科学中的一个术语，它是指同时运行多个进程，这些进程可以同时执行不同的任务。在计算机操作系统中，进程是分配资源的基本单位，每个进程都有自己独立的内存空间和系统资源，互不干扰。
多进程技术可以用于实现并行计算和分布式计算，其中每个进程可以独立地执行不同的任务，从而可以同时处理多个任务，提高计算机的处理效率。
PyTorch支持使用torch.multiprocessing模块来实现多进程训练。这个模块提供了类似于Python标准库中的multiprocessing模块的功能，但是在PyTorch中进行了扩展，以便更好地支持分布式训练。
使用torch.multiprocessing模块，你可以创建多个进程，每个进程都可以有自己的PyTorch张量和模型参数。这样，你可以将数据分发到不同的进程中，让它们并行地执行训练过程。
1、创建并运行并行进程
import torch.multiprocessing as mp
def action(name,times):init = 0 for i in range(times):init += iprint("this process is " + name)
if __name__ =='__main__':process1 = mp.Process(target=action,args=('process1',10000000))process2 = mp.Process(target=action,args=('process2',1000))
process1.start()process2.start()#等待进程process2执行完毕后再继续执行后面的代码#process2.join()print("main process")main process
this process is process2
this process is process1
2、使用队列（Queue）来共享数据
import torch.multiprocessing as mp
def action(q,name,times):init = 0 for i in range(times):init += iprint("this process is " + name)q.put(init)
if __name__ =='__main__':q = mp.Queue()process1 = mp.Process(target=action,args=(q,'process1',10000000))process2 = mp.Process(target=action,args=(q,'process2',1000))
process1.start()process2.start()#等待进程process1执行完毕process1.join()#等待进程process2执行完毕process2.join()#从队列中取出进程process1的计算结果result1 = q.get()#从队列中取出进程process2的计算结果result2 = q.get()
print(result1)print(result2)print("main process")this process is process2
this process is process1
499500
49999995000000
main process
3、进程池
import torch.multiprocessing as mp
def action(times):init = 0 for i in range(times):init += ireturn init
if __name__ =='__main__':times = [1000,1000000]#创建一个包含两个进程的进程池pool = mp.Pool(processes=2)res = pool.map(action,times)print(res)[499500, 499999500000]
4、进程锁
import torch.multiprocessing as mp
import time
def action(v,num,lock):lock.acquire()for i in range(5):time.sleep(0.1)v.value += numprint(v.value)lock.release()
if __name__ =='__main__':#创建一个新的锁对象lock = mp.Lock()#创建一个新的共享变量v，初始值为0，数据类型为'i'（整数）v = mp.Value('i',0)p1 = mp.Process(target=action,args=(v,1,lock))p2 = mp.Process(target=action,args=(v,2,lock))p1.start()p2.start()p1.join()p2.join()2
4
6
8
10
11
12
13
14
15
5、比较使用多进程和使用单进程执行一段代码的时间消耗
import torch.multiprocessing as mp
import time
def action(name,times):init = 0 for i in range(times):init += iprint("this process is " + name)
def mpfun():process1 = mp.Process(target=action,args=('process1',100000000))process2 = mp.Process(target=action,args=('process2',100000000))
process1.start()process2.start()
process1.join()process2.join()
def spfun():action('main process',100000000)action('main process',100000000)
if __name__ =='__main__':start_time = time.time()mpfun()end_time = time.time()print(end_time-start_time)start_time2 = time.time()spfun()end_time2 = time.time()print(end_time2-start_time2)this process is process1
this process is process2
8.2586669921875
this process is main process
this process is main process
7.6229119300842285
6、共享变量
import torch.multiprocessing as mp
import torch
def action(element,t):t[element] += (element+1) * 1000
if __name__ == "__main__":t = torch.zeros(2)t.share_memory_()print('before mp: t=')print(t)
p0 = mp.Process(target=action,args=(0,t))p1 = mp.Process(target=action,args=(1,t))p0.start()p1.start()p0.join()p1.join()print('after mp: t=')print(t)before mp: t=
tensor([0., 0.])
after mp: t=
tensor([1000., 2000.])
multigpu_lenet
multigpu_test
