前言
轻量化网络设计是一种针对移动设备等资源受限环境的深度学习模型设计方法。下面是一些常见的轻量化网络设计方法：
- 网络剪枝：移除神经网络中冗余的连接和参数，以达到模型压缩和加速的目的。
- 分组卷积：将卷积操作分解为若干个较小的卷积操作，并将它们分别作用于输入的不同通道，从而减少计算量。
- 深度可分离卷积：将标准卷积分解成深度卷积和逐点卷积两个步骤，使得在大部分情况下可以大幅减少计算量。
- 跨层连接：通过跨越多个层级的连接方式来增加神经网络的深度和复杂性，同时减少了需要训练的参数数量。
- 模块化设计：将神经网络分解为多个可重复使用的模块，以提高模型的可调节性和适应性。
传统的YOLOv8系列中，Backbone采用的是较为复杂的C2f网络结构，这使得模型计算量大幅度的增加，检测速度较慢，应用受限，在某些真实的应用场景如移动或者嵌入式设备，如此大而复杂的模型时难以被应用的。为了解决这个问题，本章节通过采用Ghostnet轻量化主干网络作为Backbone的基础结构，从而在保证检测性能的同时，将网络结构精简到最小，大大减小了模型的参数量和计算量。
目录
- 一、Ghostnet
- 二、代码实现
- 2.1 添加C2fGhost模块
- 2.2 注册C2fGhost模块
- 2.3 配置yaml文件
- yolov8-ghostnet.yaml
- 2.3 模型验证
- 2.4 模型训练
- 2.5 模型对比
- 三、总结
一、Ghostnet
2020 CVPR 论文链接：GhostNet: More Features from Cheap Operations
Pytorch code：ghostnet_pytorch
轻量级神经网络Ghostnet是专门为移动设备上的应用而设计的，由Ghost bottleneck搭建而成，而Ghost bottleneck通过Ghost模块堆叠。Ghost 模块是一种新颖的即插即用模块。Ghost 模块设计的初衷是使用更少的参数来生成更多特征图 (generate more features by using fewer parameters)。在ImageNet分类任务，GhostNet在相似计算量情况下Top-1正确率达75.7%，高于MobileNetV3的75.2%。
-
GhostConv
class GhostConv(nn.Module):"""Ghost Convolution https://github.com/huawei-noah/ghostnet."""def __init__(self, c1, c2, k=1, s=1, g=1, act=True): # ch_in, ch_out, kernel, stride, groupssuper().__init__()c_ = c2 // 2 # hidden channelsself.cv1 = Conv(c1, c_, k, s, None, g, act=act)self.cv2 = Conv(c_, c_, 5, 1, None, c_, act=act)def forward(self, x):"""Forward propagation through a Ghost Bottleneck layer with skip connection."""y = self.cv1(x)return torch.cat((y, self.cv2(y)), 1) -
GhostBottleneck
class GhostBottleneck(nn.Module):"""Ghost Bottleneck https://github.com/huawei-noah/ghostnet."""def __init__(self, c1, c2, k=3, s=1): # ch_in, ch_out, kernel, stridesuper().__init__()c_ = c2 // 2self.conv = nn.Sequential(GhostConv(c1, c_, 1, 1), # pwDWConv(c_, c_, k, s, act=False) if s == 2 else nn.Identity(), # dwGhostConv(c_, c2, 1, 1, act=False)) # pw-linearself.shortcut = nn.Sequential(DWConv(c1, c1, k, s, act=False), Conv(c1, c2, 1, 1,act=False)) if s == 2 else nn.Identity()def forward(self, x):"""Applies skip connection and concatenation to input tensor."""return self.conv(x) + self.shortcut(x) -
C2fGhost（需要自己添加）
class C2fGhost(C2f):"""C2f module with GhostBottleneck()."""def __init__(self, c1, c2, n=1, shortcut=True, g=1, e=0.5):"""Initialize 'SPP' module with various pooling sizes for spatial pyramid pooling."""super().__init__(c1, c2, n, shortcut, g, e)c_ = int(c2 * e) # hidden channelsself.m = nn.Sequential(*(GhostBottleneck(c_, c_) for _ in range(n)))
二、代码实现
2.1 添加C2fGhost模块
在ultralytics/nn/modules/block.py文件中加入以下代码：
class C2fGhost(C2f):"""C2f module with GhostBottleneck()."""def __init__(self, c1, c2, n=1, shortcut=True, g=1, e=0.5):"""Initialize 'SPP' module with various pooling sizes for spatial pyramid pooling."""super().__init__(c1, c2, n, shortcut, g, e)c_ = int(c2 * e) # hidden channelsself.m = nn.Sequential(*(GhostBottleneck(c_, c_) for _ in range(n)))
2.2 注册C2fGhost模块
修改ultralytics/nn/modules/__init__.py文件：
from .block import (ASFF2, ASFF3, C1, C2, C3, C3TR, DFL, SPP, SPPF, Bottleneck, BottleneckCSP, C2f, C3Ghost, C3x,GhostBottleneck, HGBlock, HGStem, Proto, RepC3, C2fGhost)
__all__ = ('Conv', 'Conv2', 'LightConv', 'RepConv', 'DWConv', 'DWConvTranspose2d', 'ConvTranspose', 'Focus','GhostConv', 'ChannelAttention', 'SpatialAttention', 'CBAM', 'Concat', 'TransformerLayer','TransformerBlock', 'MLPBlock', 'LayerNorm2d', 'DFL', 'HGBlock', 'HGStem', 'SPP', 'SPPF', 'C1', 'C2', 'C3','C2f', 'C3x', 'C3TR', 'C3Ghost', 'GhostBottleneck', 'Bottleneck', 'BottleneckCSP', 'Proto', 'Detect','Segment', 'Pose', 'Classify', 'TransformerEncoderLayer', 'RepC3', 'RTDETRDecoder', 'AIFI','DeformableTransformerDecoder', 'DeformableTransformerDecoderLayer', 'MSDeformAttn', 'MLP', 'ASFF2', 'ASFF3','C2fGhost')
修改ultralytics/nn/tasks.py文件中的parse_model函数：
from ultralytics.nn.modules import (AIFI, ASFF2, ASFF3, C1, C2, C3, C3TR, SPP, SPPF, Bottleneck, BottleneckCSP, C2f,C3Ghost, C3x, Classify, Concat, Conv, Conv2, ConvTranspose, Detect, DWConv,DWConvTranspose2d, Focus, GhostBottleneck, GhostConv, HGBlock, HGStem, Pose, RepC3,RepConv, RTDETRDecoder, Segment, C2fGhost)
if m in (Classify, Conv, ConvTranspose, GhostConv, Bottleneck, GhostBottleneck, SPP, SPPF, DWConv, Focus,BottleneckCSP, C1, C2, C2f, C3, C3TR, C3Ghost, nn.ConvTranspose2d, DWConvTranspose2d, C3x, RepC3,C2fGhost):c1, c2 = ch[f], args[0]if c2 != nc: # if c2 not equal to number of classes (i.e. for Classify() output)c2 = make_divisible(min(c2, max_channels) * width, 8)args = [c1, c2, *args[1:]]if m in (BottleneckCSP, C1, C2, C2f, C3, C3TR, C3Ghost, C3x, RepC3, C2fGhost):args.insert(2, n) # number of repeatsn = 1
2.3 配置yaml文件
这里我们选择替换Backbone中的所有Conv和C2f模块。当然也可以将所有Conv和C2f模块全部替换掉，哪个效果更好，需要各位去实测一番。
yolov8-ghostnet.yaml
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLOv8 object detection model with P3-P5 outputs. For Usage examples see https://docs.ultralytics.com/tasks/detect# Parameters
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n.yaml' will call yolov8.yaml with scale 'n'# [depth, width, max_channels]n: [0.33, 0.25, 1024] # YOLOv8n summary: 225 layers, 3157200 parameters, 3157184 gradients, 8.9 GFLOPss: [0.33, 0.50, 1024] # YOLOv8s summary: 225 layers, 11166560 parameters, 11166544 gradients, 28.8 GFLOPsm: [0.67, 0.75, 768] # YOLOv8m summary: 295 layers, 25902640 parameters, 25902624 gradients, 79.3 GFLOPsl: [1.00, 1.00, 512] # YOLOv8l summary: 365 layers, 43691520 parameters, 43691504 gradients, 165.7 GFLOPsx: [1.00, 1.25, 512] # YOLOv8x summary: 365 layers, 68229648 parameters, 68229632 gradients, 258.5 GFLOPs# YOLOv8.0n backbone
backbone:# [from, repeats, module, args]- [-1, 1, GhostConv, [64, 3, 2]] # 0-P1/2- [-1, 1, GhostConv, [128, 3, 2]] # 1-P2/4- [-1, 3, C2fGhost, [128, True]]- [-1, 1, GhostConv, [256, 3, 2]] # 3-P3/8- [-1, 6, C2fGhost, [256, True]]- [-1, 1, GhostConv, [512, 3, 2]] # 5-P4/16- [-1, 6, C2fGhost, [512, True]]- [-1, 1, GhostConv, [1024, 3, 2]] # 7-P5/32- [-1, 3, C2fGhost, [1024, True]]- [-1, 1, SPPF, [1024, 5]] # 9# YOLOv8.0n head
head:- [-1, 1, nn.Upsample, [None, 2, 'nearest']]- [[-1, 6], 1, Concat, [1]] # cat backbone P4- [-1, 3, C2f, [512]] # 12- [-1, 1, nn.Upsample, [None, 2, 'nearest']]- [[-1, 4], 1, Concat, [1]] # cat backbone P3- [-1, 3, C2f, [256]] # 15 (P3/8-small)- [-1, 1, Conv, [256, 3, 2]]- [[-1, 12], 1, Concat, [1]] # cat head P4- [-1, 3, C2f, [512]] # 18 (P4/16-medium)- [-1, 1, Conv, [512, 3, 2]]- [[-1, 9], 1, Concat, [1]] # cat head P5- [-1, 3, C2f, [1024]] # 21 (P5/32-large)- [[15, 18, 21], 1, Detect, [nc]] # Detect(P3, P4, P5)
2.3 模型验证
from ultralytics import YOLO# Load a model
model = YOLO("backbone/yolov8n-ghostnet.yaml") # build a new model from scratch
YOLOv8-ghostnet summary: 323 layers, 2502908 parameters, 2502892 gradients, 7.1 GFLOPs
2.4 模型训练
from ultralytics import YOLO# Load a model
model = YOLO("backbone/yolov8n-ghostnet.yaml") # build a new model from scratch# Use the model
model.train(data="./mydata/data.yaml",epochs=300,batch=48) # train the model
2.5 模型对比
模型参数量和计算量对比（以自己实测为主，仅供参考）
| 模型 | 参数量（parameters） | 计算量（GFLOPs） |
|---|---|---|
| YOLOv8n | 3157200 | 8.9 |
| YOLOv8n-Ghostnet | 2502908（↓20.72%） | 7.1（↓20.22%） |
三、总结
- 模型的训练具有很大的随机性，您可能需要点运气和更多的训练次数才能达到最高的 mAP。
