前两篇我们我们把控制台、Python环境Anaconda 搞定了,接下来,我们快速进入主题,把AI 界的“Hello World” 实现一下,有个感觉,再逐步了解一些AI的概念。
1、Pytorch 安装
1) 什么是Pytorch?
一个深度学习框架,封装了很多深度学习相关函数,目前是pytorch已成为最受欢迎的深度学习框架之一,除了它,目前还有一些在用的TensorFlow、Keras、MXNet、Caffe 等。
2) 为什么用Pytorch?
我们公司在用它,行业主流也在用它,我没必要一开始学冷门,出了问题,找不到“知音”
3)如何用上Pytorch?
官网介绍在这里:Start Locally | PyTorch
macOS 通过Anaconda 安装pytorch
conda install pytorch torchvision -c pytorch
正常情况下,你会遇到我一样的错误
❯ conda install pytorch torchvision -c pytorchChannels:- pytorch- https://mirrors.ustc.edu.cn/anaconda/pkgs/free- https://mirrors.ustc.edu.cn/anaconda/pkgs/main- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main- defaults
Platform: osx-arm64
Collecting package metadata (repodata.json): failedCondaHTTPError: HTTP 429 TOO MANY REQUESTS for url <https://mirrors.ustc.edu.cn/anaconda/pkgs/free/osx-arm64/repodata.json>
Elapsed: 00:26.256248An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
'https//mirrors.ustc.edu.cn/anaconda/pkgs/free/osx-arm64'
解决办法我放到下方的附录部分了:
重新执行安装
❯ conda install pytorch torchvision -c pytorchChannels:- pytorch- defaults
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: done## Package Plan ##environment location: /opt/anaconda3added / updated specs:- pytorch- torchvisionThe following packages will be downloaded:package | build---------------------------|-----------------libjpeg-turbo-2.0.0 | h1a28f6b_0 386 KB defaultspytorch-2.4.0 | py3.12_0 57.6 MB pytorchtorchvision-0.19.0 | py312_cpu 6.8 MB pytorch------------------------------------------------------------Total: 64.8 MBThe following NEW packages will be INSTALLED:libjpeg-turbo anaconda/pkgs/main/osx-arm64::libjpeg-turbo-2.0.0-h1a28f6b_0pytorch pytorch/osx-arm64::pytorch-2.4.0-py3.12_0torchvision pytorch/osx-arm64::torchvision-0.19.0-py312_cpuProceed ([y]/n)? yDownloading and Extracting Packages:Preparing transaction: done
Verifying transaction: done
Executing transaction: done
验证一下,没有报错,则安装成功了
❯ python
Python 3.12.4 | packaged by Anaconda, Inc. | (main, Jun 18 2024, 10:07:17) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>>
2、开始写程序
先有个体感,从MNIST 手写的一批数字中,训练出一个模型,然后再拿一部分MNIST数据区验证模型准确率。下面的程序打印了一些识别错误的反例。
先跑跑,下篇我们逐行分析一下。
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader
from torchvision import datasets, transforms
from torch.utils.tensorboard import SummaryWriter
import matplotlib.pyplot as plt # 数据加载与预处理
transform = transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))
]) train_dataset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)
test_dataset = datasets.MNIST(root='./data', train=False, download=True, transform=transform)
print("load success")
train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True)
test_loader = DataLoader(test_dataset, batch_size=64, shuffle=False) # 模型定义
class SimpleCNN(nn.Module): def __init__(self): super(SimpleCNN, self).__init__() self.conv1 = nn.Conv2d(1, 32, kernel_size=3, stride=1, padding=1) self.conv2 = nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1) self.pool = nn.MaxPool2d(2, 2) self.fc1 = nn.Linear(64 * 7 * 7, 128) self.fc2 = nn.Linear(128, 10) def forward(self, x): x = self.pool(nn.ReLU()(self.conv1(x))) x = self.pool(nn.ReLU()(self.conv2(x))) x = x.view(-1, 64 * 7 * 7) x = nn.ReLU()(self.fc1(x)) x = self.fc2(x) return x # 训练设置
model = SimpleCNN()
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)
num_epochs = 10 # TensorBoard 初始化
writer = SummaryWriter() # 记录损失和准确率
loss_values = []
accuracy_values = [] # 训练循环
for epoch in range(num_epochs): running_loss = 0.0 correct = 0 total = 0 for images, labels in train_loader: optimizer.zero_grad() outputs = model(images) loss = criterion(outputs, labels) loss.backward() optimizer.step() running_loss += loss.item() _, predicted = torch.max(outputs.data, 1) total += labels.size(0) correct += (predicted == labels).sum().item() avg_loss = running_loss / len(train_loader) accuracy = 100 * correct / total # 记录到 TensorBoard writer.add_scalar('Loss/train', avg_loss, epoch) writer.add_scalar('Accuracy/train', accuracy, epoch) # 记录到列表 loss_values.append(avg_loss) accuracy_values.append(accuracy) print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {avg_loss:.4f}, Accuracy: {accuracy:.2f}%') writer.close() # 测试模型
model.eval()
test_loss = 0.0
correct = 0
total = 0
i=0with torch.no_grad(): for images, labels in test_loader: outputs = model(images) loss = criterion(outputs, labels) test_loss += loss.item() _, predicted = torch.max(outputs.data, 1) total += labels.size(0) correct += (predicted == labels).sum().item() if (predicted[i] != labels[i] ):print("我猜是",predicted[i],"实际是",labels[i])plt.imshow(images[i].reshape(28,28),cmap='Greys', interpolation='nearest')plt.show()test_loss /= len(test_loader)
test_accuracy = 100 * correct / total
print(f'Test Loss: {test_loss:.4f}, Test Accuracy: {test_accuracy:.2f}%') # 使用 Matplotlib 绘制损失和准确率
plt.figure(figsize=(12, 4)) # 绘制损失图
plt.subplot(1, 2, 1)
plt.plot(range(1, num_epochs + 1), loss_values, label='Loss')
plt.title('Training Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend() # 绘制准确率图
plt.subplot(1, 2, 2)
plt.plot(range(1, num_epochs + 1), accuracy_values, label='Accuracy', color='orange')
plt.title('Training Accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy (%)')
plt.legend() plt.tight_layout()
plt.show()
load success
Epoch [1/10], Loss: 0.1447, Accuracy: 95.53%
Epoch [2/10], Loss: 0.0447, Accuracy: 98.61%
Epoch [3/10], Loss: 0.0303, Accuracy: 99.06%
Epoch [4/10], Loss: 0.0218, Accuracy: 99.28%
Epoch [5/10], Loss: 0.0164, Accuracy: 99.47%
Epoch [6/10], Loss: 0.0136, Accuracy: 99.52%
Epoch [7/10], Loss: 0.0105, Accuracy: 99.65%
Epoch [8/10], Loss: 0.0079, Accuracy: 99.75%
Epoch [9/10], Loss: 0.0076, Accuracy: 99.75%
Epoch [10/10], Loss: 0.0078, Accuracy: 99.72%
附录
1、Http 429 限制问题解决,找到 用户目录的 .condarc 文件
vim ~/.condarc
vim 编辑后,替换文件内容一下为下方的内容
channels:- defaults
show_channel_urls: true
default_channels:- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
custom_channels:conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudmsys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudbioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudmenpo: https://mirrors.tuna.tsinghua.edu
再执行安装,即可