Mindspore框架循环神经网络RNN模型实现情感分类|（五）模型训练

Mindspore框架循环神经网络RNN模型实现情感分类

模型训练与推理

1. 模型加载

hidden_size = 256
output_size = 1
num_layers = 2
bidirectional = True
lr = 0.001
pad_idx = vocab.tokens_to_ids('<pad>')
# 模型加载
model = RNN(embeddings, hidden_size, output_size, num_layers, bidirectional, pad_idx)

其中：vocab, embeddings = load_glove(glove_path)
模型构建和实例化参数：

  embeddings:输入向量，是数据集经过glove模型统一处理的词向量数值特征,hidden_dim:隐藏层特征的维度, output_dim：输出维数, n_layers：RNN 层的数量,bidirectional:是否为双向 RNN, pad_idx:padding_idx参数用于标记输入中的填充值（padding value）。在自然语言处理任务中，文本序列的长度不一致是非常常见的。为了能够对不同长度的文本序列进行批处理，我们通常会使用填充值对较短的序列进行填补。

2.模型训练

def train():# 音频数据集imdb_path = r'./IMDB/aclImdb_v1.tar.gz'# 训练集和测试集生成imdb_train, imdb_test = load_imdb(imdb_path)  # review评论-标签，数据集# 预训练词向量表glove_path = r"./IMDB/glove.6B.zip"vocab, embeddings = load_glove(glove_path)  # 预定义词向量表# 语句标签-数据集。将文本序列统一长度，不足的使用<pad>补齐，超出的进行截断。每条评论500字。lookup_op = ds.text.Lookup(vocab, unknown_token='<unk>')pad_op = ds.transforms.PadEnd([500],pad_value=vocab.tokens_to_ids('<pad>'))  # 使用PadEnd接口，定义最大长度和补齐值(pad_value)，取最大长度为500type_cast_op = ds.transforms.TypeCast(ms.float32)  # 将label数据转为float32格式# 预处理操作流水线imdb_train = imdb_train.map(operations=[lookup_op, pad_op], input_columns=['text'])imdb_train = imdb_train.map(operations=[type_cast_op], input_columns=['label'])imdb_test = imdb_test.map(operations=[lookup_op, pad_op], input_columns=['text'])imdb_test = imdb_test.map(operations=[type_cast_op], input_columns=['label'])# 由于IMDB数据集本身不包含验证集，我们手动将其分割为训练和验证两部分，比例取0.7, 0.3。imdb_train, imdb_valid = imdb_train.split([0.7, 0.3])# 调用数据集的map、split、batch为数据集处理流水线增加对应操作，返回值为新的Dataset类型。现在仅定义流水线操作，在执行时开始执行数据处理流水线，获取最终处理好的数据并送入模型进行训练。imdb_train = imdb_train.batch(64, drop_remainder=True)imdb_valid = imdb_valid.batch(64, drop_remainder=True)# 定义训练参数hidden_size = 256output_size = 1num_layers = 2bidirectional = Truelr = 0.001pad_idx = vocab.tokens_to_ids('<pad>')model = RNN(embeddings, hidden_size, output_size, num_layers, bidirectional, pad_idx)loss_fn = nn.BCEWithLogitsLoss(reduction='mean')optimizer = nn.Adam(model.trainable_params(), learning_rate=lr)def forward_fn(data, label):logits = model(data)loss = loss_fn(logits, label)return lossgrad_fn = ms.value_and_grad(forward_fn, None, optimizer.parameters)def train_step(data, label):loss, grads = grad_fn(data, label)optimizer(grads)return lossdef train_one_epoch(model, train_dataset, epoch=0):model.set_train()total = train_dataset.get_dataset_size()loss_total = 0step_total = 0with tqdm(total=total) as t:t.set_description('Epoch %i' % epoch)for i in train_dataset.create_tuple_iterator():loss = train_step(*i)loss_total += loss.asnumpy()step_total += 1t.set_postfix(loss=loss_total / step_total)t.update(1)num_epochs = 50best_valid_loss = float('inf')ckpt_file_name = os.path.join(cache_dir, 'sentiment-analysis.ckpt')for epoch in range(num_epochs):train_one_epoch(model, imdb_train, epoch)valid_loss = evaluate(model, imdb_valid, loss_fn, epoch)if valid_loss < best_valid_loss:best_valid_loss = valid_lossms.save_checkpoint(model, ckpt_file_name)if __name__ == "__main__":train()