<jupyter_start><jupyter_text>加载模型用于推理或迁移学习 概述在模型训练过程中保存在本地的CheckPoint文件，或从MindSpore Hub下载的CheckPoint文件，都可以帮助用户进行推理或迁移学习使用，提高效率。以下通过示例来介绍如何通过本地加载加载模型，用于推理验证和迁移学习。> 本文档适用于CPU、GPU和Ascend环境。  整体流程1. 准备环节。下载数据集，配置运行信息。2. 数据处理。创建可用于网络训练的数据集，可视化数据集图像。3. 预训练模型。生成CheckPoint文件。4. 本地加载模型用于推理验证。5. 本地加载模型用于迁移学习。   准备环节 下载数据集运行以下一段代码,将数据集下载至当前工作目录中`./datasets/MNIST_Data`目录下。<jupyter_code>!wget -N https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/datasets/MNIST_Data.zip
!unzip -o MNIST_Data.zip -d ./datasets
!tree ./datasets/MNIST_Data/<jupyter_output>--2020-11-30 16:32:07--  https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/datasets/MNIST_Data.zip
Resolving proxy-notebook.modelarts-dev-proxy.com (proxy-notebook.modelarts-dev-proxy.com)... 192.168.0.172
Connecting to proxy-notebook.modelarts-dev-proxy.com (proxy-notebook.modelarts-dev-proxy.com)|192.168.0.172|:8083... connected.
Proxy request sent, awaiting response... 200 OK
Length: 10754903 (10M) [application/zip]
Saving to: ‘MNIST_Data.zip’

MNIST_Data.zip      100%[===================>]  10.26M  --.-KB/s    in 0.06s   

2020-11-30 16:32:07 (160 MB/s) - ‘MNIST_Data.zip’ saved [10754903/10754903]

Archive:  MNIST_Data.zip
   creating: MNIST_Data/test/
  inflating: MNIST_Data/test/t10k-images-idx3-ubyte  
  inflating: MNIST_Data/test/t10k-labels-idx1-ubyte  
   creating: MNIST_Data/train/
  inflating: MNIST_Data/train/train-images-idx3-ubyte  
  inflating: MNIST_Data/train/train-labels-idx1-ubyte  
./datasets/MNIST_Data/
├── test
│   ├── t10k-images-idx3[...]<jupyter_text>配置运行环境运行以下一段代码，配置训练网络参数，配置运行平台为CPU，并指定模型文件目录和数据集目录。相关参数含义为：- `device_target`：硬件平台。- `data_pat`：数据集目录。- `ckpt_path`：训练后的模型文件存放目录。- `epoch_size`：迭代训练次数。<jupyter_code>"""
network config setting, will be used in train.py
"""

from easydict import EasyDict as edict
from mindspore import context


args = edict({
    'device_target': 'CPU',
    'data_path': './datasets/MNIST_Data',
    'ckpt_path': './models/ckpt/mindspore_load_model_for_inference_and_transfer/',
    'num_classes': 10,
    'lr': 0.01,
    'momentum': 0.9,
    'epoch_size': 1,
    'batch_size': 32,
    'buffer_size': 1000,
    'image_height': 32,                                                                                                                                                                                             
    'image_width': 32,
    'save_checkpoint_steps': 1875,
    'keep_checkpoint_max': 10,
    'air_name': "lenet.air",
})


context.set_context(mode=context.GRAPH_MODE, device_target=args.device_target)<jupyter_output><empty_output><jupyter_text>以下一段代码中定义预训练模型使用的损失函数`CrossEntropyLoss`。<jupyter_code>import mindspore.nn as nn
from mindspore import Tensor
import mindspore.ops as ops
from mindspore import dtype as mstype


class CrossEntropyLoss(nn.Cell):
    def __init__(self):
        super(CrossEntropyLoss, self).__init__()
        self.cross_entropy = ops.SoftmaxCrossEntropyWithLogits()
        self.mean = ops.ReduceMean()
        self.one_hot = ops.OneHot()
        self.one = Tensor(1.0, mstype.float32)
        self.zero = Tensor(0.0, mstype.float32)

    def construct(self, logits, label):
        label = self.one_hot(label, ops.shape(logits)[1], self.one, self.zero)
        loss_func = self.cross_entropy(logits, label)[0]
        loss_func = self.mean(loss_func, (-1,))
        return loss_func<jupyter_output><empty_output><jupyter_text>数据预处理使用`create_dataset`函数来创建数据集，对数据集进行预处理操作。<jupyter_code>import os
import mindspore.dataset as ds
import mindspore.dataset.vision.c_transforms as CV
import mindspore.dataset.transforms.c_transforms as C
from mindspore.dataset.vision import Inter
from mindspore import dtype as mstype

def create_dataset(data_path, batch_size=32, repeat_size=1,
                   num_parallel_workers=1):
    """
    create dataset for train or test
    """
    # define dataset
    mnist_ds = ds.MnistDataset(data_path)

    resize_height, resize_width = 32, 32
    rescale = 1.0 / 255.0
    shift = 0.0
    rescale_nml = 1 / 0.3081
    shift_nml = -1 * 0.1307 / 0.3081

    # define map operations
    resize_op = CV.Resize((resize_height, resize_width), interpolation=Inter.LINEAR)  # Bilinear mode
    rescale_nml_op = CV.Rescale(rescale_nml, shift_nml)
    rescale_op = CV.Rescale(rescale, shift)
    hwc2chw_op = CV.HWC2CHW()
    type_cast_op = C.TypeCast(mstype.int32)

    # apply map operations on images
    mnist_ds = mnist_ds.map(operations=type_cast_op, input_columns="label", num_parallel_workers=num_parallel_workers)
    mnist_ds = mnist_ds.map(operations=resize_op, input_columns="image", num_parallel_workers=num_parallel_workers)
    mnist_ds = mnist_ds.map(operations=rescale_op, input_columns="image", num_parallel_workers=num_parallel_workers)
    mnist_ds = mnist_ds.map(operations=rescale_nml_op, input_columns="image", num_parallel_workers=num_parallel_workers)
    mnist_ds = mnist_ds.map(operations=hwc2chw_op, input_columns="image", num_parallel_workers=num_parallel_workers)

    # apply DatasetOps
    buffer_size = 10000
    mnist_ds = mnist_ds.shuffle(buffer_size=buffer_size)  # 10000 as in LeNet train script
    mnist_ds = mnist_ds.batch(batch_size, drop_remainder=True)
    mnist_ds = mnist_ds.repeat(repeat_size)

    return mnist_ds


ds_train = create_dataset(os.path.join(args.data_path, "train"), args.batch_size)<jupyter_output><empty_output><jupyter_text>可视化数据集使用`matplotlib`可视化工具查看第一个batch中的32张图像。<jupyter_code>from matplotlib import pyplot as plt
import numpy as np


print("The 32 images with label of the first batch in ds_train are showed below:")
ds_iterator = ds_train.create_dict_iterator()
next(ds_iterator)
batch_1 = next(ds_iterator)
batch_image = batch_1["image"].asnumpy()
batch_label = batch_1["label"].asnumpy()
%matplotlib inline
plt.figure(dpi=144)
for i,image in enumerate(batch_image):
    plt.subplot(4, 8, i+1)
    plt.subplots_adjust(wspace=0.2, hspace=0.2)
    plt.imshow(np.squeeze(image), cmap='gray')
    plt.title(f"image {i+1}\nlabel: {batch_label[i]}", y=-0.65, fontdict={"fontsize":8})
    plt.axis('off')    
plt.show()<jupyter_output>The 32 images with label of the first batch in ds_train are showed below:<jupyter_text>定义网络使用下面一段代码定义LeNet网络<jupyter_code>import mindspore.nn as nn
from mindspore.common.initializer import Normal

class LeNet5(nn.Cell):
    """Lenet network structure."""
    # define the operator required
    def __init__(self, num_class=10, num_channel=1):
        super(LeNet5, self).__init__()
        self.conv1 = nn.Conv2d(num_channel, 6, 5, pad_mode='valid')
        self.conv2 = nn.Conv2d(6, 16, 5, pad_mode='valid')
        self.fc1 = nn.Dense(16 * 5 * 5, 120, weight_init=Normal(0.02))
        self.fc2 = nn.Dense(120, 84, weight_init=Normal(0.02))
        self.fc3 = nn.Dense(84, num_class, weight_init=Normal(0.02))
        self.relu = nn.ReLU()
        self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2)
        self.flatten = nn.Flatten()

    # use the preceding operators to construct networks
    def construct(self, x):
        x = self.max_pool2d(self.relu(self.conv1(x)))
        x = self.max_pool2d(self.relu(self.conv2(x)))
        x = self.flatten(x)
        x = self.relu(self.fc1(x))
        x = self.relu(self.fc2(x))
        x = self.fc3(x) 
        return x<jupyter_output><empty_output><jupyter_text>预训练模型运行以下一段代码进行预训练，生成后续加载模型进行推理或迁移学习所需的预训练模型CheckPoint文件。<jupyter_code>import mindspore.nn as nn
from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor
from mindspore import Model
from mindspore.nn.metrics import Accuracy
import os 

# clean up old run files before in Linux
os.system('rm -f {}*.ckpt {}*.meta {}*.pb'.format(args.ckpt_path,args.ckpt_path,args.ckpt_path))

network = LeNet5(args.num_classes)
net_loss = CrossEntropyLoss()
net_opt = nn.Momentum(network.trainable_params(), args.lr, args.momentum)
time_cb = TimeMonitor(data_size=ds_train.get_dataset_size())
config_ck = CheckpointConfig(save_checkpoint_steps=args.save_checkpoint_steps,
                             keep_checkpoint_max=args.keep_checkpoint_max)
ckpoint_cb = ModelCheckpoint(prefix="checkpoint_lenet", directory=args.ckpt_path, config=config_ck)
model = Model(network, net_loss, net_opt, metrics={"Accuracy": Accuracy()})

print("============== Starting Training ==============")
model.train(args['epoch_size'], ds_train, callbacks=[time_cb, ckpoint_cb, LossMonitor(per_print_times=375)], dataset_sink_mode=False)<jupyter_output>============== Starting Training ==============
epoch: 1 step: 375, loss is 2.2881835
epoch: 1 step: 750, loss is 2.311615
epoch: 1 step: 1125, loss is 0.03919042
epoch: 1 step: 1500, loss is 0.10693374
epoch: 1 step: 1875, loss is 0.19927995
Epoch time: 18645.494, per step time: 9.944<jupyter_text>预训练完成后，在`ckpt`目录下生成CheckPoint模型文件（`checkpoint_lenet-1_1875.ckpt`文件）。此时`ckpt`目录结构如下：<jupyter_code>!tree ./models/ckpt/mindspore_load_model_for_inference_and_transfer<jupyter_output>./models/ckpt/mindspore_load_model_for_inference_and_transfer
├── checkpoint_lenet-1_1875.ckpt
└── checkpoint_lenet-graph.meta

0 directories, 2 files<jupyter_text>加载模型用于推理验证针对仅推理场景可以使用`load_checkpoint`方法把参数直接加载到网络中，以便进行后续的推理验证。使用`load_checkpoint`方法，导入预训练模型文件`checkpoint_lenet-1_1875.ckpt`中的参数到网络中。其中：- `load_checkpoint`方法会把参数文件中的网络参数加载到模型中。加载后，网络中的参数就是CheckPoint保存的参数。- `eval`方法会验证训练后模型的精度。运行以下一段代码，进行本地模型加载进行推理验证，得到推理精度值数据。<jupyter_code>import os
import mindspore.nn as nn
from mindspore import load_checkpoint, load_param_into_net
from mindspore import Model
from mindspore.nn.metrics import Accuracy


network = LeNet5(args.num_classes)
net_loss = CrossEntropyLoss()
load_checkpoint(ckpt_file_name="./models/ckpt/mindspore_load_model_for_inference_and_transfer/checkpoint_lenet-1_1875.ckpt", net=network)
model = Model(network, net_loss, metrics={"Accuracy": Accuracy()})

print("============== Starting Testing ==============")
ds_eval = create_dataset(os.path.join(args.data_path, "test"),
                         args.batch_size,
                         1)

acc = model.eval(ds_eval, dataset_sink_mode=False)
print("============== {} ==============".format(acc))<jupyter_output>============== Starting Testing ==============
============== {'Accuracy': 0.9668469551282052} ==============<jupyter_text>通过运行的输出结果得到推理精度达到0.96以上。 加载模型用于迁移学习针对任务中断再训练及微调（Fine Tune）场景，可以加载网络参数和优化器参数到模型中。同理，使用`load_checkpoint`接口导入预训练模型文件`checkpoint_lenet-3_1875.ckpt`，并使用`load_param_into_net`接口将预训练模型参数加载进网络中。其中：- `load_checkpoint`方法会返回一个预训练模型中的参数字典。- `load_param_into_net`会把参数字典中相应的参数加载到网络或优化器中。运行以下一段代码，进行本地模型加载并进行迁移学习（重训练）。在重训练过程中使用损失函数`SoftmaxCrossEntropyWithLogits`。<jupyter_code>network = LeNet5(args.num_classes)
net_opt = nn.Momentum(network.trainable_params(), args.lr, args.momentum)

# return a parameter dict for model
param_dict = load_checkpoint(ckpt_file_name="./models/ckpt/mindspore_load_model_for_inference_and_transfer/checkpoint_lenet-1_1875.ckpt")
# load the parameter into net
load_param_into_net(network, param_dict)
# load the parameter into operator
load_param_into_net(net_opt, param_dict)
net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean")
time_cb = TimeMonitor(data_size=ds_train.get_dataset_size())
config_ck = CheckpointConfig(save_checkpoint_steps=args.save_checkpoint_steps,
                             keep_checkpoint_max=args.keep_checkpoint_max)
ckpoint_cb = ModelCheckpoint(prefix="checkpoint_lenet", directory=args.ckpt_path, config=config_ck)
model = Model(network, net_loss, net_opt, metrics={"Accuracy": Accuracy()})
ds_train = create_dataset(os.path.join(args.data_path, "train"), args.batch_size)

print("============== Starting Training ==============")
model.train(3, ds_train, callbacks=[time_cb, ckpoint_cb, LossMonitor(per_print_times=375)], dataset_sink_mode=False)<jupyter_output>============== Starting Training ==============
epoch: 1 step: 375, loss is 0.011971839
epoch: 1 step: 750, loss is 0.010163541
epoch: 1 step: 1125, loss is 0.12516475
epoch: 1 step: 1500, loss is 0.22787872
epoch: 1 step: 1875, loss is 0.15650317
Epoch time: 18165.255, per step time: 9.688
epoch: 2 step: 375, loss is 0.04129391
epoch: 2 step: 750, loss is 0.19876541
epoch: 2 step: 1125, loss is 0.04154645
epoch: 2 step: 1500, loss is 0.011596533
epoch: 2 step: 1875, loss is 0.21028273
Epoch time: 17832.701, per step time: 9.511
epoch: 3 step: 375, loss is 0.12846899
epoch: 3 step: 750, loss is 0.0042161625
epoch: 3 step: 1125, loss is 0.024917062
epoch: 3 step: 1500, loss is 0.024145193
epoch: 3 step: 1875, loss is 0.031243846
Epoch time: 17812.343, per step time: 9.500<jupyter_text>完成重训练后，将在CheckPoint文件保存目录生成新的CheckPoint文件。此时`./ckpt`目录结构为：<jupyter_code>!tree ./models/ckpt/mindspore_load_model_for_inference_and_transfer<jupyter_output>./models/ckpt/mindspore_load_model_for_inference_and_transfer
├── checkpoint_lenet_1-1_1875.ckpt
├── checkpoint_lenet-1_1875.ckpt
├── checkpoint_lenet_1-2_1875.ckpt
├── checkpoint_lenet_1-3_1875.ckpt
├── checkpoint_lenet_1-graph.meta
└── checkpoint_lenet-graph.meta

0 directories, 6 files<jupyter_text>可以看出重训练保存的新的CheckPoint文件为`checkpoint_lenet_1-3_1875.ckpt`。运行以下一段代码，使用`eval`方法测试加载模型重训练后的得到的新的模型的精度。<jupyter_code>network = LeNet5(args.num_classes)
net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean")
net_opt = nn.Momentum(network.trainable_params(), args.lr, args.momentum)
model = Model(network, net_loss, net_opt, metrics={"Accuracy": Accuracy()})

print("============== Starting Testing ==============")
load_checkpoint(ckpt_file_name="./models/ckpt/mindspore_load_model_for_inference_and_transfer/checkpoint_lenet_1-3_1875.ckpt", net=network)
ds_eval = create_dataset(os.path.join(args.data_path, "test"),
                         args.batch_size,
                         1)

acc = model.eval(ds_eval, dataset_sink_mode=False)
print("============== {} ==============".format(acc))<jupyter_output>============== Starting Testing ==============
============== {'Accuracy': 0.9827724358974359} ==============
{'fraction_non_alphanumeric': 0.1372003424657534, 'fraction_numerical': 0.05108447488584475, 'mean_word_length': 5.164028144239226, 'pattern_counts': {'":': 5, '<': 41, '<?xml version=': 0, '>': 43, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '71861', 'license': 'Apache-2.0', 'n_tokens_mistral': 6153, 'n_tokens_neox': 5823, 'n_words': 856, 'path': 'tutorials/notebook/mindspore_load_model_for_inference_and_transfer.ipynb', 'repo_name': 'xuweichn/docs', 'size': '14016'}