TinyMS SSD300 教程¶
在本教程中，我们会演示使用TinyMS API进行训练/推理一个SSD300模型过程。
环境要求¶
Ubuntu:
18.04Python:
3.7.xFlask:
1.1.2MindSpore:
CPU-1.1.1TinyMS:
0.1.0numpy:
1.17.5Pillow:
8.1.0pip:
21.0.1requests:
2.18.4
介绍¶
TinyMS是一个高级API，目的是让新手用户能够更加轻松地上手深度学习。TinyMS可以有效地减少用户在构建、训练、验证和推理一个模型过程中的操作次数。TinyMS也提供了教程和文档帮助开发者更好的上手和开发。
本教程中，包含6个步骤：构建模型、下载数据集、训练模型、定义servable.json、启动服务器和推理，其中服务器在子进程中启动。
[1]:
import os
import json
import time
import tinyms as ts
import xml.etree.ElementTree as et
from PIL import Image
from tinyms import context, layers, primitives as P, Tensor
from tinyms.serving import start_server, predict, list_servables, shutdown, server_started
from tinyms.data import VOCDataset, download_dataset
from tinyms.vision import voc_transform, coco_eval, ImageViewer
from tinyms.model import Model, ssd300_mobilenetv2, ssd300_infer
from tinyms.losses import net_with_loss
from tinyms.optimizers import Momentum
from tinyms.callbacks import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor
from tinyms.utils.train.lr_generator import mobilenetv2_lr as ssd300_lr
from tinyms.initializers import initializer, TruncatedNormal
[WARNING] ME(17758:139967989708608,MainProcess):2021-03-19-15:42:00.473.223 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
1. 构建模型¶
[2]:
# 构建网络
net = ssd300_mobilenetv2(class_num=21)
2. 下载数据集¶
VOC数据集会被自动下载并存放到根目录，如果voc文件夹已经存在于根目录 ，则此步操作会被跳过
[3]:
# download the dataset
voc_path = '/root/voc'
if not os.path.exists(voc_path):
download_dataset('voc', '/root')
print('************Download complete*************')
else:
print('************Dataset already exists.**************')
************** Downloading the VOC2007 dataset **************
[███████████████████████████████████████████████████████████████████████████████████████████████████ ] 100.00%************Download complete*************
3. 训练模型¶
数据集中的训练集、验证集都会在此步骤中定义，同时也会定义训练参数。训练后生成的ckpt文件会保存到/etc/tinyms/serving/ssd300文件夹以便后续使用，训练完成后会进行验证并输出 Accuracy指标。
提示：训练过程非常漫长，建议跳过训练步骤并直接下载、使用本教程提供的ckpt文件进行后续的推理
[ ]:
class TrainingWrapper(layers.Layer):
"""
Encapsulation class of SSD300 network training.
Append an optimizer to the training network after that the construct
function can be called to create the backward graph.
Args:
network (Layer): The training network. Note that loss function should have been added.
optimizer (Optimizer): Optimizer for updating the weights.
sens (float): The adjust parameter. Default: 1.0.
"""
def __init__(self, network, optimizer, sens=1.0):
super(TrainingWrapper, self).__init__(auto_prefix=False)
self.network = network
self.network.set_grad()
self.weights = ts.ParameterTuple(network.trainable_params())
self.optimizer = optimizer
self.grad = P.GradOperation(get_by_list=True, sens_param=True)
self.sens = sens
self.hyper_map = P.HyperMap()
def construct(self, *args):
weights = self.weights
loss = self.network(*args)
sens = P.Fill()(P.DType()(loss), P.Shape()(loss), self.sens)
grads = self.grad(self.network, weights)(*args, sens)
return P.depend(loss, self.optimizer(grads))
def create_voc_label(voc_dir, voc_cls, usage='val'):
"""Get image path and annotation from VOC."""
if not os.path.isdir(voc_dir):
raise ValueError(f'Cannot find {voc_dir} dataset path.')
anno_dir = voc_dir
if os.path.isdir(os.path.join(voc_dir, 'Annotations')):
anno_dir = os.path.join(voc_dir, 'Annotations')
cls_map = {name: i for i, name in enumerate(voc_cls)}
# Fetch the specific xml files path
xml_files = []
with open(os.path.join(voc_dir, 'ImageSets', 'Main', usage+'.txt'), 'r') as f:
for line in f:
xml_files.append(line.strip('\n')+'.xml')
json_dict = {"images": [], "type": "instances", "annotations": [],
"categories": []}
bnd_id = 1
for xml_file in xml_files:
img_id = xml_files.index(xml_file)
tree = et.parse(os.path.join(anno_dir, xml_file))
root_node = tree.getroot()
file_name = root_node.find('filename').text
for obj in root_node.iter('object'):
cls_name = obj.find('name').text
if cls_name not in cls_map:
print(f'Label "{cls_name}" not in "{cls_map}"')
continue
bnd_box = obj.find('bndbox')
x_min = int(float(bnd_box.find('xmin').text)) - 1
y_min = int(float(bnd_box.find('ymin').text)) - 1
x_max = int(float(bnd_box.find('xmax').text)) - 1
y_max = int(float(bnd_box.find('ymax').text)) - 1
o_width = abs(x_max - x_min)
o_height = abs(y_max - y_min)
ann = {'area': o_width * o_height, 'iscrowd': 0,
'image_id': img_id,
'bbox': [x_min, y_min, o_width, o_height],
'category_id': cls_map[cls_name], 'id': bnd_id,
'ignore': 0,
'segmentation': []}
json_dict['annotations'].append(ann)
bnd_id = bnd_id + 1
size = root_node.find("size")
width = int(size.find('width').text)
height = int(size.find('height').text)
image = {'file_name': file_name, 'height': height, 'width': width,
'id': img_id}
json_dict['images'].append(image)
for cls_name, cid in cls_map.items():
cat = {'supercategory': 'none', 'id': cid, 'name': cls_name}
json_dict['categories'].append(cat)
anno_file = os.path.join(anno_dir, 'annotation.json')
with open(anno_file, 'w') as f:
json.dump(json_dict, f)
return anno_file
# 检查ckpt文件路径
ckpt_folder = '/etc/tinyms/serving/ssd300'
ckpt_path = '/etc/tinyms/serving/ssd300/ssd300.ckpt'
if not os.path.exists(ckpt_folder):
!mkdir -p /etc/tinyms/serving/ssd300
else:
print('ssd300 ckpt folder already exists')
# 设置训练参数
epoch_size = 800 # default is 800
batch_size = 32
voc_path = '/root/voc/VOCdevkit/VOC2007'
# 设置环境
context.set_context(mode=context.GRAPH_MODE, device_target="CPU")
dataset_sink_mode = False
# 创建数据集
train_dataset = VOCDataset(voc_path, task='Detection', usage='trainval', num_parallel_workers=4, shuffle=True, decode=True)
train_dataset = voc_transform.apply_ds(train_dataset, repeat_size=1, batch_size=batch_size, num_parallel_workers=4, is_training=True)
eval_dataset = VOCDataset(voc_path, task='Detection', usage='val', num_parallel_workers=4, shuffle=True, decode=True)
eval_dataset = voc_transform.apply_ds(eval_dataset, repeat_size=1, batch_size=batch_size, num_parallel_workers=4, is_training=False)
dataset_size = train_dataset.get_dataset_size()
total = eval_dataset.get_dataset_size()
# 定义loss函数
net = net_with_loss(net)
params = net.trainable_params()
for p in params:
if 'beta' not in p.name and 'gamma' not in p.name and 'bias' not in p.name:
p.set_data(initializer(TruncatedNormal(0.02), p.data.shape, p.data.dtype))
# 定义optimizer
pre_trained_epoch_size = 0
save_checkpoint_epochs = 10
lr = 0.01
lr = ssd300_lr(global_step=pre_trained_epoch_size * dataset_size,
lr_init=0.001, lr_end=0.001 * lr, lr_max=lr,
warmup_epochs=2, total_epochs=epoch_size,
steps_per_epoch=dataset_size)
loss_scale = 1.0
opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), lr,0.9, 1.5e-4, loss_scale)
model = Model(TrainingWrapper(net, opt, loss_scale))
model.compile()
ckpoint_cb = ModelCheckpoint(prefix="ssd300", config=CheckpointConfig(
save_checkpoint_steps=save_checkpoint_epochs * dataset_size,
keep_checkpoint_max=10))
print('************************Start training*************************')
model.train(epoch_size, train_dataset, callbacks=[ckpoint_cb, LossMonitor(), TimeMonitor(data_size=dataset_size)],
dataset_sink_mode=dataset_sink_mode)
model.save_checkpoint(ckpt_path)
print('************************Finished training*************************')
eval_net = ssd300_infer(class_num=21)
model = Model(eval_net)
model.load_checkpoint(ckpt_path)
# perform the model predict operation
print("\n========================================\n")
print("total images num: ", total)
print("Processing, please wait a moment...")
start = time.time()
pred_data = []
id_iter = 0
for data in eval_dataset.create_dict_iterator(output_numpy=True):
image_np = data['image']
image_shape = data['image_shape']
output = model.predict(Tensor(image_np))
for batch_idx in range(image_np.shape[0]):
pred_data.append({"boxes": output[0].asnumpy()[batch_idx],
"box_scores": output[1].asnumpy()[batch_idx],
"img_id": id_iter,
"image_shape": image_shape[batch_idx]})
id_iter += 1
cost_time = int((time.time() - start) * 1000)
print(f' 100% [{total}/{total}] cost {cost_time} ms')
# calculate mAP for the predict data
voc_cls = ['background',
'aeroplane', 'bicycle', 'bird', 'boat', 'bottle',
'bus', 'car', 'cat', 'chair', 'cow',
'diningtable', 'dog', 'horse', 'motorbike', 'person',
'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor']
anno_file = create_voc_label(voc_path, voc_cls)
mAP = coco_eval(pred_data, anno_file)
print("\n========================================\n")
print(f"mAP: {mAP}")
提示：如果跳过了训练步骤，下载预训练的ckpt文件并继续推理步骤
点击这里进行下载，并将ckpt文件保存到/etc/tinyms/serving/ssd300/ssd300.ckpt
或者运行以下代码下载 ssd300 ckpt文件：
[4]:
ssd300_ckpt_folder = '/etc/tinyms/serving/ssd300'
ssd300_ckpt_path = '/etc/tinyms/serving/ssd300/ssd300.ckpt'
# 创建路径，下载并存储ssd300 ckpt
if not os.path.exists(ssd300_ckpt_folder):
!mkdir -p /etc/tinyms/serving/ssd300
!wget -P /etc/tinyms/serving/ssd300 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/voc/ssd300.ckpt
else:
print('ssd300 ckpt folder already exists')
if not os.path.exists(ssd300_ckpt_path):
!wget -P /etc/tinyms/serving/ssd300 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/voc/ssd300.ckpt
else:
print('ssd300 ckpt file already exists')
ssd300 ckpt folder already exists
--2021-03-19 15:47:44-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/voc/ssd300.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 49.4.112.90, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 28056511 (27M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/ssd300/ssd300.ckpt’
ssd300.ckpt 100%[===================>] 26.76M 19.2MB/s in 1.4s
2021-03-19 15:47:46 (19.2 MB/s) - ‘/etc/tinyms/serving/ssd300/ssd300.ckpt’ saved [28056511/28056511]
4. 定义servable.json¶
运行下列代码定义servable json文件：
[5]:
servable_json = [{'name': 'ssd300',
'description': 'This servable hosts an ssd300 model predicting bounding boxes',
'model': {
"name": "ssd300",
"format": "ckpt",
"class_num": 21}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)
with open('servable.json', 'w') as json_file:
json_file.write(json_data)
5. 启动服务器¶
5.1 介绍¶
TinyMS推理是C/S（Client/Server）架构。TinyMS使用Flask这个轻量化的网页服务器架构作为C/S通讯的基础架构。为了能够对模型进行推理，用户必须首先启动服务器。如果成功启动，服务器会在子进程中运行并且会监听从地址127.0.0.1，端口号5000发送来的POST请求并且使用MindSpore作为后端来处理这些请求。后端会构建模型，运行推理并且返回结果给客户端
5.2 启动服务器¶
运行下列代码以启动服务器：
[6]:
start_server()
Server starts at host 127.0.0.1, port 5000
6. 推理¶
6.1 上传图片¶
用户需要上传一张图片作为输入，图片中要求含有以下类别的物体以供识别：
['background', 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 'bus', 'car', 'cat', 'chair', 'cow', 'diningtable', 'dog', 'horse', 'motorbike', 'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor']
点击这里下载本教程中使用的图片。上传图片，如果使用命令行终端，可以使用’scp’或者’wget’获取图片，如果使用Jupyter，点击菜单右上方的’Upload’按钮并且选择上传的图片。将图片保存在根目录下，重命名为’ssd300_test.jpeg’(或其他自定义名字)。
或者运行下列代码下载本教程使用的图片：
[7]:
# 下载图片
if not os.path.exists('/root/ssd300_test.jpeg'):
!wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/ssd300_test/ssd300_test.jpeg
else:
print('ssd300_test.jpeg already exists')
--2021-03-19 15:47:51-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/ssd300_test/ssd300_test.jpeg
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 49.4.112.90, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 70412 (69K) [image/jpeg]
Saving to: ‘/root/ssd300_test.jpeg’
ssd300_test.jpeg 100%[===================>] 68.76K --.-KB/s in 0.1s
2021-03-19 15:47:52 (560 KB/s) - ‘/root/ssd300_test.jpeg’ saved [70412/70412]
6.2 List servables¶
使用list_servables函数检查当前后端的serving模型
[8]:
list_servables()
[8]:
[{'description': 'This servable hosts an ssd300 model predicting bounding boxes',
'model': {'class_num': 21, 'format': 'ckpt', 'name': 'ssd300'},
'name': 'ssd300'}]
如果输出的description字段显示这是一个ssd300的模型，则可以继续到下一步发送推理请求
6.3 发送推理请求¶
运行predict函数发送推理请求，目前仅支持TOP1_CLASS输出策略，再运行ImageViewer.draw绘制边框
[9]:
# 设置图片路径和输出策略（目前仅支持'TOP1_CLASS'）
image_path = "/root/ssd300_test.jpeg"
strategy = "TOP1_CLASS"
labels = ['background',
'aeroplane', 'bicycle', 'bird', 'boat', 'bottle',
'bus', 'car', 'cat', 'chair', 'cow',
'diningtable', 'dog', 'horse', 'motorbike', 'person',
'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor']
# predict(image_path, servable_name, dataset_name, strategy)
# ImageViewer(img, title)
# ImageViewer.draw(predict_result, labels)
if server_started() is True:
res = predict(image_path, 'ssd300', 'voc', strategy)
img_viewer = ImageViewer(Image.open(image_path))
img_viewer.draw(res, labels)
else:
print("Server not started")
检查输出¶
如果输出了包含推测边框的图片，则表示已经进行了一次成功的输出，物体的类别和分数也会在图片中显示
关闭服务器¶
[10]:
shutdown()
[10]:
'Server shutting down...'