TinyMS MobileNetV2 Tutorial

In this tutorial, using TinyMS API to train/serve a MobileNetV2 model will be demonstrated.

Prerequisite

• Ubuntu: 18.04

• Python: 3.7.x

• Flask: 1.1.2

• MindSpore: CPU-1.1.1

• TinyMS: 0.1.0

• numpy: 1.17.5

• Pillow: 8.1.0

• pip: 21.0.1

• requests: 2.18.4

Introduction

TinyMS is a high-level API which is designed for amateur of deep learning. It minimizes the number of actions of users required to construct, train, evaluate and serve a model. TinyMS also provides tutorials and documentations for developers.

This tutorial consists of six parts, constructing the model, downloading dataset, training, define servable json, starting server and making predictions in which the server will be run in a sub process.

import os
import json

from PIL import Image
from tinyms import context
from tinyms.serving import start_server, predict, list_servables, shutdown, server_started
from tinyms.model import Model, mobilenetv2
from tinyms.data import Cifar10Dataset, download_dataset
from tinyms.vision import cifar10_transform, ImageViewer
from tinyms.metrics import Accuracy
from tinyms.optimizers import Momentum
from tinyms.losses import CrossEntropyWithLabelSmooth
from tinyms.utils.train.loss_manager import FixedLossScaleManager
from tinyms.utils.train.lr_generator import mobilenetv2_lr
from tinyms.utils.train.cb_config import mobilenetv2_cb

[WARNING] ME(8895:140407127349056,MainProcess):2021-03-19-15:03:40.515.970 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.

WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.

1. Construct the model

TinyMS encapsulates init and construct of the MobileNetV2 model, the line of the code is reduced to construct the model:

# build the model
net = mobilenetv2(class_num=10, is_training=True)
model = Model(net)

2. Download dataset

The cifar10 dataset will be downloaded if cifar10 folder didn’t exist at the root. If cifar10 folder already exists, this step will not be performed.

# download the dataset
cifar10_path = '/root/cifar10/cifar-10-batches-bin'
if not os.path.exists(cifar10_path):
    download_dataset('cifar10', '/root')
    print('************Download complete*************')
else:
    print('************Dataset already exists.**************')

************** Downloading the Cifar10 dataset **************
[████████████████████████████████████████████████████████████████████████████████████████████████████] 100.00%
============== /root/cifar10/cifar-10-binary.tar.gz is ready ==============
************Download complete*************

3. Train the model & evaluation

The dataset for both training and evaluation will be defined here, and the parameters for training also set in this block. A trained ckpt file will be saved to /etc/tinyms/serving/mobilenetv2 folder for later use, meanwhile the evaluation will be performed and the Accuracy can be checked

Notice: Since training MobileNetV2 on CPU is time consuming, we recommend skip training and using provided ckpt files to run.

# check ckpt folder exists or not
ckpt_folder = '/etc/tinyms/serving/mobilenetv2'
ckpt_path = '/etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt'
if not os.path.exists(ckpt_folder):
    !mkdir -p  /etc/tinyms/serving/mobilenetv2
else:
    print('mobilenetv2 ckpt folder already exists')

# Declare common variables
epoch_size = 60 # default is 60
batch_size = 32
class_num = 10

# set runtime environment
device_target="CPU"
dataset_sink_mode = False
context.set_context(mode=context.GRAPH_MODE, device_target=device_target)

# create cifar10 dataset
train_dataset = Cifar10Dataset(cifar10_path, num_parallel_workers=4, shuffle=True)
train_dataset = cifar10_transform.apply_ds(train_dataset, repeat_size=1, batch_size=32, is_training=True)
eval_dataset = Cifar10Dataset(cifar10_path, num_parallel_workers=4, shuffle=True)
eval_dataset = cifar10_transform.apply_ds(eval_dataset, repeat_size=1, batch_size=32, is_training=False)
step_size = train_dataset.get_dataset_size()

# define the loss function
label_smooth = 0.1
loss = CrossEntropyWithLabelSmooth(smooth_factor=label_smooth,num_classes=class_num)

# get learning rate
lr_max = 0.001
lr_init_scale = 0.01
lr_end_scale = 0.01
lr = mobilenetv2_lr(global_step=0,
                    lr_init=lr_max*lr_init_scale,
                    lr_end=lr_max*lr_end_scale,
                    lr_max=lr_max,
                    warmup_epochs=2,
                    total_epochs=epoch_size,
                    steps_per_epoch=step_size)

# define the optimizer
loss_scale = FixedLossScaleManager(1024, drop_overflow_update=False)
opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()),lr, 0.9, 4e-5, 1024)
model.compile(loss_fn=loss, optimizer=opt, metrics={"Accuracy": Accuracy()},loss_scale_manager=loss_scale)

# configure checkpoint to save weights and do training job
save_checkpoint_epochs = 10
ckpoint_cb = mobilenetv2_cb(device_target=device_target,
                            lr=lr,
                            is_saving_checkpoint=True,
                            save_checkpoint_epochs=save_checkpoint_epochs,
                            step_size=step_size)


print('************************Start training*************************')
model.train(epoch_size, train_dataset, callbacks=ckpoint_cb, dataset_sink_mode=dataset_sink_mode)
model.save_checkpoint(ckpt_path)
print('************************Finished training*************************')

model.load_checkpoint(ckpt_path)
print('************************Start evaluation*************************')
acc = model.eval(eval_dataset, dataset_sink_mode=dataset_sink_mode)
print("============== Accuracy:{} ==============".format(acc))

Notice: If skipped training process, download the pretrained ckpt file and continue to serving

Click HERE to download MobileNetV2 ckpt file and save this file to /etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt.

Or run the following code to download and store the ckpt file:

mobilenetv2_ckpt_folder = '/etc/tinyms/serving/mobilenetv2'
mobilenetv2_ckpt_path = '/etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt'

if not os.path.exists(mobilenetv2_ckpt_folder):
    !mkdir -p  /etc/tinyms/serving/mobilenetv2
    !wget -P /etc/tinyms/serving/mobilenetv2 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/mobilenetv2.ckpt
else:
    print('mobilenetv2 ckpt folder already exists')
    if not os.path.exists(mobilenetv2_ckpt_path):
        !wget -P /etc/tinyms/serving/mobilenetv2 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/mobilenetv2.ckpt
    else:
        print('mobilenetv2 ckpt file already exists')

mobilenetv2 ckpt folder already exists
--2021-03-19 15:06:26--  https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/mobilenetv2.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.5, 49.4.112.90, 49.4.112.113, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.5|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 18509001 (18M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt’

mobilenetv2.ckpt    100%[===================>]  17.65M  16.1MB/s    in 1.1s

2021-03-19 15:06:29 (16.1 MB/s) - ‘/etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt’ saved [18509001/18509001]

4. Define servable.json

Define the MobileNetV2 servable json file for model name, format and number of classes for later use.

servable_json = [{'name': 'mobilenetv2',
                  'description': 'This servable hosts a mobilenetv2 model predicting 10 classes of objects',
                  'model': {
                      "name": "mobilenetv2",
                      "format": "ckpt",
                      "class_num": 10}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)

with open('servable.json', 'w') as json_file:
    json_file.write(json_data)

5. Start server

5.1 Introduction

TinyMS Serving is a C/S(client/server) structure. There is a server and client. TinyMS using Flask which is a micro web framework written in python as the C/S communication tool. In order to serve a model, user must start server first. If successfully started, the server will be run in a subprocess and listening to POST requests from 127.0.0.1 port 5000 sent by client and handle the requests using MindSpore backend which will construct the model, run the prediction and send the result back to the client.

5.2 start server

Run the following code block to start the server:

start_server()

Server starts at host 127.0.0.1, port 5000

6. Make predictions

6.1 Upload the pic

A picture is required to be the input. This ckpt requires a picture containing the following objects:

['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']

Click airplane to download the picture used in this tutorial. Upload the pic, if using terminal, either scp or wget will do, if running in Jupyter, click Upload button at the top right and select the picture.

Save the picture at the root folder, rename to airplane.jpg(or any name you want).

Or run the following code to download the picture used in this tutorial

# download the airplane pic
if not os.path.exists('/root/airplane.jpg'):
    !wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/objects/airplane.jpg
else:
    print('airplane.jpg already exists')

--2021-03-19 15:06:37--  https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/objects/airplane.jpg
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.5, 49.4.112.90, 49.4.112.113, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.5|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 151188 (148K) [image/jpeg]
Saving to: ‘/root/airplane.jpg’

airplane.jpg        100%[===================>] 147.64K   560KB/s    in 0.3s

2021-03-19 15:06:38 (560 KB/s) - ‘/root/airplane.jpg’ saved [151188/151188]

6.2 List servables

Now, we can use list_servables function to check what model is servable right now.

list_servables()

[{'description': 'This servable hosts a mobilenetv2 model predicting 10 classes of objects',
  'model': {'class_num': 10, 'format': 'ckpt', 'name': 'mobilenetv2'},
  'name': 'mobilenetv2'}]

If the output description shows it is a MobileNetV2 model, then we can continue to next step to send our request.

6.3 Sending request and get the result

Run predict function to send the request, select between TOP1_CLASS and TOP5_CLASS to check the output:

# set image path and output strategy(TOP1_CLASS or TOP5_CLASS)
image_path = "/root/airplane.jpg"
strategy = "TOP1_CLASS"

# predict(image_path, servable_name, dataset_name, strategy)
if server_started() is True:
    img_viewer = ImageViewer(Image.open(image_path), image_path)
    img_viewer.show()
    print(predict(image_path, 'mobilenetv2', 'cifar10', strategy))
else:
    print("Server not started")

TOP1: airplane, score: 0.22268821299076080322

Check output

If user can see output similar to this:

TOP1: airplane, score: 0.22268821299076080322

that means the prediction result is returned and the inference is completed.

## Shutdown server

To restart and try another checkpoint file, click Kernel at the top, then Restart & Clear Output, replace the servable_json code and predict() function

Run the following code to shutdown Flask server:

shutdown()

'Server shutting down...'