tinyms.model¶
-
class
tinyms.model.
Model
(network)[source]¶ High-Level API for Training or Evaluation.
Model groups layers into an object with training and inference features.
- Parameters
network (layers.Layer) – A training or testing network.
Examples
>>> from tinyms.model import Model, lenet5 >>> form tinyms.losses import SoftmaxCrossEntropyWithLogits >>> from tinyms.optimizers import Momentum >>> >>> net = lenet5(class_num=10) >>> model = Model(net) >>> net_loss = SoftmaxCrossEntropyWithLogits() >>> net_opt = Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> model.compile(loss_fn=net_loss, optimizer=net_opt, metrics=None) >>> # For details about how to build the dataset, please refer to the API document on the official website. >>> ds_train = create_custom_dataset() >>> model.train(2, ds_train)
-
build
(train_dataset=None, valid_dataset=None, sink_size=-1, epoch=1)[source]¶ Build computational graphs and data graphs with the sink mode.
Warning
This is an experimental prototype that is subject to change and/or deletion.
Note
Pre-build process only supports GRAPH_MODE and Ascend target currently. The interface builds the computational graphs, when the interface is executed first, ‘model.train’ only performs the graphs execution. It only support dataset sink mode.
- Parameters
train_dataset (Dataset) – A training dataset iterator. If train_dataset is defined, training graphs will be initialized. Default: None.
valid_dataset (Dataset) – An evaluating dataset iterator. If valid_dataset is defined, evaluation graphs will be initialized, and metrics in Model can not be None. Default: None.
sink_size (int) – Control the amount of data in each sink. Default: -1.
epoch (int) – Control the training epochs. Default: 1.
Examples
>>> from mindspore import Model, nn, FixedLossScaleManager >>> >>> # For details about how to build the dataset, please refer to the tutorial >>> # document on the official website. >>> dataset = create_custom_dataset() >>> net = Net() >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> loss_scale_manager = FixedLossScaleManager() >>> optim = nn.Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> model = Model(net, loss_fn=loss, optimizer=optim, metrics=None, loss_scale_manager=loss_scale_manager) >>> model.build(dataset, epoch=2) >>> model.train(2, dataset)
-
compile
(loss_fn=None, optimizer=None, metrics=None, eval_network=None, amp_level='O0', **kwargs)[source]¶ High-Level API for configure the train or eval network.
- Parameters
loss_fn (layers.Layer) – Objective function, if loss_fn is None, the network should contain the logic of loss and grads calculation, and the logic of parallel if needed. Default: None.
optimizer (layers.Layer) – Optimizer for updating the weights. Default: None.
metrics (Union[dict, set]) – A Dictionary or a set of metrics to be evaluated by the model during training and testing. eg: {‘accuracy’, ‘recall’}. Default: None.
eval_network (layers.Layer) – Network for evaluation. If not defined, network and loss_fn would be wrapped as eval_network. Default: None.
amp_level (str) –
Option for argument level in mindspore.amp.build_train_network, level for mixed precision training. Supports [“O0”, “O2”, “O3”, “auto”]. Default: “O0”.
O0: Do not change.
O2: Cast network to float16, keep batchnorm run in float32, using dynamic loss scale.
O3: Cast network to float16, with additional property ‘keep_batchnorm_fp32=False’.
auto: Set to level to recommended level in different devices. Set level to O2 on GPU, Set
level to O3 Ascend. The recommended level is choose by the export experience, cannot always generalize. User should specify the level for special network.
O2 is recommended on GPU, O3 is recommended on Ascend.
-
eval
(valid_dataset, callbacks=None, dataset_sink_mode=True)[source]¶ Evaluation API where the iteration is controlled by python front-end.
Configure to pynative mode or CPU, the evaluating process will be performed with dataset non-sink mode.
Note
If dataset_sink_mode is True, data will be sent to device. If the device is Ascend, features of data will be transferred one by one. The limitation of data transmission per time is 256M. When dataset_sink_mode is True, the step_end method of the Callback class will be executed when the epoch_end method is called.
- Parameters
- Returns
Dict, the key is the metric name defined by users and the value is the metrics value for the model in the test mode.
Examples
>>> from mindspore import Model, nn >>> >>> # For details about how to build the dataset, please refer to the tutorial >>> # document on the official website. >>> dataset = create_custom_dataset() >>> net = Net() >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=None, metrics={'acc'}) >>> acc = model.eval(dataset, dataset_sink_mode=False)
-
property
eval_network
¶ Get the model’s eval_network.
-
export
(inputs, file_name, file_format='MINDIR', **kwargs)[source]¶ Export the TinyMS prediction model to a file in the specified format.
- Parameters
inputs (Tensor) – Inputs of the net.
file_name (str) – File name of the model to be exported.
file_format (str) –
MindSpore currently supports AIR, ONNX and MINDIR format for exported model. Default: MINDIR.
AIR: Ascend Intermediate Representation. An intermediate representation format of Ascend model.
Recommended suffix for output file is ‘.air’.
ONNX: Open Neural Network eXchange. An open format built to represent machine learning models.
Recommended suffix for output file is ‘.onnx’.
MINDIR: MindSpore Native Intermediate Representation for Anf. An intermediate representation format
for MindSpore models. Recommended suffix for output file is ‘.mindir’.
kwargs (dict) –
Configuration options dictionary.
quant_mode: The mode of quant.
mean: Input data mean. Default: 127.5.
std_dev: Input data variance. Default: 127.5.
-
infer_predict_layout
(*predict_data)[source]¶ Generate parameter layout for the predict network in auto or semi auto parallel mode.
Data could be a single tensor or multiple tensors.
Note
Batch data should be put together in one tensor.
- Parameters
predict_data (Tensor) – One tensor or multiple tensors of predict data.
- Returns
Dict, Parameter layout dictionary used for load distributed checkpoint.
- Raises
RuntimeError – If get_context is not GRAPH_MODE.
Examples
>>> # This example should be run with multiple devices. Refer to the tutorial > Distributed Training on >>> # mindspore.cn. >>> import numpy as np >>> import mindspore as ms >>> from mindspore import Model, context, Tensor >>> from mindspore.context import ParallelMode >>> from mindspore.communication import init >>> >>> context.set_context(mode=context.GRAPH_MODE) >>> init() >>> context.set_auto_parallel_context(full_batch=True, parallel_mode=ParallelMode.SEMI_AUTO_PARALLEL) >>> input_data = Tensor(np.random.randint(0, 255, [1, 1, 32, 32]), ms.float32) >>> model = Model(Net()) >>> predict_map = model.infer_predict_layout(input_data)
-
infer_train_layout
(train_dataset, dataset_sink_mode=True, sink_size=-1)[source]¶ Generate parameter layout for the train network in auto or semi auto parallel mode. Only dataset sink mode is supported for now.
Warning
This is an experimental prototype that is subject to change and/or deletion.
Note
This is a pre-compile function. The arguments should be the same with model.train() function.
- Parameters
train_dataset (Dataset) – A training dataset iterator. If there is no loss_fn, a tuple with multiple data (data1, data2, data3, …) should be returned and passed to the network. Otherwise, a tuple (data, label) should be returned. The data and label would be passed to the network and loss function respectively.
dataset_sink_mode (bool) – Determines whether to pass the data through dataset channel. Configure pynative mode or CPU, the training process will be performed with dataset not sink. Default: True.
sink_size (int) – Control the amount of data in each sink. If sink_size = -1, sink the complete dataset for each epoch. If sink_size > 0, sink sink_size data for each epoch. If dataset_sink_mode is False, set sink_size as invalid. Default: -1.
- Returns
Dict, Parameter layout dictionary used for load distributed checkpoint
Examples
>>> # This example should be run with multiple devices. Refer to the tutorial > Distributed Training on >>> # mindspore.cn. >>> import numpy as np >>> import mindspore as ms >>> from mindspore import Model, context, Tensor, nn, FixedLossScaleManager >>> from mindspore.context import ParallelMode >>> from mindspore.communication import init >>> >>> context.set_context(mode=context.GRAPH_MODE) >>> init() >>> context.set_auto_parallel_context(parallel_mode=ParallelMode.SEMI_AUTO_PARALLEL) >>> >>> # For details about how to build the dataset, please refer to the tutorial >>> # document on the official website. >>> dataset = create_custom_dataset() >>> net = Net() >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> loss_scale_manager = FixedLossScaleManager() >>> optim = nn.Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> model = Model(net, loss_fn=loss, optimizer=optim, metrics=None, loss_scale_manager=loss_scale_manager) >>> layout_dict = model.infer_train_layout(dataset)
-
load_checkpoint
(ckpt_file_name, strict_load=False)[source]¶ Loads checkpoint info from a specified file.
- Parameters
- Returns
Dict, key is parameter name, value is a Parameter.
- Raises
ValueError – Checkpoint file is incorrect.
Examples
>>> ckpt_file_name = "./checkpoint/LeNet5-1_32.ckpt" >>> param_dict = model.load_checkpoint(ckpt_file_name)
-
predict
(*predict_data)[source]¶ Generate output predictions for the input samples.
Data could be a single tensor, a list of tensor, or a tuple of tensor.
Note
This is a pre-compile function. The arguments should be the same with model.predict() function.
- Parameters
predict_data (Optional[Tensor, list[Tensor], tuple[Tensor]]) – The predict data, can be a single tensor, a list of tensor, or a tuple of tensor.
- Returns
Tensor, array(s) of predictions.
Examples
>>> import mindspore as ms >>> from mindspore import Model, Tensor >>> >>> input_data = Tensor(np.random.randint(0, 255, [1, 1, 32, 32]), ms.float32) >>> model = Model(Net()) >>> result = model.predict(input_data)
-
property
predict_network
¶ Get the model’s predict_network.
-
train
(epoch, train_dataset, callbacks=None, dataset_sink_mode=True, sink_size=-1)[source]¶ Training API where the iteration is controlled by python front-end.
When setting pynative mode or CPU, the training process will be performed with dataset not sink.
Note
If dataset_sink_mode is True, data will be sent to device. If the device is Ascend, features of data will be transferred one by one. The limitation of data transmission per time is 256M. When dataset_sink_mode is True, the step_end method of the Callback class will be executed when the epoch_end method is called. If sink_size > 0, each epoch of the dataset can be traversed unlimited times until you get sink_size elements of the dataset. The next epoch continues to traverse from the end position of the previous traversal. The interface builds the computational graphs and then executes the computational graphs. However, when the ‘model.build’ is executed first, it only performs the graphs execution.
- Parameters
epoch (int) – Generally, total number of iterations on the data per epoch. When dataset_sink_mode is set to true and sink_size>0, each epoch sink sink_size steps on the data instead of total number of iterations.
train_dataset (Dataset) – A training dataset iterator. If there is no loss_fn, a tuple with multiple data (data1, data2, data3, …) should be returned and passed to the network. Otherwise, a tuple (data, label) should be returned. The data and label would be passed to the network and loss function respectively.
callbacks (Optional[list[Callback], Callback]) – List of callback objects or callback object, which should be executed while training. Default: None.
dataset_sink_mode (bool) – Determines whether to pass the data through dataset channel. Configure pynative mode or CPU, the training process will be performed with dataset not sink. Default: True.
sink_size (int) – Control the amount of data in each sink. If sink_size = -1, sink the complete dataset for each epoch. If sink_size > 0, sink sink_size data for each epoch. If dataset_sink_mode is False, set sink_size as invalid. Default: -1.
Examples
>>> from mindspore import Model, nn, FixedLossScaleManager >>> >>> # For details about how to build the dataset, please refer to the tutorial >>> # document on the official website. >>> dataset = create_custom_dataset() >>> net = Net() >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> loss_scale_manager = FixedLossScaleManager() >>> optim = nn.Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> model = Model(net, loss_fn=loss, optimizer=optim, metrics=None, loss_scale_manager=loss_scale_manager) >>> model.train(2, dataset)
-
property
train_network
¶ Get the model’s train_network.
-
tinyms.model.
load
(file_name)[source]¶ Load MindIR graph and return the network with parameters.
The returned object is wrapperred by a GraphLayer. However, there are some limitations to the current use of GraphLayer, see class
tinyms.layers.GraphLayer
for more details.- Parameters
file_name (str) – MindIR file name.
- Returns
GraphLayer instance, a compiled graph with parameters.
- Raises
ValueError – MindIR file is incorrect.
Examples
>>> import tinyms as ts >>> import tinyms.layers as layers >>> from tinyms.model import Model, load >>> >>> net = layers.Conv2d(1, 1, kernel_size=3) >>> model = Model(net) >>> input = ts.ones([1, 1, 3, 3]) >>> model.export(input, "net", file_format="MINDIR") ... >>> net = load("net.mindir") >>> print(net(input)) [[[[ 0.02548009 0.04010789 0.03120251] [ 0.00268656 0.02353744 0.03807815] [-0.00896441 -0.00303641 0.01502199]]]]
-
tinyms.model.
lenet5
(**kwargs)[source]¶ Get LeNet5 neural network.
- Parameters
class_num (int) – Class number. Default: 10.
- Returns
layers.Layer, layer instance of LeNet5 neural network.
Examples
>>> net = lenet5(class_num=10)
-
class
tinyms.model.
LeNet
(class_num=10, channel_num=1)[source]¶ LeNet architecture.
- Parameters
- Returns
Tensor, output tensor.
Examples
>>> LeNet(class_num=10)
-
tinyms.model.
resnet50
(**kwargs)[source]¶ Get ResNet50 neural network.
- Parameters
class_num (int) – Class number. Default: 10.
- Returns
layers.Layer, layer instance of ResNet50 neural network.
Examples
>>> net = resnet50(10)
-
class
tinyms.model.
ResNet
(block, layer_nums, in_channels, out_channels, strides, num_classes)[source]¶ ResNet architecture.
- Parameters
block (layers.Layer) – Block for network.
layer_nums (list) – Numbers of block in different layers.
in_channels (list) – Input channel in each layer.
out_channels (list) – Output channel in each layer.
strides (list) – Stride size in each layer.
num_classes (int) – The number of classes that the training images are belonging to.
- Returns
Tensor, output tensor.
Examples
>>> ResNet(ResidualBlock, >>> [3, 4, 6, 3], >>> [64, 256, 512, 1024], >>> [256, 512, 1024, 2048], >>> [1, 2, 2, 2], >>> 10)
-
tinyms.model.
mobilenetv2
(**kwargs)[source]¶ Get MobileNetV2 instance for model training, evaluation and prediction.
-
class
tinyms.model.
MobileNetV2
(class_num=1000, width_mult=1.0, round_nearest=8, input_channel=32, last_channel=1280, is_training=True)[source]¶ MobileNetV2 architecture.
- Parameters
- Returns
Tensor, output tensor.
-
tinyms.model.
ssd300_mobilenetv2
(**kwargs)[source]¶ Get SSD300 model instance for training, evaluation and prediction.
-
class
tinyms.model.
SSD300
(backbone, class_num=21, is_training=True)[source]¶ SSD300 Network. Default backbone is MobileNetV2.
- Parameters
backbone (layers.Layer) – backbone of ssd300 model.
class_num (int) – number of classes. Default: 21.
is_training (bool) – Specify if in training step. Default: True.
- Returns
Tensor, localization predictions. Tensor, class conf scores.
-
tinyms.model.
cycle_gan
(G_A, G_B)[source]¶ Get Cycle GAN network.
- Parameters
G_A (layers.Layer) – The generator net, currently it should be in [resnet, unet].
G_B (layers.Layer) – The generator net, currently it should be in [resnet, unet].
- Returns
Cycle GAN instance.
Examples
>>> gan_net = cycle_gan(G_A, G_B)
-
tinyms.model.
cycle_gan_infer
(g_model='resnet')[source]¶ Get Cycle GAN network for predict.
- Parameters
g_model (str) – The generator network type, currently it should be in [‘resnet’, ‘unet’]. Default: resnet.
- Returns
Cycle GAN instance.
Examples
>>> gan_net = cycle_gan(G_A, G_B)
-
tinyms.model.
densenet100
(**kwargs)[source]¶ Get DenseNet instance for model training, evaluation and prediction.
- Parameters
class_num (int) – The number of classes. Default: 10.
- Returns
model.DenseNet, DenseNet instance.
-
tinyms.model.
alexnet
(**kwargs)[source]¶ Get AlexNet neural network.
- Parameters
class_num (int) – Class number. Default: 10.
- Returns
layers.Layer, layer instance of AlexNet neural network.
Examples
>>> from tinyms.model import alexnet >>> >>> net = alexnet(class_num=10)
-
class
tinyms.model.
AlexNet
(class_num=1000)[source]¶ Get AlexNet neural network.
- Parameters
class_num (int) – Class number. Default: 1000.
- Returns
layers.Layer, layer instance of AlexNet neural network.
Examples
>>> from tinyms.model import AlexNet >>> >>> net = AlexNet(class_num=1000)
-
tinyms.model.
sentimentnet
(vocab_size, embed_size, num_hiddens, num_layers, bidirectional, num_classes, weight, batch_size)[source]¶ Sentiment network structure.
-
class
tinyms.model.
SentimentNet
(vocab_size, embed_size, num_hiddens, num_layers, bidirectional, num_classes, weight, batch_size)[source]¶ Sentiment network structure.
-
tinyms.model.
bert
(config, is_training, use_one_hot_embeddings=False)[source]¶ Get bert neural network.
-
class
tinyms.model.
Bert
(config, is_training, use_one_hot_embeddings=False)[source]¶ Bidirectional Encoder Representations from Transformers.
- Parameters
-
tinyms.model.
vgg11
(**kwargs)[source]¶ Get vgg11 neural network.
- Parameters
- Returns
layers.Layer, layer instance of vgg11 neural network.
Examples
>>> from tinyms.model import vgg11 >>> >>> net = vgg11(class_num=10)
-
tinyms.model.
vgg13
(**kwargs)[source]¶ Get vgg13 neural network.
- Parameters
- Returns
layers.Layer, layer instance of vgg13 neural network.
Examples
>>> from tinyms.model import vgg13 >>> >>> net = vgg13(class_num=10)
-
tinyms.model.
vgg16
(**kwargs)[source]¶ Get vgg16 neural network.
- Parameters
- Returns
layers.Layer, layer instance of vgg16 neural network.
Examples
>>> from tinyms.model import vgg16 >>> >>> net = vgg16(class_num=10)
-
tinyms.model.
vgg19
(**kwargs)[source]¶ Get vgg19 neural network.
- Parameters
- Returns
layers.Layer, layer instance of vgg19 neural network.
Examples
>>> from tinyms.model import vgg19 >>> >>> net = vgg19(class_num=10)
-
class
tinyms.model.
VGG
(features, class_num=1000)[source]¶ Get VGG neural network.
- Parameters
features (layers.Layer) – Feature extractor.
class_num (int) – Class number. Default: 1000.
- Returns
layers.Layer, layer instance of AlexNet neural network.
Examples
>>> from tinyms.model import VGG >>> >>> net = VGG(features=make_layers(cfg=cfgs['A']),class_num=1000)
-
tinyms.model.
deepfm
(**kwargs)[source]¶ Get DeepFM neural network.
- Parameters
- Returns
layers.Layer, layer instance of DeepFM neural network.
Examples
>>> from tinyms.model import deepfm >>> >>> net = deepfm(39, 184965, 80)
-
class
tinyms.model.
DeepFM
(field_size, vocab_size, embed_size, keep_prob=0.9, convert_dtype=False)[source]¶ DeepFM architecture.
- Parameters
- Returns
Tensor, output tensor.
Examples
>>> from tinyms.model import DeepFM >>> >>> DeepFM(field_size=39, vocab_size=184965, embed_size=80)
-
class
tinyms.model.
DeepFMEvalModel
(network)[source]¶ Provide DeepFM training network.
- Parameters
network (layers.Layer) – The base network.
- Returns
logits (Tensor) - With the shape as [batch_size, vocab_size] and data type as float32. predict_probs (Tensor) - With the same shape as [batch_size, vocab_size] and data type as float32. labels (Tensor) - With the same shape as labels.
Examples
>>> from tinyms.model import deepfm, DeepFMEvalModel >>> >>> net = deepfm() >>> eval_net = DeepFMEvalModel(net)
-
class
tinyms.model.
DeepFMWithLoss
(network, l2_coef=1e-06)[source]¶ Provide DeepFM training loss through network.
- Parameters
network (layers.Layer) – The training network.
l2_coef (float) – value for l2 loss. Default: 1e-6.
- Returns
Tensor, the loss of the network.
Examples
>>> from tinyms.model import deepfm, DeepFMWithLoss, DeepFMTrainModel >>> >>> net = deepfm() >>> train_net = DeepFMTrainModel(DeepFMWithLoss(net))
-
class
tinyms.model.
DeepFMTrainModel
(network, learning_rate=0.0005, eps=5e-08, loss_scale=1024.0)[source]¶ Provide DeepFM training network.
- Parameters
network (layers.Layer) – The base network.
learning_rate (float) – A value or a graph for the learning rate. Default: 0.0005.
eps (float) – Term added to the denominator to improve numerical stability.
be greater than 0. Default (Should) – 0.00000005.
loss_scale (float) – A floating point value for the loss scale.
be greater than 0. Default – 1024.0.
- Returns
Tensor, the value passed by last operator.
Examples
>>> from tinyms.model import deepfm, DeepFMWithLoss, DeepFMTrainModel >>> >>> net = deepfm() >>> train_net = DeepFMTrainModel(DeepFMWithLoss(net))