Welcome to TinyMS’s documentation!¶
TinyMS is an Easy-to-Use deep learning development toolkit based on MindSpore, designed to providing quick-start guidelines for machine learning beginners.
What is TinyMS¶
TinyMS is an Easy-to-Use deep learning development toolkit based on MindSpore, designed to providing quick-start guidelines for machine learning beginners.
Install TinyMS¶
Installation For Beginners¶
Pypi¶
For users who own a clean environment, it is recommended to use pypi to install TinyMS given that the following requirements are meet. For those who don’t, Anaconda is a good choice for setting up the python environment.
Prerequisites
OS:
Ubuntu 18.04orWindows 10Python:
3.7.5
For China based users it is recommended to run the following command lines to help with faster download
mkdir -pv /root/.pip \
&& echo "[global]" > /root/.pip/pip.conf \
&& echo "trusted-host=mirrors.aliyun.com" >> /root/.pip/pip.conf \
&& echo "index-url=http://mirrors.aliyun.com/pypi/simple/" >> /root/.pip/pip.conf
pip install tinyms==0.1.0
Docker¶
For those who don’t want to affect the local develop environment due to difficulty of meeting the prerequisites, using docker to install is recommended
docker:
v18.06.1-ce
If user wants to try the tutorials that are written in .ipynb files，please pull jupyter version of TinyMS in which jupyter components are installed by default
Default version
docker pull tinyms/tinyms:0.1.0
Jupyter version
If user wants to try jupyter, run the following command line
docker pull tinyms/tinyms:0.1.0-jupyter
docker run -it --net=host tinyms/tinyms:0.1.0-jupyter
Open a browser on the local machine, type in
<Your_external_IP_address>:8888
Example: 188.8.8.88:8888, the default password is tinyms，then user can log in to jupyter
Installation For Experienced Developers¶
For developers who want to develop based on TinyMS, install from source
sudo apt-get install -y libssl-dev
git clone https://github.com/tinyms-ai/tinyms.git
cd tinyms
pip install -r requirements.txt
python setup.py install
Validate installation¶
Create a python or jupyter kernel, input the following codes
import tinyms as ts
from tinyms.primitives import tensor_add
x = ts.ones([2, 3])
y = ts.ones([2, 3])
print(tensor_add(x, y))
If the output is similar to below, then the installation is valid
[[2. 2. 2.]
[2. 2. 2.]]
Implementing an Image Classification App in One Minute¶
In this tutorial, constructing a LeNet5 model, downloading dataset, training, starting the server and making predictions of the model using TinyMS API will be demonstrated.
Prerequisite¶
Ubuntu:
18.04Python:
3.7.xFlask:
1.1.2MindSpore:
CPU-1.1.1TinyMS:
0.1.0numpy:
1.17.5Pillow:
8.1.0pip:
21.0.1requests:
2.18.4
Introduction¶
TinyMS is a high-level API which is designed for amateur of deep learning. It minimizes the number of actions of users required to construct, train, evaluate and serve a model. TinyMS also provides tutorials and documentations for developers.
This tutorial consists of six parts, constructing the model, downloading dataset, training, define servable json, starting server and making predictions in which the server will be run in a sub process.
[1]:
import os
import json
import tinyms.optimizers as opt
from PIL import Image
from tinyms import context
from tinyms.data import MnistDataset, download_dataset
from tinyms.vision import mnist_transform, ImageViewer
from tinyms.model import Model, lenet5
from tinyms.serving import start_server, predict, list_servables, shutdown, server_started
from tinyms.metrics import Accuracy
from tinyms.losses import SoftmaxCrossEntropyWithLogits
from tinyms.callbacks import ModelCheckpoint, CheckpointConfig, LossMonitor
[WARNING] ME(14780:139834376955712,MainProcess):2021-03-19-15:56:12.182.640 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
1. Construct the model¶
TinyMS encapsulates init and construct of the LeNet5 model, the line of the code is reduced to construct the LeNet5 model:
[2]:
# build the network
net = lenet5(class_num=10)
model = Model(net)
2. Download dataset¶
The MNIST dataset will be downloaded if mnist folder didn’t exist at the root. If mnist folder already exists, this step will not be performed.
[3]:
# download the dataset
mnist_path = '/root/mnist'
if not os.path.exists(mnist_path):
download_dataset('mnist', '/root')
print('************Download complete*************')
else:
print('************Dataset already exists.**************')
************Dataset already exists.**************
3. Train the model & evaluation¶
The dataset for both training and evaluation will be defined here, and the parameters for training also set in this block. A trained ckpt file will be saved to /etc/tinyms/serving/lenet5 folder for later use, meanwhile the evaluation will be performed and the Accuracy can be checked
[ ]:
# check lenet folder exists or not
ckpt_folder = '/etc/tinyms/serving/lenet5'
ckpt_path = '/etc/tinyms/serving/lenet5/lenet5.ckpt'
if not os.path.exists(ckpt_folder):
!mkdir -p /etc/tinyms/serving/lenet5
else:
print('lenet5 ckpt folder already exists')
# set environment parameters
device_target = "CPU"
context.set_context(mode=context.GRAPH_MODE, device_target=device_target)
dataset_sink_mode = False
# define the training and evaluation dataset
train_dataset = MnistDataset(os.path.join(mnist_path, "train"), shuffle=True)
train_dataset = mnist_transform.apply_ds(train_dataset)
eval_dataset = MnistDataset(os.path.join(mnist_path, "test"), shuffle=True)
eval_dataset = mnist_transform.apply_ds(eval_dataset)
# parameters for training
lr = 0.01
momentum = 0.9
epoch_size = 1
batch_size = 32
# define the loss function
net_loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')
# define the optimizer
net_opt = opt.Momentum(net.trainable_params(), lr, momentum)
net_metrics={"Accuracy": Accuracy()}
model.compile(loss_fn=net_loss, optimizer=net_opt, metrics=net_metrics)
print('************************Start training*************************')
ckpoint_cb = ModelCheckpoint(prefix="checkpoint_lenet", config=CheckpointConfig(save_checkpoint_steps=1875, keep_checkpoint_max=10))
model.train(epoch_size, train_dataset, callbacks=[ckpoint_cb, LossMonitor()],dataset_sink_mode=dataset_sink_mode)
print('************************Finished training*************************')
model.save_checkpoint(ckpt_path)
model.load_checkpoint(ckpt_path)
print('************************Start evaluation*************************')
acc = model.eval(eval_dataset, dataset_sink_mode=dataset_sink_mode)
print("============== Accuracy:{} ==============".format(acc))
4. Define servable.json¶
Define the lenet5 servable json file for model name, format and number of classes for serving.
[5]:
servable_json = [{'name': 'lenet5',
'description': 'This servable hosts a lenet5 model predicting numbers',
'model': {
"name": "lenet5",
"format": "ckpt",
"class_num": 10}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)
with open('servable.json', 'w') as json_file:
json_file.write(json_data)
5. Start server¶
5.1 Introduction¶
TinyMS Serving is a C/S(client/server) structure. TinyMS using Flask which is a micro web framework written in python as the C/S communication tool. In order to serve a model, user must start server first. If successfully started, the server will be run in a subprocess and listening to POST requests from 127.0.0.1 port 5000 sent by client and handle the requests using MindSpore backend which constructs the model, run the prediction and send the result back to the client.
5.2 Start server¶
Run the following code block to start the server:
[6]:
start_server()
Server starts at host 127.0.0.1, port 5000
6. Make predictions¶
6.1 Upload the pic¶
A picture of a single digit number is required to be the input. The picture we use in this tutorial can be found HERE, then save the picture to the root folder, and rename it to 7.png (or any other name you like).
Or run the following code to download the pic for this tutorial:
[7]:
if not os.path.exists('/root/7.png'):
!wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/numbers/7.png
else:
print('7.png already exists')
--2021-03-19 15:56:37-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/numbers/7.png
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.90, 49.4.112.113, 121.36.121.44, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.90|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 34970 (34K) [image/png]
Saving to: ‘/root/7.png’
7.png 100%[===================>] 34.15K --.-KB/s in 0.1s
2021-03-19 15:56:37 (345 KB/s) - ‘/root/7.png’ saved [34970/34970]
6.2 List servables¶
Use list_servables function to check what model is being served right now.
[8]:
list_servables()
[8]:
[{'description': 'This servable hosts a lenet5 model predicting numbers',
'model': {'class_num': 10, 'format': 'ckpt', 'name': 'lenet5'},
'name': 'lenet5'}]
If the output description shows it is a lenet5 model, then we can continue to next step to send our request.
6.3 Sending request and get the result¶
Run predict function to send the request, select between TOP1_CLASS and TOP5_CLASS:
[9]:
image_path = "/root/7.png"
strategy = "TOP1_CLASS"
# predict(image_path, servable_name, dataset='mnist', strategy='TOP1_CLASS')
if server_started() is True:
img_viewer = ImageViewer(Image.open(image_path), image_path)
img_viewer.show()
print(predict(image_path, 'lenet5', 'mnist', strategy))
else:
print("Server not started")
TOP1: 7, score: 0.99977773427963256836
If user can see the output similar to this:
TOP1: 7, score: 0.99934917688369750977
that means the prediction is successfully performed
TinyMS ResNet50 Tutorial¶
In this tutorial, using TinyMS API to train/serve a ResNet50 model will be demonstrated.
Prerequisite¶
Ubuntu:
18.04Python:
3.7.xFlask:
1.1.2MindSpore:
CPU-1.1.1TinyMS:
0.1.0numpy:
1.17.5Pillow:
8.1.0pip:
21.0.1requests:
2.18.4
Introduction¶
TinyMS is a high-level API which is designed for amateur of deep learning. It minimizes the number of actions of users required to construct, train, evaluate and serve a model. TinyMS also provides tutorials and documentations for developers.
This tutorial consists of six parts, constructing the model, downloading dataset, training, define servable json, starting server and making predictions in which the server will be run in a sub process.
[1]:
import os
import json
from PIL import Image
from tinyms import context
from tinyms.serving import start_server, predict, list_servables, shutdown, server_started
from tinyms.data import Cifar10Dataset, download_dataset, ImageFolderDataset
from tinyms.vision import cifar10_transform, ImageViewer, imagefolder_transform
from tinyms.model import Model, resnet50
from tinyms.callbacks import ModelCheckpoint, CheckpointConfig, LossMonitor
from tinyms.metrics import Accuracy
from tinyms.optimizers import Momentum
from tinyms.losses import SoftmaxCrossEntropyWithLogits
[WARNING] ME(12569:140321685239616,MainProcess):2021-03-19-15:21:33.633.399 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
1. Construct the model¶
TinyMS encapsulates init and construct of the ResNet50 model, the line of the code is reduced to construct the model:
[2]:
# build the network
net = resnet50(class_num=10)
model = Model(net)
2. Download dataset¶
Training ResNet50 model with cifar10 dataset will be demonstrated, while we provide two pre-trained ckpt files for users to download, one is trained with cifar10 dataset and the other one is trained with ImageNet2012 dataset.
[3]:
# download the cifar10 dataset
cifar10_path = '/root/cifar10/cifar-10-batches-bin'
if not os.path.exists(cifar10_path):
download_dataset('cifar10', '/root')
print('************Download complete*************')
else:
print('************Dataset already exists.**************')
************** Downloading the Cifar10 dataset **************
[███████████████████████████████████████████████████████████████████████████████████████████████████ ] 99.81%************Download complete*************
3. Train the model & evaluation¶
The dataset for both training and evaluation will be defined here, and the parameters for training also set in this block. A trained ckpt file will be saved to /etc/tinyms/serving/resnet50_cifar10 folder for later use, meanwhile the evaluation will be performed and the Accuracy can be checked
Notice: Since training ResNet50 on CPU is time consuming, we recommend skip training and using provided ckpt files to run.
[ ]:
# check ckpt folder exists or not
cifar10_ckpt_folder = '/etc/tinyms/serving/resnet50_cifar10'
cifar10_ckpt_path = '/etc/tinyms/serving/resnet50_cifar10/resnet50.ckpt'
if not os.path.exists(cifar10_ckpt_folder):
!mkdir -p /etc/tinyms/serving/resnet50_cifar10
else:
print('resnet50_cifar10 ckpt folder already exists')
epoch_size = 90 # default is 90
batch_size = 32
# set environment parameters
dataset_sink_mode = False
device_target = "CPU"
context.set_context(mode=context.GRAPH_MODE, device_target=device_target)
# set dataset parameters
train_dataset = Cifar10Dataset(cifar10_path, num_parallel_workers=4, shuffle=True)
train_dataset = cifar10_transform.apply_ds(train_dataset, repeat_size=1, batch_size=32, is_training=True)
eval_dataset = Cifar10Dataset(cifar10_path, num_parallel_workers=4, shuffle=True)
eval_dataset = cifar10_transform.apply_ds(eval_dataset, repeat_size=1, batch_size=32, is_training=False)
step_size = train_dataset.get_dataset_size()
save_checkpoint_epochs = 5
ckpoint_cb = ModelCheckpoint(prefix="resnet_cifar10", config=CheckpointConfig(
save_checkpoint_steps=save_checkpoint_epochs * train_dataset.get_dataset_size(),
keep_checkpoint_max=10))
# define the loss function
net_loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean")
# define the optimizer
net_opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), 0.01, 0.9)
model.compile(loss_fn=net_loss, optimizer=net_opt, metrics={"Accuracy": Accuracy()})
print('************************Start training*************************')
model.train(epoch_size, train_dataset, callbacks=[ckpoint_cb, LossMonitor()],dataset_sink_mode=dataset_sink_mode)
model.save_checkpoint(cifar10_ckpt_path)
print('************************Finished training*************************')
model.load_checkpoint(cifar10_ckpt_path)
print('************************Start evaluation*************************')
acc = model.eval(eval_dataset, dataset_sink_mode=dataset_sink_mode)
print("============== Accuracy:{} ==============".format(acc))
Notice: If skipped training process, download the pretrained ckpt files and continue to serving
Click resnet_imagenet to download resnet-imagenet ckpt file and click resnet_cifar to download resnet-cifar ckpt file. Save this file to /etc/tinyms/serving/resnet50_<dataset_name>/resnet50.ckpt.
Or run the following code to download the resnet_imagenet and resnet_cifar ckpt file:
[4]:
# check lenet folder exists or not, and download resnet50_imagenet
imagenet2012_ckpt_folder = '/etc/tinyms/serving/resnet50_imagenet2012'
imagenet2012_ckpt_path = '/etc/tinyms/serving/resnet50_imagenet2012/resnet50.ckpt'
if not os.path.exists(imagenet2012_ckpt_folder):
!mkdir -p /etc/tinyms/serving/resnet50_imagenet2012
!wget -P /etc/tinyms/serving/resnet50_imagenet2012 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/imagenet2012/resnet50.ckpt
else:
print('imagenet2012 ckpt folder already exists')
if not os.path.exists(imagenet2012_ckpt_path):
!wget -P /etc/tinyms/serving/resnet50_imagenet2012 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/imagenet2012/resnet50.ckpt
else:
print('imagenet2012 ckpt file already exists')
# check lenet folder exists or not
cifar10_ckpt_folder = '/etc/tinyms/serving/resnet50_cifar10'
cifar10_ckpt_path = '/etc/tinyms/serving/resnet50_cifar10/resnet50.ckpt'
if not os.path.exists(cifar10_ckpt_folder):
!mkdir -p /etc/tinyms/serving/resnet50_cifar10
!wget -P /etc/tinyms/serving/resnet50_cifar10 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/resnet50.ckpt
else:
print('cifar10 ckpt folder already exists')
if not os.path.exists(cifar10_ckpt_path):
!wget -P /etc/tinyms/serving/resnet50_cifar10 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/resnet50.ckpt
else:
print('cifar10 ckpt file already exists')
imagenet2012 ckpt folder already exists
--2021-03-19 15:23:45-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/imagenet2012/resnet50.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 121.36.121.44, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 188521005 (180M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/resnet50_imagenet2012/resnet50.ckpt’
resnet50.ckpt 100%[===================>] 179.79M 36.7MB/s in 5.9s
2021-03-19 15:23:52 (30.4 MB/s) - ‘/etc/tinyms/serving/resnet50_imagenet2012/resnet50.ckpt’ saved [188521005/188521005]
cifar10 ckpt folder already exists
--2021-03-19 15:23:52-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/resnet50.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 121.36.121.44, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 188462121 (180M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/resnet50_cifar10/resnet50.ckpt’
resnet50.ckpt 100%[===================>] 179.73M 35.7MB/s in 5.6s
2021-03-19 15:23:58 (32.3 MB/s) - ‘/etc/tinyms/serving/resnet50_cifar10/resnet50.ckpt’ saved [188462121/188462121]
4. Define servable.json¶
Choose only one of the following two code blocks to define the servable json file which will be used later
Run this code to define the servable json file for ResNet50_imagenet2012 model:
[5]:
servable_json = [{'name': 'resnet50_imagenet2012',
'description': 'This servable hosts a resnet50 model predicting mushrooms',
'model': {
"name": "resnet50",
"format": "ckpt",
"class_num": 9}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)
with open('servable.json', 'w') as json_file:
json_file.write(json_data)
Or run this code to define the servable json file for ResNet50_cifar10 model:
[ ]:
servable_json = [{'name': 'resnet50_cifar10',
'description': 'This servable hosts a resnet50 model predicting 10 classes of objects',
'model': {
"name": "resnet50",
"format": "ckpt",
"class_num": 10}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)
with open('servable.json', 'w') as json_file:
json_file.write(json_data)
5. Start server¶
5.1 Introduction¶
TinyMS Serving is a C/S(client/server) structure. There is a server and client. TinyMS using Flask which is a micro web framework written in python as the C/S communication tool. In order to serve a model, user must start server first. If successfully started, the server will be run in a subprocess and listening to POST requests from 127.0.0.1 port 5000 sent by client and handle the requests using MindSpore backend which will construct the model, run the prediction and send the result back to the client.
5.2 start server¶
Run the following code block to start the server:
[6]:
start_server()
Server starts at host 127.0.0.1, port 5000
6. Make predictions¶
6.1 Upload the pic¶
A picture is required to be the input. The resnet_imagenet ckpt requires a mushroom picture, while the resnet_cifar ckpt requires a picture of
['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Click mushroom which is used in this tutorial for resnet_imagenet and airplane for resnet-cifar. Upload the pic, if using terminal, either scp or wget will do, if running in Jupyter, click Upload button at the top right and select the picture. Save the picture at the root
folder, rename to mushroom.jpeg(for resnet-imagenet) or airplane.jpg(for resnet-cifar).
Or run this code to download mushroom pic for resnet_imagenet and airplane for resnet_cifar:
[7]:
# download mushroom pic
if not os.path.exists('/root/mushroom.jpeg'):
!wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/mushrooms/mushroom.jpeg
else:
print('mushroom.jpeg already exists')
# download airplane pic
if not os.path.exists('/root/airplane.jpg'):
!wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/objects/airplane.jpg
else:
print('airplane.jpg already exists')
--2021-03-19 15:24:11-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/mushrooms/mushroom.jpeg
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 121.36.121.44, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 76020 (74K) [image/jpeg]
Saving to: ‘/root/mushroom.jpeg’
mushroom.jpeg 100%[===================>] 74.24K 370KB/s in 0.2s
2021-03-19 15:24:12 (370 KB/s) - ‘/root/mushroom.jpeg’ saved [76020/76020]
--2021-03-19 15:24:12-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/objects/airplane.jpg
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 121.36.121.44, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 151188 (148K) [image/jpeg]
Saving to: ‘/root/airplane.jpg’
airplane.jpg 100%[===================>] 147.64K 561KB/s in 0.3s
2021-03-19 15:24:12 (561 KB/s) - ‘/root/airplane.jpg’ saved [151188/151188]
6.2 List servables¶
Now, use list_servables function to check what model is servable right now.
[8]:
list_servables()
[8]:
[{'description': 'This servable hosts a resnet50 model predicting mushrooms',
'model': {'class_num': 9, 'format': 'ckpt', 'name': 'resnet50'},
'name': 'resnet50_imagenet2012'}]
If the output description shows it is a resnet50 model, run the following code which will automatically detect whether it is a imagenet model or a cifar model
6.3 Sending request and get the result¶
Run predict function to send the request, select between TOP1_CLASS and TOP5_CLASS to check the output
[9]:
# set image_path and output strategy
imagenet_image_path = "/root/mushroom.jpeg"
cifar_image_path = "/root/airplane.jpg"
strategy = "TOP1_CLASS"
# predict(image_path, servable_name, dataset_name, strategy='TOP1_CLASS')
if server_started() is True:
servable_name = list_servables()[0]['name']
if servable_name == 'resnet50_imagenet2012':
img_viewer = ImageViewer(Image.open(imagenet_image_path), imagenet_image_path)
img_viewer.show()
print(predict(imagenet_image_path, "resnet50_imagenet2012", "imagenet2012", strategy))
else:
img_viewer = ImageViewer(Image.open(cifar_image_path), cifar_image_path)
img_viewer.show()
print(predict(cifar_image_path, "resnet50_cifar10", 'cifar10', strategy))
else:
print('Server not started')
TOP1: Amanita毒蝇伞,伞菌目,鹅膏菌科,鹅膏菌属,主要分布于我国黑龙江、吉林、四川、西藏、云南等地,有毒, score: 0.99750286340713500977
Check output¶
If user runs resnet_imagenet and see output similar to this:
TOP1: Amanita毒蝇伞,伞菌目,鹅膏菌科,鹅膏菌属,主要分布于我国黑龙江、吉林、四川、西藏、云南等地,有毒, score: 0.99119007587432861328
that means the prediction result is returned and the inference is completed.
Or user runs resnet_cifar, and the output is expected to be like this:
TOP1: airplane, score: 0.99997282028198242188
## Change model
Run the following code to train a ResNet50 model with ImageNet2012 mushroom dataset, then run another servable_json code block to define servable. The dataset will be downloaded here.
[ ]:
# download the imagenet2012 mushroom dataset
imagenet_path = '/root/mushrooms'
if not os.path.exists(imagenet_path):
!wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/resnet-50/mushrooms/mushrooms.zip
!mkdir /root/mushrooms/
!unzip /root/mushrooms.zip -d /root/mushrooms/
print('************Download complete*************')
else:
print('************Dataset already exists.**************')
# check ckpt folder exists or not
imagenet_ckpt_folder = '/etc/tinyms/serving/resnet50_imagenet2012'
imagenet_ckpt_path = '/etc/tinyms/serving/resnet50_imagenet2012/resnet50.ckpt'
if not os.path.exists(imagenet_ckpt_folder):
!mkdir -p /etc/tinyms/serving/resnet50_imagenet2012
else:
print('resnet50_imagenet2012 ckpt folder already exists')
epoch_size = 90 # default is 90
batch_size = 32
# set environment parameters
dataset_sink_mode = False
device_target = "CPU"
context.set_context(mode=context.GRAPH_MODE, device_target=device_target)
# set dataset parameters
imagenet_train_path = '/root/mushrooms/train'
train_dataset = ImageFolderDataset(imagenet_train_path, num_parallel_workers=4, shuffle=True)
train_dataset = imagefolder_transform.apply_ds(train_dataset, repeat_size=1, batch_size=32, is_training=True)
imagenet_eval_path = '/root/mushrooms/eval'
eval_dataset = ImageFolderDataset(imagenet_eval_path, num_parallel_workers=4, shuffle=True)
eval_dataset = imagefolder_transform.apply_ds(eval_dataset, repeat_size=1, batch_size=32, is_training=False)
step_size = train_dataset.get_dataset_size()
save_checkpoint_epochs = 5
ckpoint_cb = ModelCheckpoint(prefix="resnet_imagenet2012", config=CheckpointConfig(
save_checkpoint_steps=save_checkpoint_epochs * train_dataset.get_dataset_size(),
keep_checkpoint_max=10))
# define the loss function
net_loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean")
# define the optimizer
net_opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), 0.01, 0.9)
model.compile(loss_fn=net_loss, optimizer=net_opt, metrics={"Accuracy": Accuracy()})
print('************************Start training*************************')
model.train(epoch_size, train_dataset, callbacks=[ckpoint_cb, LossMonitor()],dataset_sink_mode=dataset_sink_mode)
model.save_checkpoint(imagenet_ckpt_path)
print('************************Finished training*************************')
model.load_checkpoint(imagenet_ckpt_path)
print('************************Start evaluation*************************')
acc = model.eval(eval_dataset, dataset_sink_mode=dataset_sink_mode)
print("============== Accuracy:{} ==============".format(acc))
TinyMS MobileNetV2 Tutorial¶
In this tutorial, using TinyMS API to train/serve a MobileNetV2 model will be demonstrated.
Prerequisite¶
Ubuntu:
18.04Python:
3.7.xFlask:
1.1.2MindSpore:
CPU-1.1.1TinyMS:
0.1.0numpy:
1.17.5Pillow:
8.1.0pip:
21.0.1requests:
2.18.4
Introduction¶
TinyMS is a high-level API which is designed for amateur of deep learning. It minimizes the number of actions of users required to construct, train, evaluate and serve a model. TinyMS also provides tutorials and documentations for developers.
This tutorial consists of six parts, constructing the model, downloading dataset, training, define servable json, starting server and making predictions in which the server will be run in a sub process.
[1]:
import os
import json
from PIL import Image
from tinyms import context
from tinyms.serving import start_server, predict, list_servables, shutdown, server_started
from tinyms.model import Model, mobilenetv2
from tinyms.data import Cifar10Dataset, download_dataset
from tinyms.vision import cifar10_transform, ImageViewer
from tinyms.metrics import Accuracy
from tinyms.optimizers import Momentum
from tinyms.losses import CrossEntropyWithLabelSmooth
from tinyms.utils.train.loss_manager import FixedLossScaleManager
from tinyms.utils.train.lr_generator import mobilenetv2_lr
from tinyms.utils.train.cb_config import mobilenetv2_cb
[WARNING] ME(8895:140407127349056,MainProcess):2021-03-19-15:03:40.515.970 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
1. Construct the model¶
TinyMS encapsulates init and construct of the MobileNetV2 model, the line of the code is reduced to construct the model:
[2]:
# build the model
net = mobilenetv2(class_num=10, is_training=True)
model = Model(net)
2. Download dataset¶
The cifar10 dataset will be downloaded if cifar10 folder didn’t exist at the root. If cifar10 folder already exists, this step will not be performed.
[3]:
# download the dataset
cifar10_path = '/root/cifar10/cifar-10-batches-bin'
if not os.path.exists(cifar10_path):
download_dataset('cifar10', '/root')
print('************Download complete*************')
else:
print('************Dataset already exists.**************')
************** Downloading the Cifar10 dataset **************
[████████████████████████████████████████████████████████████████████████████████████████████████████] 100.00%
============== /root/cifar10/cifar-10-binary.tar.gz is ready ==============
************Download complete*************
3. Train the model & evaluation¶
The dataset for both training and evaluation will be defined here, and the parameters for training also set in this block. A trained ckpt file will be saved to /etc/tinyms/serving/mobilenetv2 folder for later use, meanwhile the evaluation will be performed and the Accuracy can be checked
Notice: Since training MobileNetV2 on CPU is time consuming, we recommend skip training and using provided ckpt files to run.
[ ]:
# check ckpt folder exists or not
ckpt_folder = '/etc/tinyms/serving/mobilenetv2'
ckpt_path = '/etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt'
if not os.path.exists(ckpt_folder):
!mkdir -p /etc/tinyms/serving/mobilenetv2
else:
print('mobilenetv2 ckpt folder already exists')
# Declare common variables
epoch_size = 60 # default is 60
batch_size = 32
class_num = 10
# set runtime environment
device_target="CPU"
dataset_sink_mode = False
context.set_context(mode=context.GRAPH_MODE, device_target=device_target)
# create cifar10 dataset
train_dataset = Cifar10Dataset(cifar10_path, num_parallel_workers=4, shuffle=True)
train_dataset = cifar10_transform.apply_ds(train_dataset, repeat_size=1, batch_size=32, is_training=True)
eval_dataset = Cifar10Dataset(cifar10_path, num_parallel_workers=4, shuffle=True)
eval_dataset = cifar10_transform.apply_ds(eval_dataset, repeat_size=1, batch_size=32, is_training=False)
step_size = train_dataset.get_dataset_size()
# define the loss function
label_smooth = 0.1
loss = CrossEntropyWithLabelSmooth(smooth_factor=label_smooth,num_classes=class_num)
# get learning rate
lr_max = 0.001
lr_init_scale = 0.01
lr_end_scale = 0.01
lr = mobilenetv2_lr(global_step=0,
lr_init=lr_max*lr_init_scale,
lr_end=lr_max*lr_end_scale,
lr_max=lr_max,
warmup_epochs=2,
total_epochs=epoch_size,
steps_per_epoch=step_size)
# define the optimizer
loss_scale = FixedLossScaleManager(1024, drop_overflow_update=False)
opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()),lr, 0.9, 4e-5, 1024)
model.compile(loss_fn=loss, optimizer=opt, metrics={"Accuracy": Accuracy()},loss_scale_manager=loss_scale)
# configure checkpoint to save weights and do training job
save_checkpoint_epochs = 10
ckpoint_cb = mobilenetv2_cb(device_target=device_target,
lr=lr,
is_saving_checkpoint=True,
save_checkpoint_epochs=save_checkpoint_epochs,
step_size=step_size)
print('************************Start training*************************')
model.train(epoch_size, train_dataset, callbacks=ckpoint_cb, dataset_sink_mode=dataset_sink_mode)
model.save_checkpoint(ckpt_path)
print('************************Finished training*************************')
model.load_checkpoint(ckpt_path)
print('************************Start evaluation*************************')
acc = model.eval(eval_dataset, dataset_sink_mode=dataset_sink_mode)
print("============== Accuracy:{} ==============".format(acc))
Notice: If skipped training process, download the pretrained ckpt file and continue to serving
Click HERE to download MobileNetV2 ckpt file and save this file to /etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt.
Or run the following code to download and store the ckpt file:
[4]:
mobilenetv2_ckpt_folder = '/etc/tinyms/serving/mobilenetv2'
mobilenetv2_ckpt_path = '/etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt'
if not os.path.exists(mobilenetv2_ckpt_folder):
!mkdir -p /etc/tinyms/serving/mobilenetv2
!wget -P /etc/tinyms/serving/mobilenetv2 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/mobilenetv2.ckpt
else:
print('mobilenetv2 ckpt folder already exists')
if not os.path.exists(mobilenetv2_ckpt_path):
!wget -P /etc/tinyms/serving/mobilenetv2 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/mobilenetv2.ckpt
else:
print('mobilenetv2 ckpt file already exists')
mobilenetv2 ckpt folder already exists
--2021-03-19 15:06:26-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/mobilenetv2.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.5, 49.4.112.90, 49.4.112.113, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.5|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 18509001 (18M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt’
mobilenetv2.ckpt 100%[===================>] 17.65M 16.1MB/s in 1.1s
2021-03-19 15:06:29 (16.1 MB/s) - ‘/etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt’ saved [18509001/18509001]
4. Define servable.json¶
Define the MobileNetV2 servable json file for model name, format and number of classes for later use.
[5]:
servable_json = [{'name': 'mobilenetv2',
'description': 'This servable hosts a mobilenetv2 model predicting 10 classes of objects',
'model': {
"name": "mobilenetv2",
"format": "ckpt",
"class_num": 10}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)
with open('servable.json', 'w') as json_file:
json_file.write(json_data)
5. Start server¶
5.1 Introduction¶
TinyMS Serving is a C/S(client/server) structure. There is a server and client. TinyMS using Flask which is a micro web framework written in python as the C/S communication tool. In order to serve a model, user must start server first. If successfully started, the server will be run in a subprocess and listening to POST requests from 127.0.0.1 port 5000 sent by client and handle the requests using MindSpore backend which will construct the model, run the prediction and send the result back to the client.
5.2 start server¶
Run the following code block to start the server:
[6]:
start_server()
Server starts at host 127.0.0.1, port 5000
6. Make predictions¶
6.1 Upload the pic¶
A picture is required to be the input. This ckpt requires a picture containing the following objects:
['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Click airplane to download the picture used in this tutorial. Upload the pic, if using terminal, either scp or wget will do, if running in Jupyter, click Upload button at the top right and select the picture.
Save the picture at the root folder, rename to airplane.jpg(or any name you want).
Or run the following code to download the picture used in this tutorial
[7]:
# download the airplane pic
if not os.path.exists('/root/airplane.jpg'):
!wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/objects/airplane.jpg
else:
print('airplane.jpg already exists')
--2021-03-19 15:06:37-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/objects/airplane.jpg
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.5, 49.4.112.90, 49.4.112.113, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.5|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 151188 (148K) [image/jpeg]
Saving to: ‘/root/airplane.jpg’
airplane.jpg 100%[===================>] 147.64K 560KB/s in 0.3s
2021-03-19 15:06:38 (560 KB/s) - ‘/root/airplane.jpg’ saved [151188/151188]
6.2 List servables¶
Now, we can use list_servables function to check what model is servable right now.
[8]:
list_servables()
[8]:
[{'description': 'This servable hosts a mobilenetv2 model predicting 10 classes of objects',
'model': {'class_num': 10, 'format': 'ckpt', 'name': 'mobilenetv2'},
'name': 'mobilenetv2'}]
If the output description shows it is a MobileNetV2 model, then we can continue to next step to send our request.
6.3 Sending request and get the result¶
Run predict function to send the request, select between TOP1_CLASS and TOP5_CLASS to check the output:
[9]:
# set image path and output strategy(TOP1_CLASS or TOP5_CLASS)
image_path = "/root/airplane.jpg"
strategy = "TOP1_CLASS"
# predict(image_path, servable_name, dataset_name, strategy)
if server_started() is True:
img_viewer = ImageViewer(Image.open(image_path), image_path)
img_viewer.show()
print(predict(image_path, 'mobilenetv2', 'cifar10', strategy))
else:
print("Server not started")
TOP1: airplane, score: 0.22268821299076080322
Check output¶
If user can see output similar to this:
TOP1: airplane, score: 0.22268821299076080322
that means the prediction result is returned and the inference is completed.
## Shutdown server
To restart and try another checkpoint file, click Kernel at the top, then Restart & Clear Output, replace the servable_json code and predict() function
Run the following code to shutdown Flask server:
[10]:
shutdown()
[10]:
'Server shutting down...'
TinyMS SSD300 Tutorial¶
In this tutorial, using TinyMS API to train/serve an SSD300 model will be demonstrated.
Prerequisite¶
Ubuntu:
18.04Python:
3.7.xFlask:
1.1.2MindSpore:
CPU-1.1.1TinyMS:
0.1.0numpy:
1.17.5Pillow:
8.1.0pip:
21.0.1requests:
2.18.4
Introduction¶
TinyMS is a high-level API which is designed for amateur of deep learning. It minimizes the number of actions of users required to construct, train, evaluate and serve a model. TinyMS also provides tutorials and documentations for developers.
This tutorial consists of six parts, constructing the model, downloading dataset, training, define servable json, starting server and making predictions in which the server will be run in a sub process.
[1]:
import os
import json
import time
import tinyms as ts
import xml.etree.ElementTree as et
from PIL import Image
from tinyms import context, layers, primitives as P, Tensor
from tinyms.serving import start_server, predict, list_servables, shutdown, server_started
from tinyms.data import VOCDataset, download_dataset
from tinyms.vision import voc_transform, coco_eval, ImageViewer
from tinyms.model import Model, ssd300_mobilenetv2, ssd300_infer
from tinyms.losses import net_with_loss
from tinyms.optimizers import Momentum
from tinyms.callbacks import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor
from tinyms.utils.train.lr_generator import mobilenetv2_lr as ssd300_lr
from tinyms.initializers import initializer, TruncatedNormal
[WARNING] ME(14174:140126738016064,MainProcess):2021-03-19-15:33:42.136.028 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
1. Construct the model¶
[2]:
# build network
net = ssd300_mobilenetv2(class_num=21)
2. Download dataset¶
The VOC dataset will be downloaded if voc folder didn’t exist at the root. If voc folder already exists, this step will not be performed.
[3]:
# download the dataset
voc_path = '/root/voc'
if not os.path.exists(voc_path):
download_dataset('voc', '/root')
print('************Download complete*************')
else:
print('************Dataset already exists.**************')
************** Downloading the VOC2007 dataset **************
[████████████████████████████████████████████████████████████████████████████████████████████████████] 100.00%
============== /root/voc/VOCtrainval_06-Nov-2007.tar is ready ==============
************Download complete*************
3. Train the model & evaluation¶
The dataset for both training and evaluation will be defined here, and the parameters for training also set in this block. A trained ckpt file will be saved to /etc/tinyms/serving/ssd300 folder for later use, meanwhile the evaluation will be performed and the Accuracy can be checked
Notice: Since training SSD300 on CPU is time consuming, we recommend skip training and using provided ckpt files to run.
[ ]:
class TrainingWrapper(layers.Layer):
"""
Encapsulation class of SSD300 network training.
Append an optimizer to the training network after that the construct
function can be called to create the backward graph.
Args:
network (Layer): The training network. Note that loss function should have been added.
optimizer (Optimizer): Optimizer for updating the weights.
sens (float): The adjust parameter. Default: 1.0.
"""
def __init__(self, network, optimizer, sens=1.0):
super(TrainingWrapper, self).__init__(auto_prefix=False)
self.network = network
self.network.set_grad()
self.weights = ts.ParameterTuple(network.trainable_params())
self.optimizer = optimizer
self.grad = P.GradOperation(get_by_list=True, sens_param=True)
self.sens = sens
self.hyper_map = P.HyperMap()
def construct(self, *args):
weights = self.weights
loss = self.network(*args)
sens = P.Fill()(P.DType()(loss), P.Shape()(loss), self.sens)
grads = self.grad(self.network, weights)(*args, sens)
return P.depend(loss, self.optimizer(grads))
def create_voc_label(voc_dir, voc_cls, usage='val'):
"""Get image path and annotation from VOC."""
if not os.path.isdir(voc_dir):
raise ValueError(f'Cannot find {voc_dir} dataset path.')
anno_dir = voc_dir
if os.path.isdir(os.path.join(voc_dir, 'Annotations')):
anno_dir = os.path.join(voc_dir, 'Annotations')
cls_map = {name: i for i, name in enumerate(voc_cls)}
# Fetch the specific xml files path
xml_files = []
with open(os.path.join(voc_dir, 'ImageSets', 'Main', usage+'.txt'), 'r') as f:
for line in f:
xml_files.append(line.strip('\n')+'.xml')
json_dict = {"images": [], "type": "instances", "annotations": [],
"categories": []}
bnd_id = 1
for xml_file in xml_files:
img_id = xml_files.index(xml_file)
tree = et.parse(os.path.join(anno_dir, xml_file))
root_node = tree.getroot()
file_name = root_node.find('filename').text
for obj in root_node.iter('object'):
cls_name = obj.find('name').text
if cls_name not in cls_map:
print(f'Label "{cls_name}" not in "{cls_map}"')
continue
bnd_box = obj.find('bndbox')
x_min = int(float(bnd_box.find('xmin').text)) - 1
y_min = int(float(bnd_box.find('ymin').text)) - 1
x_max = int(float(bnd_box.find('xmax').text)) - 1
y_max = int(float(bnd_box.find('ymax').text)) - 1
o_width = abs(x_max - x_min)
o_height = abs(y_max - y_min)
ann = {'area': o_width * o_height, 'iscrowd': 0,
'image_id': img_id,
'bbox': [x_min, y_min, o_width, o_height],
'category_id': cls_map[cls_name], 'id': bnd_id,
'ignore': 0,
'segmentation': []}
json_dict['annotations'].append(ann)
bnd_id = bnd_id + 1
size = root_node.find("size")
width = int(size.find('width').text)
height = int(size.find('height').text)
image = {'file_name': file_name, 'height': height, 'width': width,
'id': img_id}
json_dict['images'].append(image)
for cls_name, cid in cls_map.items():
cat = {'supercategory': 'none', 'id': cid, 'name': cls_name}
json_dict['categories'].append(cat)
anno_file = os.path.join(anno_dir, 'annotation.json')
with open(anno_file, 'w') as f:
json.dump(json_dict, f)
return anno_file
# check ckpt folder exists or not
ckpt_folder = '/etc/tinyms/serving/ssd300'
ckpt_path = '/etc/tinyms/serving/ssd300/ssd300.ckpt'
if not os.path.exists(ckpt_folder):
!mkdir -p /etc/tinyms/serving/ssd300
else:
print('ssd300 ckpt folder already exists')
# set parameters
epoch_size = 800 # default is 800
batch_size = 32
voc_path = '/root/voc/VOCdevkit/VOC2007'
# set environment parameters
context.set_context(mode=context.GRAPH_MODE, device_target="CPU")
dataset_sink_mode = False
# set dataset parameters
train_dataset = VOCDataset(voc_path, task='Detection', usage='trainval', num_parallel_workers=4, shuffle=True, decode=True)
train_dataset = voc_transform.apply_ds(train_dataset, repeat_size=1, batch_size=batch_size, num_parallel_workers=4, is_training=True)
eval_dataset = VOCDataset(voc_path, task='Detection', usage='val', num_parallel_workers=4, shuffle=True, decode=True)
eval_dataset = voc_transform.apply_ds(eval_dataset, repeat_size=1, batch_size=batch_size, num_parallel_workers=4, is_training=False)
dataset_size = train_dataset.get_dataset_size()
total = eval_dataset.get_dataset_size()
# define the loss function
net = net_with_loss(net)
params = net.trainable_params()
for p in params:
if 'beta' not in p.name and 'gamma' not in p.name and 'bias' not in p.name:
p.set_data(initializer(TruncatedNormal(0.02), p.data.shape, p.data.dtype))
# define the optimizer
pre_trained_epoch_size = 0
save_checkpoint_epochs = 10
lr = 0.01
lr = ssd300_lr(global_step=pre_trained_epoch_size * dataset_size,
lr_init=0.001, lr_end=0.001 * lr, lr_max=lr,
warmup_epochs=2, total_epochs=epoch_size,
steps_per_epoch=dataset_size)
loss_scale = 1.0
opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), lr,0.9, 1.5e-4, loss_scale)
model = Model(TrainingWrapper(net, opt, loss_scale))
model.compile()
ckpoint_cb = ModelCheckpoint(prefix="ssd300", config=CheckpointConfig(
save_checkpoint_steps=save_checkpoint_epochs * dataset_size,
keep_checkpoint_max=10))
print('************************Start training*************************')
model.train(epoch_size, train_dataset, callbacks=[ckpoint_cb, LossMonitor(), TimeMonitor(data_size=dataset_size)],
dataset_sink_mode=dataset_sink_mode)
model.save_checkpoint(ckpt_path)
print('************************Finished training*************************')
eval_net = ssd300_infer(class_num=21)
model = Model(eval_net)
model.load_checkpoint(ckpt_path)
# perform the model predict operation
print("\n========================================\n")
print("total images num: ", total)
print("Processing, please wait a moment...")
start = time.time()
pred_data = []
id_iter = 0
for data in eval_dataset.create_dict_iterator(output_numpy=True):
image_np = data['image']
image_shape = data['image_shape']
output = model.predict(Tensor(image_np))
for batch_idx in range(image_np.shape[0]):
pred_data.append({"boxes": output[0].asnumpy()[batch_idx],
"box_scores": output[1].asnumpy()[batch_idx],
"img_id": id_iter,
"image_shape": image_shape[batch_idx]})
id_iter += 1
cost_time = int((time.time() - start) * 1000)
print(f' 100% [{total}/{total}] cost {cost_time} ms')
# calculate mAP for the predict data
voc_cls = ['background',
'aeroplane', 'bicycle', 'bird', 'boat', 'bottle',
'bus', 'car', 'cat', 'chair', 'cow',
'diningtable', 'dog', 'horse', 'motorbike', 'person',
'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor']
anno_file = create_voc_label(voc_path, voc_cls)
mAP = coco_eval(pred_data, anno_file)
print("\n========================================\n")
print(f"mAP: {mAP}")
Notice: If skipped training process, download the pretrained ckpt file and continue to serving
Click HERE to download the ckpt file and save this file to /etc/tinyms/serving/ssd300/ssd300.ckpt.
Or run the following code to download and store the ckpt file:
[4]:
ssd300_ckpt_folder = '/etc/tinyms/serving/ssd300'
ssd300_ckpt_path = '/etc/tinyms/serving/ssd300/ssd300.ckpt'
if not os.path.exists(ssd300_ckpt_folder):
!mkdir -p /etc/tinyms/serving/ssd300
!wget -P /etc/tinyms/serving/ssd300 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/voc/ssd300.ckpt
else:
print('ssd300 ckpt folder already exists')
if not os.path.exists(ssd300_ckpt_path):
!wget -P /etc/tinyms/serving/ssd300 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/voc/ssd300.ckpt
else:
print('ssd300 ckpt file already exists')
ssd300 ckpt folder already exists
--2021-03-19 15:38:53-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/voc/ssd300.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 49.4.112.90, 121.36.121.44, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 28056511 (27M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/ssd300/ssd300.ckpt’
ssd300.ckpt 100%[===================>] 26.76M 20.7MB/s in 1.3s
2021-03-19 15:38:55 (20.7 MB/s) - ‘/etc/tinyms/serving/ssd300/ssd300.ckpt’ saved [28056511/28056511]
4. Define servable.json¶
Run this code to define the servable json file for later use:
[5]:
servable_json = [{'name': 'ssd300',
'description': 'This servable hosts an ssd300 model predicting bounding boxes',
'model': {
"name": "ssd300",
"format": "ckpt",
"class_num": 21}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)
with open('servable.json', 'w') as json_file:
json_file.write(json_data)
5. Start server¶
5.1 Introduction¶
TinyMS Serving is a C/S(client/server) structure. There is a server and client. TinyMS using Flask which is a micro web framework written in python as the C/S communication tool. In order to serve a model, user must start server first. If successfully started, the server will be run in a subprocess and listening to POST requests from 127.0.0.1 port 5000 sent by client and handle the requests using MindSpore backend which will construct the model, run the prediction and send the result back to the client.
5.2 start server¶
Run the following code block to start the server:
[6]:
start_server()
Server starts at host 127.0.0.1, port 5000
6. Make predictions¶
6.1 Upload the pic¶
A picture is required to be the input. In this tutorial, the SSD300 ckpt requires a picture containing objects of
['background', 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 'bus', 'car', 'cat', 'chair', 'cow', 'diningtable', 'dog', 'horse', 'motorbike', 'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor']
Click HERE to download the picture which is used in this tutorial. Upload the pic, if using terminal, either scp or wget will do, if running in Jupyter, click Upload button at the top right and select the picture. Save the picture at the root folder, rename to ssd300_test.jpeg(or any name you want).
Or run the following code to download the picture used in this tutorial:
[7]:
# download the test pic
if not os.path.exists('/root/ssd300_test.jpeg'):
!wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/ssd300_test/ssd300_test.jpeg
else:
print('ssd300_test.jpeg already exists')
--2021-03-19 15:38:59-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/ssd300_test/ssd300_test.jpeg
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 49.4.112.90, 121.36.121.44, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 70412 (69K) [image/jpeg]
Saving to: ‘/root/ssd300_test.jpeg’
ssd300_test.jpeg 100%[===================>] 68.76K 338KB/s in 0.2s
2021-03-19 15:39:00 (338 KB/s) - ‘/root/ssd300_test.jpeg’ saved [70412/70412]
6.2 List servables¶
Now, use list_servables function to check what model is servable right now.
[8]:
list_servables()
[8]:
[{'description': 'This servable hosts an ssd300 model predicting bounding boxes',
'model': {'class_num': 21, 'format': 'ckpt', 'name': 'ssd300'},
'name': 'ssd300'}]
If the output description shows it is an ssd300 model, run the following code to predict the bounding boxes
6.3 Sending request and get the result¶
Run predict function and get the result, right now only TOP1CLASS strategy is supported for SSD300. Call ImageViewer.draw to draw the bounding boxes
[9]:
# set image path and output strategy(only TOP1_CLASS)
image_path = "/root/ssd300_test.jpeg"
strategy = "TOP1_CLASS"
labels = ['background',
'aeroplane', 'bicycle', 'bird', 'boat', 'bottle',
'bus', 'car', 'cat', 'chair', 'cow',
'diningtable', 'dog', 'horse', 'motorbike', 'person',
'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor']
# predict(image_path, servable_name, dataset_name, strategy)
# ImageViewer(img, title)
# ImageViewer.draw(predict_result, labels)
if server_started() is True:
res = predict(image_path, 'ssd300', 'voc', strategy)
img_viewer = ImageViewer(Image.open(image_path))
img_viewer.draw(res, labels)
else:
print("Server not started")
Check output¶
If the input picture is shown with bounding boxes labeled with object classes and score, that means the prediction is successfully performed.
TinyMS CycleGAN Tutorial¶
In this tutorial, using TinyMS API to train/serve a CycleGAN model will be demonstrated.
Prerequisite¶
Ubuntu:
18.04Python:
3.7.xFlask:
1.1.2MindSpore:
CPU-1.1.1TinyMS:
0.1.0numpy:
1.17.5Pillow:
8.1.0pip:
21.0.1requests:
2.18.4matplotlib:
3.3.4
Introduction¶
TinyMS is a high-level API which is designed for amateur of deep learning. It minimizes the number of actions of users required to construct, train, evaluate and serve a model. TinyMS also provides tutorials and documentations for developers.
This tutorial consists of five parts, downloading dataset, training, define servable json, starting server and making predictions in which the server will be run in a sub process.
[1]:
import os
import argparse
import json
import tinyms as ts
import numpy as np
import matplotlib.pyplot as plt
from PIL import Image
from tinyms import context, Tensor
from tinyms.serving import start_server, predict, list_servables, shutdown, server_started
from tinyms.data import GeneratorDataset, UnalignedDataset, GanImageFolderDataset, DistributedSampler
from tinyms.vision import cyclegan_transform
from tinyms.model.cycle_gan.cycle_gan import get_generator_discriminator, cycle_gan, TrainOneStepG, TrainOneStepD
from tinyms.losses import CycleGANDiscriminatorLoss, CycleGANGeneratorLoss
from tinyms.optimizers import Adam
from tinyms.data.utils import save_image, generate_image_list
from tinyms.utils.common_utils import GanReporter, gan_load_ckpt, GanImagePool
from tinyms.utils.train import cyclegan_lr
from tinyms.utils.eval import CityScapes, fast_hist, get_scores
[WARNING] ME(25552:140556571072320,MainProcess):2021-03-21-15:01:26.554.568 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
1. Download dataset¶
In this tutorial, the cityscapes dataset is used and processed. Click the link before proceeding to the official website to submit and download the dataset.
2. Train the model & evaluation¶
Define parameters and training process
Notice: Training on CPU is time consuming, we recommend skip training and using provided ckpt files to run.
[ ]:
def parse_args():
parser = argparse.ArgumentParser(description='MindSpore Cycle GAN Example')
parser.add_argument('--device_target', type=str, default="CPU", choices=['Ascend', 'GPU', 'CPU'],
help='device where the code will be implemented (default: CPU)')
parser.add_argument('--dataset_path', type=str, default="/root/dataset/cityscapes", help='cityscape dataset path.')
parser.add_argument('--phase', type=str, default="train", help='train, eval or predict.')
parser.add_argument('--model', type=str, default="resnet", choices=("resnet", "unet"),
help='generator model, should be in [resnet, unet].')
parser.add_argument('--max_epoch', type=int, default=200, help='epoch size for training, default is 200.')
parser.add_argument('--n_epoch', type=int, default=100,
help='number of epochs with the initial learning rate, default is 100')
parser.add_argument('--batch_size', type=int, default=1, help='Batch size.')
parser.add_argument("--save_checkpoint_epochs", type=int, default=10,
help="Save checkpoint epochs, default is 10.")
parser.add_argument("--G_A_ckpt", type=str, default="/etc/tinyms/serving/cyclegan_cityscape/G_A.ckpt", help="pretrained checkpoint file path of G_A.")
parser.add_argument("--G_B_ckpt", type=str, default="/etc/tinyms/serving/cyclegan_cityscape/G_B.ckpt", help="pretrained checkpoint file path of G_B.")
parser.add_argument("--D_A_ckpt", type=str, default=None, help="pretrained checkpoint file path of D_A.")
parser.add_argument("--D_B_ckpt", type=str, default=None, help="pretrained checkpoint file path of D_B.")
parser.add_argument('--outputs_dir', type=str, default='/root/',
help='models are saved here, default is ./outputs.')
parser.add_argument('--save_imgs', type=bool, default=True,
help='whether save imgs when epoch end, default is True.')
parser.add_argument("--cityscapes_dir", type=str, default="/root/dataset/cityscapes/testA", help="Path to the original cityscapes dataset")
parser.add_argument("--result_dir", type=str, default="/root/dataset/cityscapes/testB", help="Path to the generated images to be evaluated")
args_opt = parser.parse_args(args=[])
return args_opt
args_opt = parse_args()
context.set_context(mode=context.GRAPH_MODE, device_target="CPU")
dataset_path = args_opt.dataset_path
phase = args_opt.phase
G_A_ckpt = args_opt.G_A_ckpt
G_B_ckpt = args_opt.G_B_ckpt
repeat_size = 1
model = args_opt.model
batch_size = args_opt.batch_size
max_dataset_size = float("inf")
outputs_dir = args_opt.outputs_dir
max_epoch = args_opt.max_epoch
n_epoch = args_opt.n_epoch
n_epoch = min(max_epoch, n_epoch)
def create_dataset(dataset_path, batch_size=1, repeat_size=1, max_dataset_size=None,
shuffle=True, num_parallel_workers=1, phase='train', data_dir='testA'):
""" create Mnist dataset for train or eval.
Args:
data_path: Data path
batch_size: The number of data records in each group
repeat_size: The number of replicated data records
num_parallel_workers: The number of parallel workers
"""
# define dataset and apply the transform func
if phase == 'train':
ds = UnalignedDataset(dataset_path, phase, max_dataset_size=max_dataset_size, shuffle=True)
device_num = 1
distributed_sampler = DistributedSampler(len(ds), num_replicas=device_num, rank=0, shuffle=shuffle)
gan_generator_ds = GeneratorDataset(ds, column_names=["image_A", "image_B"], sampler=distributed_sampler,
num_parallel_workers=num_parallel_workers)
else:
datadir = os.path.join(dataset_path, data_dir)
ds = GanImageFolderDataset(datadir, max_dataset_size=max_dataset_size)
gan_generator_ds = GeneratorDataset(ds, column_names=["image", "image_name"],
num_parallel_workers=num_parallel_workers)
gan_generator_ds = cyclegan_transform.apply_ds(gan_generator_ds,
repeat_size=repeat_size,
batch_size=batch_size,
num_parallel_workers=num_parallel_workers,
shuffle=shuffle,
phase=phase)
dataset_size = len(ds)
return gan_generator_ds, dataset_size
# create dataset
dataset, args_opt.dataset_size = create_dataset(dataset_path, batch_size=batch_size, repeat_size=1,
max_dataset_size=max_dataset_size, shuffle=True,
num_parallel_workers=1,
phase="train",
data_dir=None)
G_A, G_B, D_A, D_B = get_generator_discriminator(model)
gan_load_ckpt(args_opt.G_A_ckpt, args_opt.G_B_ckpt, args_opt.D_A_ckpt, args_opt.D_B_ckpt,
G_A, G_B, D_A, D_B)
generator_net = cycle_gan(G_A, G_B)
# define loss function and optimizer
loss_D = CycleGANDiscriminatorLoss(D_A, D_B)
loss_G = CycleGANGeneratorLoss(generator_net, D_A, D_B)
lr = cyclegan_lr(max_epoch, n_epoch, args_opt.dataset_size)
optimizer_G = Adam(generator_net.trainable_params(),
cyclegan_lr(max_epoch, n_epoch, args_opt.dataset_size), beta1=0.5)
optimizer_D = Adam(loss_D.trainable_params(),
cyclegan_lr(max_epoch, n_epoch, args_opt.dataset_size), beta1=0.5)
# build two net: generator net and descriminator net
net_G = TrainOneStepG(loss_G, generator_net, optimizer_G)
net_D = TrainOneStepD(loss_D, optimizer_D)
# train process
imgae_pool_A = GanImagePool(pool_size=50)
imgae_pool_B = GanImagePool(pool_size=50)
def train_process(args_opt, data_loader, net_G, net_D, imgae_pool_A, imgae_pool_B):
reporter = GanReporter(args_opt)
reporter.info('==========start training===============')
for _ in range(max_epoch):
reporter.epoch_start()
for data in data_loader:
img_A = data["image_A"]
img_B = data["image_B"]
res_G = net_G(img_A, img_B)
fake_A = res_G[0]
fake_B = res_G[1]
res_D = net_D(img_A, img_B, imgae_pool_A.query(fake_A), imgae_pool_B.query(fake_B))
reporter.step_end(res_G, res_D)
reporter.visualizer(img_A, img_B, fake_A, fake_B)
reporter.epoch_end(net_G)
reporter.info('==========end training===============')
data_loader = dataset.create_dict_iterator()
train_process(args_opt, data_loader, net_G, net_D, imgae_pool_A, imgae_pool_B)
# eval
# original image dir
cityscapes_dir = args_opt.cityscapes_dir
# fake image dir generated after predict
result_dir = args_opt.result_dir
def eval_process(args_opt, cityscapes_dir, result_dir):
CS = CityScapes()
hist_perframe = ts.zeros((CS.class_num, CS.class_num)).asnumpy()
cityscapes = generate_image_list(cityscapes_dir)
args_opt.dataset_size = len(cityscapes)
reporter = GanReporter(args_opt)
reporter.start_eval()
for i, img_path in enumerate(cityscapes):
if i % 100 == 0:
reporter.info('Evaluating: %d/%d' % (i, len(cityscapes)))
img_name = os.path.split(img_path)[1]
ids1 = CS.get_id(os.path.join(cityscapes_dir, img_name))
ids2 = CS.get_id(os.path.join(result_dir, img_name))
hist_perframe += fast_hist(ids1.flatten(), ids2.flatten(), CS.class_num)
mean_pixel_acc, mean_class_acc, mean_class_iou, per_class_acc, per_class_iou = get_scores(hist_perframe)
reporter.info("mean_pixel_acc:{}, mean_class_acc:{}, mean_class_iou: {}".format(mean_pixel_acc,
mean_class_acc,
mean_class_iou))
reporter.info("************ Per class numbers below ************")
for i, cl in enumerate(CS.classes):
while len(cl) < 15:
cl = cl + ' '
reporter.info("{}: acc = {}, iou = {}".format(cl, per_class_acc[i], per_class_iou[i]))
reporter.end_eval()
# Compare the similarity between the original image and the fake image
eval_process(args_opt, cityscapes_dir, result_dir)
Notice: If skipped training process, download the pretrained ckpt file and continue to serving
Click G_A to download G_A.ckpt file and G_B to download G_B.ckpt file. Save them to /etc/tinyms/serving/cyclegan_cityscape/
Or run the following code to download and store the ckpt files:
[2]:
ckpt_folder = '/etc/tinyms/serving/cyclegan_cityscape'
G_A_ckpt_path = '/etc/tinyms/serving/cyclegan_cityscape/G_A.ckpt'
G_B_ckpt_path = '/etc/tinyms/serving/cyclegan_cityscape/G_B.ckpt'
if not os.path.exists(ckpt_folder):
!mkdir -p /etc/tinyms/serving/cyclegan_cityscape
!wget -P /etc/tinyms/serving/cyclegan_cityscape https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cityscapes/G_A.ckpt
!wget -P /etc/tinyms/serving/cyclegan_cityscape https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cityscapes/G_B.ckpt
else:
print('ckpt folder already exists')
if not os.path.exists(G_A_ckpt_path):
!wget -P /etc/tinyms/serving/cyclegan_cityscape https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cityscapes/G_A.ckpt
if not os.path.exists(G_B_ckpt_path):
!wget -P /etc/tinyms/serving/cyclegan_cityscape https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cityscapes/G_B.ckpt
ckpt folder already exists
--2021-03-21 15:01:30-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cityscapes/G_A.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 121.36.121.44, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7785480 (7.4M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/cyclegan_cityscape/G_A.ckpt’
G_A.ckpt 100%[===================>] 7.42M 3.58MB/s in 2.1s
2021-03-21 15:01:33 (3.58 MB/s) - ‘/etc/tinyms/serving/cyclegan_cityscape/G_A.ckpt’ saved [7785480/7785480]
--2021-03-21 15:01:34-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cityscapes/G_B.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 121.36.121.44, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7785480 (7.4M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/cyclegan_cityscape/G_B.ckpt’
G_B.ckpt 100%[===================>] 7.42M 9.62MB/s in 0.8s
2021-03-21 15:01:36 (9.62 MB/s) - ‘/etc/tinyms/serving/cyclegan_cityscape/G_B.ckpt’ saved [7785480/7785480]
3. Define servable.json¶
Define the servable json file for model name, format and number of classes for later use.
[3]:
servable_json = [{'name': 'cyclegan_cityscape',
'description': 'This servable hosts a Cycle GAN model predicting for cityscape dataset',
'model': {
"name": "cycle_gan",
"format": "ckpt",
"g_model": "resnet"}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)
with open('servable.json', 'w') as json_file:
json_file.write(json_data)
4. Start server¶
4.1 Introduction¶
TinyMS Serving is a C/S(client/server) structure. There is a server and client. TinyMS using Flask which is a micro web framework written in python as the C/S communication tool. In order to serve a model, user must start server first. If successfully started, the server will be run in a subprocess and listening to POST requests from 127.0.0.1 port 5000 sent by client and handle the requests using MindSpore backend which will construct the model, run the prediction and send the result back to the client.
4.2 start server¶
Run the following code block to start the server:
[4]:
start_server()
Server starts at host 127.0.0.1, port 5000
5. Make predictions¶
5.1 List servables¶
Now, we can use list_servables function to check what model is servable right now.
[5]:
list_servables()
[5]:
[{'description': 'This servable hosts a Cycle GAN model predicting for cityscape dataset',
'model': {'format': 'ckpt', 'g_model': 'resnet', 'name': 'cycle_gan'},
'name': 'cyclegan_cityscape'}]
If the output description shows it is a CycleGAN model, then we can continue to next step to send our request.
5.2 Sending request and get the result¶
Run predict function to send the request, in this tutorial, both gray to color and color to gray will be demonstrated. Recommend using pics from cityscapes/test to run the predict. If user using own pics, resize to 256*256.
[6]:
servable_name = 'cyclegan_cityscape'
dataset_name = 'cityscape'
if server_started() is True:
# gray to color
testA_path = '/root/dataset/cityscapes/testA/1.jpg'
strategy = 'gray2color'
fakeB_data = predict(testA_path, servable_name, dataset_name, strategy)
# color to gray
testB_path = '/root/dataset/cityscapes/testB/1.jpg'
strategy = 'color2gray'
fakeA_data = predict(testB_path, servable_name, dataset_name, strategy)
# draw the plot
plt.figure(dpi=160, figsize=(10, 10))
plt.subplot(221)
plt.imshow(Image.open(testA_path))
plt.axis('off')
plt.title(testA_path)
plt.subplot(222)
plt.imshow(Image.fromarray(np.uint8(fakeB_data)))
plt.axis('off')
plt.title("fakeB.jpg")
plt.subplot(223)
plt.imshow(Image.open(testB_path))
plt.axis('off')
plt.title(testB_path)
plt.subplot(224)
plt.imshow(Image.fromarray(np.uint8(fakeA_data)))
plt.axis('off')
plt.title("fakeA.jpg")
plt.show()
else:
print("Server not started")
Check output¶
If user can see 4 pics, that means two on the left column are from original test datset, while two pic on the right column are the generated pic.
## Shutdown server
To restart and try another checkpoint file, click Kernel at the top, then Restart & Clear Output, replace the servable_json code and predict() function
Run the following code to shutdown Flask server:
[7]:
shutdown()
[7]:
'Server shutting down...'
Design Concept¶
Background¶
In recent years, with the rapid development of AI technology, deep learning frameworks such as TensorFlow, PyTorch, Apache MXNet and MindSpore have emerged. These frameworks are very good at solving problems for academic research or commercial production, however for first time beginners or application developers with limited deep learning knowledge, much simpler APIs are desired. Alongside the existing efforts like Keras for TensorFlow and Fastai for PyTorch to address the issue, the TinyMS project is a new addition in this field to provide simple high level APIs, tiny runtime footprint, modular development and agile deployment. TinyMS begins with initial focus on MindSpore integration and looks forward to more framework adaptations in the long run.
Interestingly enough, MindSpore’s high-level and mid-level Python APIs have already implemented most of the functions of Keras, and also been on par with Fastai’s design for PyTorch’s flexibility. Therefore unlike Keras and Fastai which are developed with most of their goals to compensate the underlying frameworks, TinyMS is designed to further enhance the experience of the framework, especially for all scenario development in the case of MindSpore.
With the help of TinyMS, the following goals should be achieved:
Quicker to learn: Get started with AI application development in one minute
Easier to develop: Complete the task of changing AI models and dataset from one to the other in one hour
Architecture¶
Design goals of TinyMS:
High level API that are extremely simple to learn and use.
Support complete AI development workflow from data preparation to model training/inference and finally deployment.
Decoupled modules that are could be easily extended.
Small runtime footprint that could be used on mobile, edge or cloud.
Standardizing spec for model training script format.
TinyMS Architecture
Workflow analysis¶
Typical model development workflow:
Data Acquisition: Dataset download, decompression, loading, etc.
Data Processing: Data preprocessing (enhancement) operations performed on the original dataset for better model performance.
Model Construction: Construction of the network, and also the definition of loss function, optimizer, etc.
Model Training: The process of model training, including the definition of callbacks
Accuracy Verification: The process of model accuracy verification, including the definition of metrics
Model Deployment: Model application services via an inference server
TinyMS Workflow
Module design¶
TinyMS has the following modules:
| Name | Introduction | Example Code |
|---|---|---|
| data | Dataset Loading and Downloading | from tinyms.data import MnistDataset, download_dataset |
| model | Model High Level API and Predefined Network | from tinyms.model import Model, lenet5 |
| serving | Model Serving | from tinyms.serving import predict |
| vision | Computer Vision Related Data Processing | from tinyms.vision import mnist_transform, Resize |
| callbacks | Callbacks During Model Training | from tinyms.callbacks import ModelCheckpoint |
| common | Basic Components Including Tensor, Numpy Style Functions | from tinyms import Tensor, array |
| context | Global Context | from tinyms import context |
| initializers | Ops Weight Initialization | from tinyms.initializers import Normal |
| layers | Neural Network Layer | from tinyms.layers import Layer, Conv2d |
| losses | Loss Function | from tinyms.losses import SoftmaxCrossEntropyWithLogits |
| metrics | Metrics For Model Verification | from tinyms.metrics import Accuracy |
| optimizers | Optimizer | from tinyms.optimizers import Momentum |
| primitives | Basic Ops | from tinyms.primitives import Add, tensor_add |
Implementation¶
Data loading (data)¶
The data loading module is mainly divided into two parts: dataset download and loading. Through TinyMS’s data loading API, developers can complete the entire process of downloading, decompressing, formatting, and loading common datasets with just two lines of code.
Most AI frameworks do not provide an interface for dataset download. Users need to prepare the dataset in advance, and at the same time, adjust the format (training/validation data set division, etc.) according to the data loading API format provided by the framework itself. To make dataset API usage much more easier, TinyMS provides the download_dataset interface, which supports users to complete the download, decompression and format adjustment operations of the data set with one click; use Mnist dataset as an example:
from tinyms.data import download_dataset
mnist_path = download_dataset('mnist', local_path='./')
For data loading operations, TinyMS completely inherits MindSpore’s native Data Loading API. Users can use the xxxDataset interface to instantiate different data sets very conveniently. Take MnistDataset as an example:
from tinyms.data import MnistDataset
mnist_ds = MnistDataset(mnist_path, shuffle=True)
Data preprocessing (vision)¶
Usually in the model development workflow, data processing presents a big challenge: insufficient data, heavy manual labeling task, irregular data format and many other issues. Any of them could affect the network accuracy after training. Most frameworks provide data processing modules. Take MindSpore as an example, it currently provides data processing functions for common scenarios such as CV and NLP (for relevant interface definitions, please refer to mindspore.dataset.vision and mindspore.dataset.text), the user can directly call the preset data processing operator to process pictures or text, and then construct a data processing pipeline to efficiently parallelize massive data (see here).
TinyMS has made further abstraction and encapsulation on the basis of MindSpore, and directly corresponds to the processing of the dataset itself through the DatasetTransform interface, allowing users to utilize a single piece of data or the entire dataset with just one line of code regarding preprocessing operation; take MnistTransform as an example:
from PIL import Image
from tinyms.vision import mnist_transform
# Preprocessing a single one picture
img = mnist_transform(Image.open('picture.jpg'))
# Apply preprocessing to MnistDataset class instance
mnist_ds = mnist_transform.apply_ds(mnist_ds)
Model construction (model)¶
As the core of deep learning model development, the framework’s main responsibility is to provide complete operator expressions to build different network structures. Therefore, the interfaces at the framework level focus more on functional completeness and flexibility, whereas ModelZoo is provided for application development. TinyMS encapsulates the relevant network call API on the ModelZoo script; take the LeNet5 network as an example:
from tinyms.model import lenet5
net = lenet5(class_num=10)
In addition to encapsulating the commonly used network structures, TinyMS also provides a Model high-level API interface (based on MindSpore Model interface package), by drawing on the design idea of Keras Model interface, it not only improves the original API functionalities, but also provides a consistent development experience for Keras users:
from tinyms.model import Model
model = Model(net)
model.compile(loss_fn=net_loss, optimizer=net_opt)
Model training (losses, optimizers, callbacks)¶
For the model training phase, the most important factors are the definitions of loss functions, optimizers, and callback functions. For beginners, it is not difficult to understand the basic principles of loss functions and optimizers, but a strong mathematical background is required to understand the principles of implementation. Therefore, the TinyMS high-level API encapsulates the loss function and optimizer at the network level, so that users can complete the initialization work with one line of code whether they are training simple or complex networks; take the LeNet5 network as an example:
from tinyms.losses import SoftmaxCrossEntropyWithLogits
from tinyms.optimizers import Momentum
lr = 0.01
momentum = 0.9
net_loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')
net_opt = Momentum(net.trainable_params(), lr, momentum)
Regarding the definition of callback functions, in addition to commonly used callback functions (such as TimeMonitor, LossMonitor, etc.), MindSpore itself provides Callback interface to facilitate user-defined callback functions. The TinyMS high-level API also provides network-level encapsulation, so that users can complete the initialization of the callback function with one line of code; take the MobileNetV2 network as an example:
from tinyms.callbacks import mobilenetv2_cb
net_cb = mobilenetv2_cb(device_target, lr, is_saving_checkpoint, save_checkpoint_epochs, step_size)
Model evaluating (metrics)¶
Model accuracy verification is an indispensable process to verify whether the model accuracy meets the SOTA criteria. MindSpore natively provides measurement interfaces for indicators such as Accuracy and Precision (see here), while providing users with a custom measurement interface Metric. In terms of metric measurement, TinyMS directly inherits the native MindSpore API:
from tinyms.model import Model
from tinyms.metrics import Accuracy
model = Model(net)
model.compile(metrics={"Accuracy": Accuracy())
Model deployment (serving)¶
Model deployment refers to the process of servicing pre-trained models so that they can quickly and efficiently process data input by users and obtain results. MindSpore provides the predict function for inference. TinyMS provides a complete set of start server (start_server), check backend (list_servables), check start status (server_started) and shut down the server (shutdown) and other functions based on Flask ; Take the LeNet5 network as an example:
from tinyms.serving import start_server, predict, list_servables, server_started, shutdown
# Start prediction server
start_server()
# List all servables available
list_servables()
# Call predict interface
if server_started():
res = predict(image_path, 'lenet5', 'mnist')
# Shutdown the prediction server
shutdown()
tinyms¶
Top-level reference to dtype of common module. This module also provides Numpy-like interfaces in TinyMS.
Examples
>>> import tinyms as ts
>>>
>>> print(ts.ones([2, 3]))
[[1. 1. 1.]
[1. 1. 1.]]
-
tinyms.dtype_to_nptype(type_)[source]¶ Convert MindSpore dtype to numpy data type.
- Parameters
type_ (
mindspore.dtype) – MindSpore’s dtype.- Returns
The data type of numpy.
-
tinyms.issubclass_(type_, dtype)[source]¶ Determine whether type_ is a subclass of dtype.
- Parameters
type_ (
mindspore.dtype) – Target MindSpore dtype.dtype (
mindspore.dtype) – Compare MindSpore dtype.
- Returns
bool, True or False.
-
tinyms.dtype_to_pytype(type_)[source]¶ Convert MindSpore dtype to python data type.
- Parameters
type_ (
mindspore.dtype) – MindSpore’s dtype.- Returns
Type of python.
-
tinyms.pytype_to_dtype(obj)[source]¶ Convert python type to MindSpore type.
- Parameters
obj (type) – A python type object.
- Returns
Type of MindSpore type.
-
tinyms.get_py_obj_dtype(obj)[source]¶ Get the MindSpore data type which corresponds to python type or variable.
- Parameters
obj – An object of python type, or a variable in python type.
- Returns
Type of MindSpore type.
-
class
tinyms.MetaTensor(dtype, shape, init=None)[source]¶ The base class of the MetaTensor. Initialization of tensor basic attributes and model weight values.
- Returns
Array, an array after being initialized.
-
property
dtype¶ Get the MetaTensor’s dtype.
-
property
shape¶ Get the MetaTensor’s shape.
-
to_tensor(slice_index=None, shape=None, opt_shard_group=None)[source]¶ Get the tensor format data of this MetaTensor.
- Parameters
slice_index (int) – Slice index of a parameter’s slices. It is used when initialize a slice of a parameter, it guarantees that devices using the same slice can generate the same tensor.
shape (list[int]) – Shape of the slice, it is used when initialize a slice of the parameter.
opt_shard_group (str) – Optimizer shard group which is used in auto or semi auto parallel mode to get one shard of a parameter’s slice.
-
class
tinyms.Tensor(input_data, dtype=None)[source]¶ Tensor is used for data storage.
Tensor inherits tensor object in C++. Some functions are implemented in C++ and some functions are implemented in Python.
- Parameters
input_data (Tensor, float, int, bool, tuple, list, numpy.ndarray) – Input data of the tensor.
dtype (
mindspore.dtype) – Input data should be None, bool or numeric type defined in mindspore.dtype. The argument is used to define the data type of the output tensor. If it is None, the data type of the output tensor will be as same as the input_data. Default: None.
- Outputs:
Tensor, with the same shape as input_data.
Examples
>>> import mindspore as ms >>> import mindspore.nn as nn >>> # initialize a tensor with input data >>> t1 = Tensor(np.zeros([1, 2, 3]), mindspore.float32) >>> assert isinstance(t1, Tensor) >>> assert t1.shape == (1, 2, 3) >>> assert t1.dtype == mindspore.float32 ... >>> # initialize a tensor with a float scalar >>> t2 = Tensor(0.1) >>> assert isinstance(t2, Tensor) >>> assert t2.dtype == mindspore.float64
-
all(axis=(), keep_dims=False)[source]¶ Check all array elements along a given axis evaluate to True.
- Parameters
- Returns
Tensor, has the same data type as x.
-
any(axis=(), keep_dims=False)[source]¶ Check any array element along a given axis evaluate to True.
- Parameters
- Returns
Tensor, has the same data type as x.
-
assign_value(self: mindspore._c_expression.Tensor, arg0: mindspore._c_expression.Tensor) → mindspore._c_expression.Tensor¶ Assign another tensor value to this.
- Arg:
value (
mindspore.tensor): The value tensor.
Examples
>>> data = mindspore.Tensor(np.ones((1, 2), np.float32)) >>> data2 = mindspore.Tensor(np.ones((2, 2), np.float32)) >>> data.assign_value(data2) >>> data.shape (2, 2)
-
data_sync(self: mindspore._c_expression.Tensor, arg0: bool) → None¶
-
dim(self: mindspore._c_expression.Tensor) → int¶ Get tensor’s data dimension.
- Returns
int, the dimension of tensor.
Examples
>>> data = mindspore.Tensor(np.ones((2, 3))) >>> data.dim() 2
-
property
dtype¶ The dtype of tensor is a mindspore type.
-
expand_as(x)[source]¶ Expand the dimension of target tensor to the dimension of input tensor.
- Parameters
shape (Tensor) – The input tensor. The shape of input tensor must obey the broadcasting rule.
- Returns
Tensor, has the same dimension as input tensor.
-
is_init(self: mindspore._c_expression.Tensor) → bool¶ Get tensor init_flag.
- Returns
bool, whether the tensor init.
Examples
>>> data = mindspore.Tensor(np.ones((2, 3))) >>> data.is_init() False
-
mean(axis=(), keep_dims=False)[source]¶ Reduces a dimension of a tensor by averaging all elements in the dimension.
- Parameters
- Returns
Tensor, has the same data type as x.
-
property
ndim¶ The ndim of tensor is an integer.
-
set_cast_dtype(self: mindspore._c_expression.Tensor, dtype: mindspore::Type = None) → None¶
-
set_dtype(self: mindspore._c_expression.Tensor, arg0: mindspore::Type) → mindspore::Type¶ Set the tensor’s data type.
- Arg:
dtype (
mindspore.dtype): The type of output tensor.
Examples
>>> data = mindspore.Tensor(np.ones((1, 2), np.float32)) >>> data.set_dtype(mindspore.int32) mindspore.int32
-
set_init_flag(self: mindspore._c_expression.Tensor, arg0: bool) → None¶ Set tensor init_flag.
Examples
>>> data = mindspore.Tensor(np.ones((2, 3))) >>> data.set_init_flag(True)
-
property
shape¶ The shape of tensor is a tuple.
-
property
size¶ The size reflects the total number of elements in tensor.
-
property
virtual_flag¶ Mark tensor is virtual.
-
class
tinyms.RowTensor(indices, values, dense_shape)[source]¶ A sparse representation of a set of tensor slices at given indices.
An RowTensor is typically used to represent a subset of a larger tensor dense of shape [L0, D1, .. , DN] where L0 >> D0.
The values in indices are the indices in the first dimension of the slices that have been extracted from the larger tensor.
The dense tensor dense represented by an RowTensor slices has dense[slices.indices[i], :, :, :, …] = slices.values[i, :, :, :, …].
RowTensor can only be used in the Cell’s construct method.
It is not supported in pynative mode at the moment.
- Parameters
- Returns
RowTensor, composed of indices, values, and dense_shape.
Examples
>>> import mindspore as ms >>> import mindspore.nn as nn >>> class Net(nn.Cell): ... def __init__(self, dense_shape): ... super(Net, self).__init__() ... self.dense_shape = dense_shape ... def construct(self, indices, values): ... x = RowTensor(indices, values, self.dense_shape) ... return x.values, x.indices, x.dense_shape >>> >>> indices = Tensor([0]) >>> values = Tensor([[1, 2]], dtype=ms.float32) >>> out = Net((3, 2))(indices, values) >>> print(out[0]) [[1. 2.]] >>> print(out[1]) [0] >>> print(out[2]) (3, 2)
-
class
tinyms.SparseTensor(indices, values, dense_shape)[source]¶ A sparse representation of a set of nonzero elememts from a tensor at given indices.
SparseTensor can only be used in the Cell’s construct method.
Pynative mode not supported at the moment.
For a tensor dense, its SparseTensor(indices, values, dense_shape) has dense[indices[i]] = values[i].
- Parameters
indices (Tensor) – A 2-D integer Tensor of shape [N, ndims], where N and ndims are the number of values and number of dimensions in the SparseTensor, respectively.
values (Tensor) – A 1-D tensor of any type and shape [N], which supplies the values for each element in indices.
dense_shape (tuple) – A integer tuple of size ndims, which specifies the dense_shape of the sparse tensor.
- Returns
SparseTensor, composed of indices, values, and dense_shape.
Examples
>>> import mindspore as ms >>> import mindspore.nn as nn >>> class Net(nn.Cell): ... def __init__(self, dense_shape): ... super(Net, self).__init__() ... self.dense_shape = dense_shape ... def construct(self, indices, values): ... x = SparseTensor(indices, values, self.dense_shape) ... return x.values, x.indices, x.dense_shape >>> >>> indices = Tensor([[0, 1], [1, 2]]) >>> values = Tensor([1, 2], dtype=ms.float32) >>> out = Net((3, 4))(indices, values) >>> print(out[0]) [1. 2.] >>> print(out[1]) [[0 1] [1 2]] >>> print(out[2]) (3, 4)
-
tinyms.ms_function(fn=None, obj=None, input_signature=None)[source]¶ Create a callable MindSpore graph from a python function.
This allows the MindSpore runtime to apply optimizations based on graph.
- Parameters
fn (Function) – The Python function that will be run as a graph. Default: None.
obj (Object) – The Python Object that provides the information for identifying the compiled function.Default: None.
input_signature (MetaTensor) – The MetaTensor which describes the input arguments. The MetaTensor specifies the shape and dtype of the Tensor and they will be supplied to this function. If input_signature is specified, each input to fn must be a Tensor. And the input parameters of fn cannot accept **kwargs. The shape and dtype of actual inputs should keep the same as input_signature. Otherwise, TypeError will be raised. Default: None.
- Returns
Function, if fn is not None, returns a callable function that will execute the compiled function; If fn is None, returns a decorator and when this decorator invokes with a single fn argument, the callable function is equal to the case when fn is not None.
Examples
>>> from mindspore.ops import functional as F ... >>> x = Tensor(np.ones([1, 1, 3, 3]).astype(np.float32)) >>> y = Tensor(np.ones([1, 1, 3, 3]).astype(np.float32)) ... >>> # create a callable MindSpore graph by calling ms_function >>> def tensor_add(x, y): ... z = x + y ... return z ... >>> tensor_add_graph = ms_function(fn=tensor_add) >>> out = tensor_add_graph(x, y) ... >>> # create a callable MindSpore graph through decorator @ms_function >>> @ms_function ... def tensor_add_with_dec(x, y): ... z = x + y ... return z ... >>> out = tensor_add_with_dec(x, y) ... >>> # create a callable MindSpore graph through decorator @ms_function with input_signature parameter >>> @ms_function(input_signature=(MetaTensor(mindspore.float32, (1, 1, 3, 3)), ... MetaTensor(mindspore.float32, (1, 1, 3, 3)))) ... def tensor_add_with_sig(x, y): ... z = x + y ... return z ... >>> out = tensor_add_with_sig(x, y)
-
class
tinyms.Parameter(default_input, name=None, requires_grad=True, layerwise_parallel=False)[source]¶ Parameter types of cell models.
After initialized Parameter is a subtype of Tensor.
In auto_parallel mode of “semi_auto_parallel” and “auto_parallel”, if init Parameter by an MetaTensor, the type of Parameter will be MetaTensor not Tensor. MetaTensor_ only saves the shape and type info of a tensor with no memory usage. The shape can be changed while compiling for auto-parallel. Call init_data will return a Tensor Parameter with initialized data.
Note
Each parameter of Cell is represented by Parameter class. A Parameter has to belong to a Cell. If there is an operator in the network that requires part of the inputs to be Parameter, then the Parameters as this part of the inputs are not allowed to be cast. It is recommended to use the default value of name when initialize a parameter as one attribute of a cell, otherwise, the parameter name may be different than expected.
- Parameters
default_input (Union[Tensor, MetaTensor, Number]) – Parameter data, to be set initialized.
name (str) – Name of the child parameter. Default: None.
requires_grad (bool) – True if the parameter requires gradient. Default: True.
layerwise_parallel (bool) – A kind of model parallel mode. When layerwise_parallel is true in parallel mode, broadcast and gradients communication would not be applied to parameters. Default: False.
Example
>>> from mindspore import Parameter, Tensor >>> from mindspore.common import initializer as init >>> from mindspore.ops import operations as P >>> from mindspore.nn import Cell >>> import mindspore >>> import numpy as np >>> from mindspore import context >>> >>> class Net(Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.matmul = P.MatMul() ... self.weight = Parameter(Tensor(np.ones((1,2))), name="w", requires_grad=True) ... ... def construct(self, x): ... out = self.matmul(self.weight, x) ... return out >>> context.set_context(mode=context.GRAPH_MODE, device_target="CPU") >>> net = Net() >>> x = Tensor(np.ones((2,1))) >>> print(net(x)) [[2.]] >>> net.weight.set_data(Tensor(np.zeros((1,2)))) Parameter (name=w) >>> print(net(x)) [[0.]]
-
clone(init='same')[source]¶ Clone the parameter.
- Parameters
init (Union[Tensor, str, MetaTensor, numbers.Number]) – Initialize the shape of the parameter. Default: ‘same’.
- Returns
Parameter, a new parameter.
-
property
dtype¶ Get the MetaTensor’s dtype.
-
init_data(layout=None, set_sliced=False)[source]¶ Initialize the parameter data.
- Parameters
- Raises
RuntimeError – If it is from Initializer, and parallel mode has changed after the Initializer created.
- Returns
Parameter, the Parameter after initializing data. If current Parameter was already initialized before, returns the same initialized Parameter.
-
property
inited_param¶ Get the new parameter after call the init_data.
Default is a None, If self is a Parameter with out data, after call the init_data the initialized Parameter with data will be recorded here.
-
property
is_init¶ Get the initialization status of the parameter.
In GE backend, the Parameter need a “init graph” to sync the data from host to device. This flag indicates whether the data as been sync to the device.
This flag only work in GE, and it will be set to False in other backend.
-
property
name¶ Get the name of the parameter.
-
property
requires_grad¶ Return whether the parameter requires gradient.
-
set_data(data, slice_shape=False)[source]¶ Set set_data of current Parameter.
- Parameters
data (Union[Tensor, MetaTensor, int, float]) – new data.
slice_shape (bool) – If slice the parameter is set to true, the shape is not checked for consistency. Default: False.
- Returns
Parameter, the parameter after set data.
-
set_param_ps(init_in_server=False)[source]¶ Set whether the trainable parameter is updated by parameter server and whether the trainable parameter is initialized on server.
Note
It only works when a running task is in the parameter server mode.
- Parameters
init_in_server (bool) – Whether trainable parameter updated by parameter server is initialized on server. Default: False.
-
property
shape¶ Get the MetaTensor’s shape.
-
property
sliced¶ Get slice status of the parameter.
-
property
unique¶ whether the parameter is already unique or not.
-
class
tinyms.ParameterTuple[source]¶ Class for storing tuple of parameters.
Note
It is used to store the parameters of the network into the parameter tuple collection.
-
count()¶ Return number of occurrences of value.
-
index()¶ Return first index of value.
Raises ValueError if the value is not present.
-
-
tinyms.set_seed(seed)[source]¶ Set global random seed.
Note
The global seed is used by numpy.random, mindspore.common.Initializer, mindspore.ops.composite.random_ops and mindspore.nn.probability.distribution.
If global seed is not set, these packages will use their own default seed independently, numpy.random and mindspore.common.Initializer will choose a random seed, mindspore.ops.composite.random_ops and mindspore.nn.probability.distribution will use zero.
Seed set by numpy.random.seed() only used by numpy.random, while seed set by this API will also used by numpy.random, so just set all seed by this API is recommended.
- Parameters
seed (int) – The seed to be set.
- Raises
ValueError – If seed is invalid (< 0).
TypeError – If seed isn’t a int.
Examples
>>> from mindspore.ops import composite as C >>> from mindspore import Tensor >>> >>> # Note: (1) Please make sure the code is running in PYNATIVE MODE; >>> # (2) Becasuse Composite-level ops need parameters to be Tensors, for below examples, >>> # when using C.uniform operator, minval and maxval are initialised as: >>> minval = Tensor(1.0, mstype.float32) >>> maxval = Tensor(2.0, mstype.float32) >>> >>> # 1. If global seed is not set, numpy.random and initializer will choose a random seed: >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A1 >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A2 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W1 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W2 >>> # Rerun the program will get diferent results: >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A3 >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A4 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W3 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W4 >>> >>> # 2. If global seed is set, numpy.random and initializer will use it: >>> set_seed(1234) >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A1 >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A2 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W1 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W2 >>> # Rerun the program will get the same results: >>> set_seed(1234) >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A1 >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A2 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W1 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W2 >>> >>> # 3. If neither global seed nor op seed is set, mindspore.ops.composite.random_ops and >>> # mindspore.nn.probability.distribution will choose a random seed: >>> c1 = C.uniform((1, 4), minval, maxval) # C1 >>> c2 = C.uniform((1, 4), minval, maxval) # C2 >>> # Rerun the program will get different results: >>> c1 = C.uniform((1, 4), minval, maxval) # C3 >>> c2 = C.uniform((1, 4), minval, maxval) # C4 >>> >>> # 4. If global seed is set, but op seed is not set, mindspore.ops.composite.random_ops and >>> # mindspore.nn.probability.distribution will caculate a seed according to global seed and >>> # default op seed. Each call will change the default op seed, thus each call get different >>> # results. >>> set_seed(1234) >>> c1 = C.uniform((1, 4), minval, maxval) # C1 >>> c2 = C.uniform((1, 4), minval, maxval) # C2 >>> # Rerun the program will get the same results: >>> set_seed(1234) >>> c1 = C.uniform((1, 4), minval, maxval) # C1 >>> c2 = C.uniform((1, 4), minval, maxval) # C2 >>> >>> # 5. If both global seed and op seed are set, mindspore.ops.composite.random_ops and >>> # mindspore.nn.probability.distribution will caculate a seed according to global seed and >>> # op seed counter. Each call will change the op seed counter, thus each call get different >>> # results. >>> set_seed(1234) >>> c1 = C.uniform((1, 4), minval, maxval, seed=2) # C1 >>> c2 = C.uniform((1, 4), minval, maxval, seed=2) # C2 >>> # Rerun the program will get the same results: >>> set_seed(1234) >>> c1 = C.uniform((1, 4), minval, maxval, seed=2) # C1 >>> c2 = C.uniform((1, 4), minval, maxval, seed=2) # C2 >>> >>> # 6. If op seed is set but global seed is not set, 0 will be used as global seed. Then >>> # mindspore.ops.composite.random_ops and mindspore.nn.probability.distribution act as in >>> # condition 5. >>> c1 = C.uniform((1, 4), minval, maxval, seed=2) # C1 >>> c2 = C.uniform((1, 4), minval, maxval, seed=2) # C2 >>> # Rerun the program will get the same results: >>> c1 = C.uniform((1, 4), minval, maxval, seed=2) # C1 >>> c2 = C.uniform((1, 4), minval, maxval, seed=2) # C2 >>> >>> # 7. Recall set_seed() in the program will reset numpy seed and op seed counter of >>> # mindspore.ops.composite.random_ops and mindspore.nn.probability.distribution. >>> set_seed(1234) >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A1 >>> c1 = C.uniform((1, 4), minval, maxval, seed=2) # C1 >>> set_seed(1234) >>> np_2 = np.random.normal(0, 1, [1]).astype(np.float32) # still get A1 >>> c2 = C.uniform((1, 4), minval, maxval, seed=2) # still get C1
-
tinyms.arange(*args, **kwargs)[source]¶ Returns evenly spaced values within a given interval.
Returns num evenly spaced samples, calculated over the interval [start, stop]. The endpoint of the interval can optionally be excluded. The current implementation is a direct wrapper on top of numpy.arange, except that the default dtype is float32 and int32, compare to float64 and int64 for numpy implementation.
- Parameters
start (Union[int, float]) – Start of interval. The interval includes this value. When stop is provided as a position argument, start must be given, when stop is a normal argument, start can be optional, and default is 0. Please see additional examples below.
stop (Union[int, float], optional) – End of interval. The interval does not include this value, except in some cases where step is not an integer and floating point round-off affects the length of out.
step (Union[int, float], optional) – Spacing between values. For any output out, this is the distance between two adjacent values, out[i+1] - out[i]. The default step size is 1. If step is specified as a position argument, start must also be given.
dtype (Union[mstype.dtype, str], optional) – Designated tensor dtype, can be in format of np.float32, or float32. If dtype is None, the data type of the new tensor will be inferred from start, stop and step. Default is None.
- Returns
arangend tensor of evenly spaced values.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.arange(0, 5, 1)) [0 1 2 3 4] >>> print(np.arange(3)) [0 1 2] >>> print(np.arange(start=0, stop=3)) [0 1 2] >>> print(np.arange(0, stop=3, step=0.5)) [0. 0.5 1. 1.5 2. 2.5] >>> print(np.arange(stop=3)) # This will lead to TypeError
-
tinyms.array(obj, dtype=None, copy=True, ndmin=0)[source]¶ Creates a tensor.
This function creates tensors from an array-like object.
- Parameters
obj (Union[int, float, bool, list, tuple, numpy.ndarray]) – Input data, in
form that can be converted to a tensor. This includes lists, lists of (any) –
tuples, tuples of tuples, tuples of lists and numpy.ndarray. (tuples,) –
dtype (Union[mstype.dtype, str], optional) – Designated tensor dtype, can be in format of np.int32, or int32. If dtype is None, the data type of the new tensor will be inferred from obj. Default is None.
copy (bool) – If true, then the object is copied. Otherwise, a copy will only be made if necessary. Default: True.
ndmin (int) – Specifies the minimum number of dimensions that the resulting tensor should have. Ones will be pre-pended to the shape as needed to meet this requirement. Default: 0
- Returns
Tensor, generated tensor with the specified dtype.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.array([1,2,3])) [1 2 3]
-
tinyms.asarray(a, dtype=None)[source]¶ Converts the input to tensor.
This function converts tensors from an array-like object.
- Parameters
a (Union[int, float, bool, list, tuple, numpy.ndarray]) – Input data, in
form that can be converted to a tensor. This includes lists, lists of (any) –
tuples, tuples of tuples, tuples of lists and ndarrays. (tuples,) –
dtype (Union[mstype.dtype, str], optional) – Designated tensor dtype, can be in format of np.int32, or int32. If dtype is None, the data type of the new tensor will be inferred from a. Default is None.
- Returns
Tensor, generated tensor with the specified dtype.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.asarray([1,2,3])) [1 2 3]
-
tinyms.asfarray(a, dtype=mindspore.float32)[source]¶ Similar to asarray, converts the input to a float tensor.
If non-float dtype is defined, this function will return a float32 tensor instead.
- Parameters
a (Union[int, float, bool, list, tuple, numpy.ndarray]) – Input data, in
form that can be converted to a tensor. This includes lists, lists of (any) –
tuples, tuples of tuples, tuples of lists and numpy.ndarray. (tuples,) –
dtype (Union[mstype.dtype, str], optional) – Designated tensor dtype, can be in format of np.float32, or float32. Default is mstype.float32.
- Returns
Tensor, generated tensor with the specified float dtype.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.asfarray([1,2,3])) [1. 2. 3.]
-
tinyms.concatenate(arrays, axis=0)[source]¶ Joins a sequence of tensors along an existing axis.
- Parameters
arrays – Union[Tensor, tuple(Tensor), list(Tensor)], a tensor or a list
tensors to be concatenated. (of) –
axis (int, optional) – The axis along which the tensors will be joined, if axis is None, tensors are flattened before use. Default is 0.
- Returns
Tensor, a tensor concatenated from a tensor or a list of tensors.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.ones((1,2,3)) >>> x2 = np.ones((1,2,1)) >>> x = np.concatenate((x1, x2), axis=-1) >>> print(x.shape) (1, 2, 4)
-
tinyms.copy(a)¶ Returns a tensor copy of the given object.
- Parameters
a (Tensor) – Input tensor.
- Returns
Tensor, has the same data as a.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.ones((2,2)) >>> print(np.copy(x)) [[1. 1.] [1. 1.]]
-
tinyms.expand_dims(a, axis)[source]¶ Expands the shape of a tensor.
Inserts a new axis that will appear at the axis position in the expanded tensor shape.
- Parameters
a (Tensor) – Input tensor array.
Union[int, list (axis) – Position in the expanded axes where
new axis is placed, (the) –
- Returns
Tensor, view of a tensor with the number of dimensions increased.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.ones((2,2)) >>> x = np.expand_dims(x,0) >>> print(x.shape) (1, 2, 2)
-
tinyms.eye(N, M=None, k=0, dtype=mindspore.float32)[source]¶ Returns a 2-D tensor with ones on the diagnoal and zeros elsewhere.
- Parameters
N (int) – Number of rows in the output, must be larger than 0.
M (int, optional) – Number of columns in the output. If None, defaults to N, if defined, must be larger than 0. Deault is None.
k (int, optional) – Index of the diagonal: 0 (the default) refers to the main diagonal, a positive value refers to an upper diagonal, and a negative value to a lower diagonal. Default is 0.
dtype (Union[mstype.dtype, str], optional) – Designated tensor dtype, can be in format of np.float32, or float32. Default is mstype.float32.
- Returns
A tensor of shape (N,M). A tensor where all elements are equal to zero, except for the k-th diagonal, whose values are equal to one.
- Return type
result (Tensor)
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.eye(2, 2)) [[1. 0.] [0. 1.]]
-
tinyms.identity(n, dtype=mindspore.float32)[source]¶ Returns the identity tensor.
- Parameters
- Returns
A tensor of shape (n,n). A tensor where all elements are equal to zero, except for the diagonal, whose values are equal to one.
- Return type
result (Tensor)
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.identity(2)) [[1. 0.] [0. 1.]]
-
tinyms.inner(a, b)[source]¶ Inner product of two tensors.
Ordinary inner product of vectors for 1-D tensors (without complex conjugation), in higher dimensions a sum product over the last axes.
Note
Numpy argument out is not supported. On GPU, the supported dtypes are mstype.float16, and mstype.float32. On CPU, the supported dtype is mstype.float32.
- Parameters
- Returns
-1] + b.shape[:-1].
- Return type
Tensor or scalar, out.shape = a.shape[
- Raises
ValueError – if x1.shape[-1] != x2.shape[-1].
- Supported Platforms:
Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.ones((5, 3)) >>> b = np.ones((2, 7, 3)) >>> output = np.inner(a, b) >>> print(output) [[[3. 3. 3. 3. 3. 3. 3.] [3. 3. 3. 3. 3. 3. 3.]]
[[3. 3. 3. 3. 3. 3. 3.] [3. 3. 3. 3. 3. 3. 3.]]
[[3. 3. 3. 3. 3. 3. 3.] [3. 3. 3. 3. 3. 3. 3.]]
[[3. 3. 3. 3. 3. 3. 3.] [3. 3. 3. 3. 3. 3. 3.]]
[[3. 3. 3. 3. 3. 3. 3.] [3. 3. 3. 3. 3. 3. 3.]]]
-
tinyms.linspace(start, stop, num=50, endpoint=True, retstep=False, dtype=None, axis=0)[source]¶ Returns evenly spaced values within a given interval.
The current implementation is a direct wrapper on top of numpy.linspace, except the default dtype is float32, compare to float64 for numpy,
- Parameters
start (Union[int, list(int), tuple(int),tensor]) – The starting value of the sequence.
stop (Union[int, list(int), tuple(int),tensor]) – The end value of the sequence, unless endpoint is set to False. In that case, the sequence consists of all but the last of ``num + 1` evenly spaced samples, so that stop is excluded. Note that the step size changes when endpoint is False.
num (int, optional) – Number of samples to generate. Default is 50.
endpoint (bool, optional) – If True, stop is the last sample. Otherwise, it is not included. Default is True.
retstep (bool, optional) – If True, return (samples, step), where step is the spacing between samples.
dtype (Union[mstype.dtype, str], optional) – Designated tensor dtype, can be in format of np.float32, or float32.If dtype is None, infer the data type from other input arguments. Default is None.
axis (int, optional) – The axis in the result to store the samples. Relevant only if start or stop are array-like. By default (0), the samples will be along a new axis inserted at the beginning. Use -1 to get an axis at the end. Default is 0.
- Returns
- There are num equally spaced samples in the closed interval
[start, stop]or the half-open interval[start, stop)(depending on whether endpoint is True or False).- step (float, optional): Only returned if retstep is True.
Size of spacing between samples.
- Return type
samples (Tensor)
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.linspace(0, 5, 6)) [0. 1. 2. 3. 4. 5.]
-
tinyms.logspace(start, stop, num=50, endpoint=True, base=10.0, dtype=None, axis=0)[source]¶ Returns numbers spaced evenly on a log scale.
In linear space, the sequence starts at base ** start (base to the power of start) and ends with base ** stop (see endpoint below). The current implementation is a direct wrapper on top of numpy.logspace, except the default dtype is float32, compare to float64 for numpy,
- Parameters
start (Union[int, list(int), tuple(int), tensor]) – The starting value of the sequence.
stop (Union[int, list(int), tuple(int), tensor]) – The end value of the sequence, unless endpoint is set to False. In that case, the sequence consists of all but the last of ``num + 1` evenly spaced samples, so that stop is excluded. Note that the step size changes when endpoint is False.
num (int, optional) – Number of samples to generate. Default is 50.
endpoint (bool, optional) – If True, stop is the last sample. Otherwise, it is not included. Default is True.
base (Union[int, float], optional) – The base of the log space. The step size between the elements in ln(samples) / ln(base) (or log_base(samples)) is uniform. Default is 10.0.
dtype (Union[mstype.dtype, str], optional) – Designated tensor dtype, can be in format of np.float32, or float32.If dtype is None, infer the data type from other input arguments. Default is None.
axis (int, optional) – The axis in the result to store the samples. Relevant only if start or stop is array-like. By default (0), the samples will be along a new axis inserted at the beginning. Use -1 to get an axis at the end. Default is 0.
- Returns
num samples, equally spaced on a log scale.
- Return type
samples (Tensor)
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.logspace(0, 5, 6, base=2.0)) [ 1. 2. 4. 8. 16. 32.]
-
tinyms.mean(a, axis=None, keepdims=False)[source]¶ Computes the arithmetic mean along the specified axis.
Returns the average of the array elements. The average is taken over the flattened array by default, otherwise over the specified axis.
Note
Numpy arguments dtype and out are not supported. On GPU, the supported dtypes are mstype.float16, and mstype.float32. On CPU, the supported dtypes are mstype.float16, and mstype.float32.
- Parameters
a (Tensor) – input tensor containing numbers whose mean is desired. If a is not an array, a conversion is attempted.
axis (None or int or tuple of ints) – optional. Axis or axes along which the means are computed. The default is to compute the mean of the flattened array. If this is a tuple of ints, a mean is performed over multiple axes.
keepdims (bool) – optional. If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input tensor.
- Returns
Tensor or scalar, an array containing the mean values.
- Raises
ValueError – if axes are out of the range of [-a.ndim, a.ndim), or
if the axes contain duplicates. –
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.arange(6, dtype='float32') >>> output = np.mean(a, 0) >>> print(output) 2.5
-
tinyms.ones(shape, dtype=mindspore.float32)[source]¶ Returns a new tensor of given shape and type, filled with ones.
- Parameters
- Returns
Tensor, with the designated shape and dtype, filled with ones.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.ones((2,2))) [[1. 1.] [1. 1.]]
-
tinyms.ravel(x)[source]¶ Returns a contiguous flattened tensor.
A 1-D tensor, containing the elements of the input, is returned.
- Parameters
x (Tensor) – A tensor to be flattened.
- Returns
Flattened tensor, has the same data type as the original tensor x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.ones((2,3,4)) >>> output = np.ravel(x) >>> print(output.shape) (24,)
-
tinyms.reshape(x, new_shape)[source]¶ Reshapes a tensor without changing its data.
- Parameters
x (Tensor) – A tensor to be reshaped.
new_shape (Union[int, list(int), tuple(int)]) – The new shape should be compatible with the original shape. If the tuple has only one element, the result will be a 1-D tensor of that length. One shape dimension can be -1. In this case, the value is inferred from the length of the tensor and remaining dimensions.
- Returns
Reshaped Tensor. Has the same data type as the original tensor x.
- Raises
TypeError – If new_shape is not integer, list or tuple.
ValueError – If new_shape does not compatible with the original shape.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.asarray([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]) >>> output = np.reshape(x, (3, 2)) >>> print(output) [[-0.1 0.3] [ 3.6 0.4] [ 0.5 -3.2]] >>> output = np.reshape(x, (3, -1)) >>> print(output) [[-0.1 0.3] [ 3.6 0.4] [ 0.5 -3.2]] >>> output = np.reshape(x, (6, )) >>> print(output) [-0.1 0.3 3.6 0.4 0.5 -3.2]
-
tinyms.rollaxis(x, axis, start=0)[source]¶ Rolls the specified axis backwards, until it lies in the given position. The positions of the other axes do not change relative to one another.
- Parameters
x (Tensor) – A Tensor to be transposed.
axis (int) – The axis to be rolled.
start (int) –
- When start >= 0:
When start <= axis: the axis is rolled back until it lies in this position (start).
When start > axis: the axis is rolled until it lies before this position (start).
- When start < 0: the start will be normalized as follows:
start ……….. Normalized start -(x.ndim+1) raise ValueError -x.ndim 0 … … -1 x.ndim-1 0 0 … … x.ndim x.ndim x.ndim+1 raise ValueError
- Returns
Transposed Tensor. Has the same data type as the original tensor x.
- Supported Platforms:
AscendGPUCPU
- Raises
TypeError – If axis or start is not integer.
ValueError – If axis is not in the range from -ndim to ndim-1 or start is not in the range from -ndim to ndim.
Examples
>>> import mindspore.numpy as np >>> x = np.ones((2,3,4)) >>> output = np.rollaxis(x, 0, 2) >>> print(output.shape) (3, 2, 4)
-
tinyms.squeeze(a, axis=None)[source]¶ Removes single-dimensional entries from the shape of an tensor.
This is a temporary solution to support CPU backend. Will be changed once CPU backend supports P.Squeeze().
- Parameters
a (Tensor) – Input tensor array.
axis – Union[None, int, list(int), tuple(list)]. Default is None.
- Returns
Tensor, with all or a subset of the dimensions of length 1 removed.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.ones((1,2,2,1)) >>> x = np.squeeze(x) >>> print(x.shape) (2, 2)
-
tinyms.swapaxes(x, axis1, axis2)[source]¶ Interchanges two axes of a tensor.
- Parameters
- Returns
Transposed tensor, has the same data type as the original tensor x.
- Raises
TypeError – If axis1 or axis2 is not integer.
ValueError – If axis1 or axis2 is not in the range from -ndim to ndim-1.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.ones((2,3,4)) >>> output = np.swapaxes(x, 0, 2) >>> print(output.shape) (4,3,2)
-
tinyms.transpose(a, axes=None)[source]¶ Reverses or permutes the axes of a tensor; returns the modified tensor.
- Parameters
- Returns
Tensor, the transposed tensor array.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.ones((1,2,3)) >>> x = np.transpose(x) >>> print(x.shape) (3, 2, 1)
-
tinyms.zeros(shape, dtype=mindspore.float32)[source]¶ Returns a new tensor of given shape and type, filled with zeros.
- Parameters
- Returns
Tensor, with the designated shape and dtype, filled with zeros.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.zeros((2,2))) [[0. 0.] [0. 0.]]
tinyms.context¶
The context of TinyMS, used to configure the current execution environment, includeing the execution mode, execution backend and other feature switches.
-
tinyms.context.set_context(**kwargs)[source]¶ Sets context for running environment.
Context should be configured before running your program. If there is no configuration, the “Ascend” device target will be used by default. GRAPH_MODE or PYNATIVE_MODE can be set by mode attribute and both modes support all backends, default mode is PYNATIVE_MODE.
When the save_graphs attribute is set to True, attribute of save_graphs_path is used to set the intermediate compilation graph storage path. By default, the graphs are saved in the current directory. For other configurations and arguments, please refer to the corresponding module description, the configuration is optional and can be enabled when needed.
Note
Attribute name is required for setting attributes. The mode is not recommended to be changed after net was initilized because the implementations of some operations are different in graph mode and pynative mode. Default: PYNATIVE_MODE.
Some configurations are device specific, see the bellow table for details:
Common(CPU/GPU/Ascend)
Ascend
GPU
check_bprop
print_file_path
max_device_memory
device_id
enable_dump
enable_graph_kernel
device_target
save_dump_path
enable_sparse
enable_graph_kernel
max_call_depth
enable_reduce_precision
mode
enable_profiling
reserve_class_name_in_scope
profiling_options
save_graphs
variable_memory_max_size
save_graphs_path
- Parameters
mode (int) – Running in GRAPH_MODE(0) or PYNATIVE_MODE(1). Default: PYNATIVE_MODE(1).
device_target (str) – The target device to run, support “Ascend”, “GPU”, and “CPU”. Default: “Ascend”.
device_id (int) – ID of the target device, the value must be in [0, device_num_per_host-1], while device_num_per_host should be no more than 4096. Default: 0.
save_graphs (bool) – Whether to save graphs. Default: False.
save_graphs_path (str) – Path to save graphs. Default: “.”
enable_graph_kernel (bool) – Whether to enable composition of basic primitives. These primitives would be compiled into a fused kernel automatically. Default: False.
reserve_class_name_in_scope (bool) – Whether to save the network class name in the scope. Default: True.
enable_reduce_precision (bool) – Whether to enable precision reduction. Default: True.
enable_dump (bool) – Whether to enable dump. Default: False.
save_dump_path (str) – When the program is executed on Ascend, operators can dump data in this path. The root dump path is configured in /home/HwHiAiUser/ide_daemon/ide_daemon.cfg. So the real dump path is “{configured root dump path}/{save_dump_path}”. Default: “.”.
variable_memory_max_size (str) – Set the maximum size of the variable memory max size. Default: “0GB”.
enable_profiling (bool) – Whether to open profiling. Default: False.
profiling_options (str) –
Set profiling collection options, operators can profiling data here. The values of profiling collection options are as follows, supporting the collection of multiple data.
training_trace: collect iterative trajectory data, that is, the training task and software information of the AI software stack, to achieve performance analysis of the training task, focusing on data enhancement, forward and backward calculation, gradient aggregation update and other related data.
task_trace: collect task trajectory data, that is, the hardware information of the HWTS/AICore of the Ascend 910 processor, and analyze the information of beginning and ending of the task.
op_trace: collect single operator performance data.
The profiling can choose the combination of training_trace, task_trace, training_trace and task_trace combination, and separated by colons; a single operator can choose op_trace, op_trace cannot be combined with training_trace and task_trace. Default: “training_trace”.
check_bprop (bool) – Whether to check bprop. Default: False.
max_device_memory (str) – Sets the maximum memory available for devices. Currently, it is only supported on GPU. The format is “xxGB”. Default: “1024GB”.
print_file_path (str) – The path of saving print data. If this parameter is set, print data is saved to a file by default, and turns off printing to the screen. If the file already exists, add a timestamp suffix to the file. Default: ‘’.
enable_sparse (bool) – Whether to enable sparsity feature. Default: False.
max_call_depth (int) – Specify the maximum depth of function call. Default: 1000.
- Raises
ValueError – If input key is not an attribute in context.
Examples
>>> context.set_context(mode=context.GRAPH_MODE) >>> context.set_context(mode=context.PYNATIVE_MODE) >>> context.set_context(device_target="Ascend") >>> context.set_context(device_id=0) >>> context.set_context(save_graphs=True, save_graphs_path="./model.ms") >>> context.set_context(enable_reduce_precision=True) >>> context.set_context(enable_dump=True, save_dump_path=".") >>> context.set_context(reserve_class_name_in_scope=True) >>> context.set_context(variable_memory_max_size="6GB") >>> context.set_context(mode=context.GRAPH_MODE, ... device_target="Ascend",device_id=0, save_graphs=True, ... save_graphs_path="/mindspore") >>> context.set_context(enable_profiling=True, profiling_options="training_trace") >>> context.set_context(max_device_memory="3.5GB") >>> context.set_context(print_file_path="print.pb") >>> context.set_context(max_call_depth=80)
-
tinyms.context.get_context(attr_key)[source]¶ Gets context attribute value according to the input key.
- Parameters
attr_key (str) – The key of the attribute.
- Returns
Object, The value of given attribute key.
- Raises
ValueError – If input key is not an attribute in context.
-
tinyms.context.set_auto_parallel_context(**kwargs)[source]¶ Set auto parallel context, which is valid only for Ascend and GPU target.
Auto parallel context should be configured before the initialization of your network.
Note
Attribute name is required for setting attributes. If a program has tasks with different parallel modes, then before setting new parallel mode for the next task, interface mindspore.context.reset_auto_parallel_context() needs to be called to reset the configuration. Setting or changing parallel modes must be called before any creating Initializer, otherwise, RuntimeError may be raised when compiling the network.
Some configurations are parallel mode specific, see the below table for details:
Common
AUTO_PARALLEL
device_num
gradient_fp32_sync
global_rank
loss_repeated_mean
gradients_mean
auto_parallel_search_mode
parallel_mode
strategy_ckpt_load_file
all_reduce_fusion_config
strategy_ckpt_save_file
enable_parallel_optimizer
full_batch
pipeline_stages
- Parameters
device_num (int) – Available device number, the value must be in [1, 4096]. Default: 1.
global_rank (int) – Global rank id, the value must be in [0, 4095]. Default: 0.
gradients_mean (bool) – Whether to perform mean operator after allreduce of gradients. “stand_alone” do not support gradients_mean. Default: False.
gradient_fp32_sync (bool) – Run allreduce of gradients in fp32. “stand_alone”, “data_parallel” and “hybrid_parallel” do not support gradient_fp32_sync. Default: True.
parallel_mode (str) –
There are five kinds of parallel modes, “stand_alone”, “data_parallel”, “hybrid_parallel”, “semi_auto_parallel” and “auto_parallel”. Default: “stand_alone”.
stand_alone: Only one processor is working.
data_parallel: Distributes the data across different processors.
hybrid_parallel: Achieves data parallelism and model parallelism manually.
semi_auto_parallel: Achieves data parallelism and model parallelism by setting parallel strategies.
auto_parallel: Achieving parallelism automatically.
auto_parallel_search_mode (str) –
There are two kinds of shard strategy search modes, “recursive_programming” and “dynamic_programming”. Default: “dynamic_programming”.
recursive_programming: Recursive programming search mode.
dynamic_programming: Dynamic programming search mode.
parameter_broadcast (bool) – Whether to broadcast parameters before training. Before training, in order to have the same network initialization parameter values for all devices, broadcast the parameters on device 0 to other devices. Parameter broadcasting in different parallel modes is different, data_parallel mode, all parameters are broadcast except for the prameter whose attribute layerwise_parallel is True. Hybrid_parallel, semi_auto_parallel and auto_parallel mode, the segmented parameters do not participate in broadcasting. Default: False.
strategy_ckpt_load_file (str) – The path to load parallel strategy checkpoint. Default: ‘’
strategy_ckpt_save_file (str) – The path to save parallel strategy checkpoint. Default: ‘’
full_batch (bool) – If you load whole batch datasets in auto_parallel mode, this parameter should be set with True. Default: False.
enable_parallel_optimizer (bool) – This is a developing feature, which shards the weight update computation for data parallel training in the benefit of time and memory saving. Currently, auto and semi auto parallel mode support all optimizers in both Ascend and GPU. Data parallel mode only supports Lamb and AdamWeightDecay in Ascend . Default: False.
all_reduce_fusion_config (list) – Set allreduce fusion strategy by parameters indices. Only support ReduceOp.SUM and HCCL_WORLD_GROUP/NCCL_WORLD_GROUP. No Default, if it is not set, the fusion is closed.
pipeline_stages (int) – Set the stage information for pipeline parallel. This indicates how the devices are distributed alone the pipeline. The total devices will be divided into ‘pipeline_stags’ stages. This currently could only be used when parallel mode semi_auto_parallel is enabled. Default: 1.
- Raises
ValueError – If input key is not attribute in auto parallel context.
Examples
>>> context.set_auto_parallel_context(device_num=8) >>> context.set_auto_parallel_context(global_rank=0) >>> context.set_auto_parallel_context(gradients_mean=True) >>> context.set_auto_parallel_context(gradient_fp32_sync=False) >>> context.set_auto_parallel_context(parallel_mode="auto_parallel") >>> context.set_auto_parallel_context(auto_parallel_search_mode="dynamic_programming") >>> context.set_auto_parallel_context(parameter_broadcast=False) >>> context.set_auto_parallel_context(strategy_ckpt_load_file="./strategy_stage1.ckpt") >>> context.set_auto_parallel_context(strategy_ckpt_save_file="./strategy_stage1.ckpt") >>> context.set_auto_parallel_context(full_batch=True) >>> context.set_auto_parallel_context(enable_parallel_optimizer=False) >>> context.set_auto_parallel_context(all_reduce_fusion_config=[8, 160]) >>> context.set_auto_parallel_context(pipeline_stages=2)
-
tinyms.context.get_auto_parallel_context(attr_key)[source]¶ Gets auto parallel context attribute value according to the key.
- Parameters
attr_key (str) – The key of the attribute.
- Returns
Returns attribute value according to the key.
- Raises
ValueError – If input key is not attribute in auto parallel context.
-
tinyms.context.reset_auto_parallel_context()[source]¶ Reset auto parallel context attributes to the default values:
device_num: 1.
global_rank: 0.
gradients_mean: False.
gradient_fp32_sync: True.
parallel_mode: ‘stand_alone’.
auto_parallel_search_mode: ‘dynamic_programming’.
parameter_broadcast: False.
strategy_ckpt_load_file: ‘’.
strategy_ckpt_save_file: ‘’.
full_batch: False.
enable_parallel_optimizer: False.
pipeline_stages: 1.
-
class
tinyms.context.ParallelMode[source]¶ Parallel mode options.
There are five kinds of parallel modes, “STAND_ALONE”, “DATA_PARALLEL”, “HYBRID_PARALLEL”, “SEMI_AUTO_PARALLEL” and “AUTO_PARALLEL”. Default: “STAND_ALONE”.
STAND_ALONE: Only one processor is working.
DATA_PARALLEL: Distributes the data across different processors.
HYBRID_PARALLEL: Achieves data parallelism and model parallelism manually.
SEMI_AUTO_PARALLEL: Achieves data parallelism and model parallelism by setting parallel strategies.
AUTO_PARALLEL: Achieves parallelism automatically.
MODE_LIST: The list of all supported parallel modes.
-
tinyms.context.set_ps_context(**kwargs)[source]¶ Set parameter server training mode context.
Note
Some other environment variables should also be set for parameter server training mode. These environment variables are listed below:
MS_SERVER_NUM # Server number MS_WORKER_NUM # Worker number MS_SCHED_HOST # Scheduler IP address MS_SCHED_PORT # Scheduler port MS_ROLE # The role of this process: # MS_SCHED represents the scheduler, # MS_WORKER represents the worker, # MS_PSERVER represents the Server
- Parameters
enable_ps (bool) – Whether to enable parameter server training mode. Only after enable_ps is set True, the environment variables will be effective. Default: False.
- Raises
ValueError – If input key is not the attribute in parameter server training mode context.
Examples
>>> context.set_ps_context(enable_ps=True)
-
tinyms.context.get_ps_context(attr_key)[source]¶ Get parameter server training mode context attribute value according to the key.
- Parameters
attr_key (str) – The key of the attribute.
- Returns
Returns attribute value according to the key.
- Raises
ValueError – If input key is not attribute in auto parallel context.
tinyms.data¶
-
class
tinyms.data.UnalignedDataset(dataset_path, phase, max_dataset_size=inf, shuffle=True)[source]¶ This dataset class can load unaligned/unpaired datasets.
- Parameters
dataset_path (str) – The path of images (should have subfolders trainA, trainB, testA, testB, etc).
phase (str) – Train or test. It requires two directories in dataset_path, like trainA and trainB to. host training images from domain A ‘{dataset_path}/trainA’ and from domain B ‘{dataset_path}/trainB’ respectively.
max_dataset_size (int) – Maximum number of return image paths.
- Returns
Two domain image path list.
-
class
tinyms.data.GanImageFolderDataset(dataset_path, max_dataset_size=inf)[source]¶ This dataset class can load images from image folder.
-
class
tinyms.data.DistributedSampler(dataset_size, num_replicas=None, rank=None, shuffle=True)[source]¶ Distributed sampler.
-
class
tinyms.data.CelebADataset(dataset_dir, num_parallel_workers=None, shuffle=None, usage='all', sampler=None, decode=False, extensions=None, num_samples=None, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset for reading and parsing CelebA dataset. Currently supported: list_attr_celeba.txt only.
Note
The generated dataset has two columns [‘image’, ‘attr’]. The type of the image tensor is uint8. The attribute tensor is uint32 and one hot type.
Citation of CelebA dataset.
@article{DBLP:journals/corr/LiuLWT14, author = {Ziwei Liu and Ping Luo and Xiaogang Wang and Xiaoou Tang}, title = {Deep Learning Face Attributes in the Wild}, journal = {CoRR}, volume = {abs/1411.7766}, year = {2014}, url = {http://arxiv.org/abs/1411.7766}, archivePrefix = {arXiv}, eprint = {1411.7766}, timestamp = {Tue, 10 Dec 2019 15:37:26 +0100}, biburl = {https://dblp.org/rec/journals/corr/LiuLWT14.bib}, bibsource = {dblp computer science bibliography, https://dblp.org}, howpublished = {http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html}, description = {CelebFaces Attributes Dataset (CelebA) is a large-scale face attributes dataset with more than 200K celebrity images, each with 40 attribute annotations. The images in this dataset cover large pose variations and background clutter. CelebA has large diversities, large quantities, and rich annotations, including * 10,177 number of identities, * 202,599 number of face images, and * 5 landmark locations, 40 binary attributes annotations per image. The dataset can be employed as the training and test sets for the following computer vision tasks: face attribute recognition, face detection, landmark (or facial part) localization, and face editing & synthesis.} }
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
num_parallel_workers (int, optional) – Number of workers to read the data (default=value set in the config).
shuffle (bool, optional) – Whether to perform shuffle on the dataset (default=None).
usage (str) – one of ‘all’, ‘train’, ‘valid’ or ‘test’.
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None).
decode (bool, optional) – decode the images after reading (default=False).
extensions (list[str], optional) – List of file extensions to be included in the dataset (default=None).
num_samples (int, optional) – The number of images to be included in the dataset. (default=None, all images).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None).
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None which means no cache is used).
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_dir = "/path/to/celeba_directory" >>> dataset = ds.CelebADataset(dataset_dir=dataset_dir, usage='train')
-
class
tinyms.data.Cifar100Dataset(dataset_dir, usage=None, num_samples=None, num_parallel_workers=None, shuffle=None, sampler=None, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset that reads cifar100 data.
The generated dataset has three columns [‘image’, ‘coarse_label’, ‘fine_label’]. The type of the image tensor is uint8. The coarse and fine labels are each a scalar uint32 tensor. This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
Citation of Cifar100 dataset.
@techreport{Krizhevsky09, author = {Alex Krizhevsky}, title = {Learning multiple layers of features from tiny images}, institution = {}, year = {2009}, howpublished = {http://www.cs.toronto.edu/~kriz/cifar.html}, description = {This dataset is just like the CIFAR-10, except it has 100 classes containing 600 images each. There are 500 training images and 100 testing images per class. The 100 classes in the CIFAR-100 are grouped into 20 superclasses. Each image comes with a "fine" label (the class to which it belongs) and a "coarse" label (the superclass to which it belongs).} }
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
usage (str, optional) – Usage of this dataset, can be “train”, “test” or “all” . “train” will read from 50,000 train samples, “test” will read from 10,000 test samples, “all” will read from all 60,000 samples. (default=None, all samples)
num_samples (int, optional) – The number of images to be included in the dataset. (default=None, all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (bool, optional) – Whether to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None).
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None which means no cache is used).
- Raises
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_dir = "/path/to/cifar100_dataset_directory" >>> >>> # 1) Get all samples from CIFAR100 dataset in sequence >>> cifar100_dataset = ds.Cifar100Dataset(dataset_dir=dataset_dir, shuffle=False) >>> >>> # 2) Randomly select 350 samples from CIFAR100 dataset >>> cifar100_dataset = ds.Cifar100Dataset(dataset_dir=dataset_dir, num_samples=350, shuffle=True) >>> >>> # In CIFAR100 dataset, each dictionary has 3 keys: "image", "fine_label" and "coarse_label"
-
class
tinyms.data.Cifar10Dataset(dataset_dir, usage=None, num_samples=None, num_parallel_workers=None, shuffle=None, sampler=None, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset that reads cifar10 data.
The generated dataset has two columns [‘image’, ‘label’]. The type of the image tensor is uint8. The label is a scalar uint32 tensor. This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
Citation of Cifar10 dataset.
@techreport{Krizhevsky09, author = {Alex Krizhevsky}, title = {Learning multiple layers of features from tiny images}, institution = {}, year = {2009}, howpublished = {http://www.cs.toronto.edu/~kriz/cifar.html}, description = {The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.} }
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
usage (str, optional) – Usage of this dataset, can be “train”, “test” or “all” . “train” will read from 50,000 train samples, “test” will read from 10,000 test samples, “all” will read from all 60,000 samples. (default=None, all samples)
num_samples (int, optional) – The number of images to be included in the dataset. (default=None, all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (bool, optional) – Whether to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None).
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None which means no cache is used).
- Raises
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_dir = "/path/to/cifar10_dataset_directory" >>> >>> # 1) Get all samples from CIFAR10 dataset in sequence >>> dataset = ds.Cifar10Dataset(dataset_dir=dataset_dir, shuffle=False) >>> >>> # 2) Randomly select 350 samples from CIFAR10 dataset >>> dataset = ds.Cifar10Dataset(dataset_dir=dataset_dir, num_samples=350, shuffle=True) >>> >>> # 3) Get samples from CIFAR10 dataset for shard 0 in a 2-way distributed training >>> dataset = ds.Cifar10Dataset(dataset_dir=dataset_dir, num_shards=2, shard_id=0) >>> >>> # In CIFAR10 dataset, each dictionary has keys "image" and "label"
-
class
tinyms.data.CLUEDataset(dataset_files, task='AFQMC', usage='train', num_samples=None, num_parallel_workers=None, shuffle=<Shuffle.GLOBAL: 'global'>, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset that reads and parses CLUE datasets. CLUE, the Chinese Language Understanding Evaluation Benchmark, is a collection of datasets, baselines, pre-trained models, corpus and leaderboard. Supported CLUE classification tasks: ‘AFQMC’, ‘TNEWS’, ‘IFLYTEK’, ‘CMNLI’, ‘WSC’ and ‘CSL’.
Citation of CLUE dataset.
@article{CLUEbenchmark, title = {CLUE: A Chinese Language Understanding Evaluation Benchmark}, author = {Liang Xu, Xuanwei Zhang, Lu Li, Hai Hu, Chenjie Cao, Weitang Liu, Junyi Li, Yudong Li, Kai Sun, Yechen Xu, Yiming Cui, Cong Yu, Qianqian Dong, Yin Tian, Dian Yu, Bo Shi, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Zhenzhong Lan}, journal = {arXiv preprint arXiv:2004.05986}, year = {2020}, howpublished = {https://github.com/CLUEbenchmark/CLUE}, description = {CLUE, a Chinese Language Understanding Evaluation benchmark. It contains eight different tasks, including single-sentence classification, sentence pair classification, and machine reading comprehension.} }
- Parameters
dataset_files (Union[str, list[str]]) – String or list of files to be read or glob strings to search for a pattern of files. The list will be sorted in a lexicographical order.
task (str, optional) – The kind of task, one of ‘AFQMC’, ‘TNEWS’, ‘IFLYTEK’, ‘CMNLI’, ‘WSC’ and ‘CSL’. (default=AFQMC).
usage (str, optional) – Need train, test or eval data (default=”train”).
num_samples (int, optional) – Number of samples (rows) to read (default=None, reads the full dataset).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (Union[bool, Shuffle level], optional) –
Perform reshuffling of the data every epoch (default=Shuffle.GLOBAL). If shuffle is False, no shuffling will be performed; If shuffle is True, the behavior is the same as setting shuffle to be Shuffle.GLOBAL Otherwise, there are two levels of shuffling:
Shuffle.GLOBAL: Shuffle both the files and samples.
Shuffle.FILES: Shuffle files only.
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None).
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None which means no cache is used).
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_files = ["/path/to/1", "/path/to/2"] # contains 1 or multiple text files >>> dataset = ds.CLUEDataset(dataset_files=dataset_files, task='AFQMC', usage='train')
-
class
tinyms.data.CocoDataset(dataset_dir, annotation_file, task='Detection', num_samples=None, num_parallel_workers=None, shuffle=None, decode=False, sampler=None, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset for reading and parsing COCO dataset.
CocoDataset support four kinds of task: 2017 Train/Val/Test Detection, Keypoints, Stuff, Panoptic.
The generated dataset has multi-columns :
task=’Detection’, column: [[‘image’, dtype=uint8], [‘bbox’, dtype=float32], [‘category_id’, dtype=uint32], [‘iscrowd’, dtype=uint32]].
task=’Stuff’, column: [[‘image’, dtype=uint8], [‘segmentation’,dtype=float32], [‘iscrowd’,dtype=uint32]].
task=’Keypoint’, column: [[‘image’, dtype=uint8], [‘keypoints’, dtype=float32], [‘num_keypoints’, dtype=uint32]].
task=’Panoptic’, column: [[‘image’, dtype=uint8], [‘bbox’, dtype=float32], [‘category_id’, dtype=uint32], [‘iscrowd’, dtype=uint32], [‘area’, dtype=uint32]].
This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. CocoDataset doesn’t support PKSampler. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
Citation of Coco dataset.
@article{DBLP:journals/corr/LinMBHPRDZ14, author = {Tsung{-}Yi Lin and Michael Maire and Serge J. Belongie and Lubomir D. Bourdev and Ross B. Girshick and James Hays and Pietro Perona and Deva Ramanan and Piotr Doll{'{a}}r and C. Lawrence Zitnick}, title = {Microsoft {COCO:} Common Objects in Context}, journal = {CoRR}, volume = {abs/1405.0312}, year = {2014}, url = {http://arxiv.org/abs/1405.0312}, archivePrefix = {arXiv}, eprint = {1405.0312}, timestamp = {Mon, 13 Aug 2018 16:48:13 +0200}, biburl = {https://dblp.org/rec/journals/corr/LinMBHPRDZ14.bib}, bibsource = {dblp computer science bibliography, https://dblp.org}, description = {COCO is a large-scale object detection, segmentation, and captioning dataset. It contains 91 common object categories with 82 of them having more than 5,000 labeled instances. In contrast to the popular ImageNet dataset, COCO has fewer categories but more instances per category.} }
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
annotation_file (str) – Path to the annotation JSON.
task (str) – Set the task type for reading COCO data. Supported task types: ‘Detection’, ‘Stuff’, ‘Panoptic’ and ‘Keypoint’ (default=’Detection’).
num_samples (int, optional) – The number of images to be included in the dataset (default=None, all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the configuration file).
shuffle (bool, optional) – Whether to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
decode (bool, optional) – Decode the images after reading (default=False).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None).
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None which means no cache is used).
- Raises
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
RuntimeError – If parse JSON file failed.
ValueError – If task is not in [‘Detection’, ‘Stuff’, ‘Panoptic’, ‘Keypoint’].
ValueError – If annotation_file is not exist.
ValueError – If dataset_dir is not exist.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_dir = "/path/to/coco_dataset_directory/image_folder" >>> annotation_file = "/path/to/coco_dataset_directory/annotation_folder/annotation.json" >>> >>> # 1) Read COCO data for Detection task >>> coco_dataset = ds.CocoDataset(dataset_dir, annotation_file=annotation_file, task='Detection') >>> >>> # 2) Read COCO data for Stuff task >>> coco_dataset = ds.CocoDataset(dataset_dir, annotation_file=annotation_file, task='Stuff') >>> >>> # 3) Read COCO data for Panoptic task >>> coco_dataset = ds.CocoDataset(dataset_dir, annotation_file=annotation_file, task='Panoptic') >>> >>> # 4) Read COCO data for Keypoint task >>> coco_dataset = ds.CocoDataset(dataset_dir, annotation_file=annotation_file, task='Keypoint') >>> >>> # In COCO dataset, each dictionary has keys "image" and "annotation"
-
class
tinyms.data.CSVDataset(dataset_files, field_delim=', ', column_defaults=None, column_names=None, num_samples=None, num_parallel_workers=None, shuffle=<Shuffle.GLOBAL: 'global'>, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset that reads and parses comma-separated values (CSV) datasets.
- Parameters
dataset_files (Union[str, list[str]]) – String or list of files to be read or glob strings to search for a pattern of files. The list will be sorted in a lexicographical order.
field_delim (str, optional) – A string that indicates the char delimiter to separate fields (default=’,’).
column_defaults (list, optional) – List of default values for the CSV field (default=None). Each item in the list is either a valid type (float, int, or string). If this is not provided, treats all columns as string type.
column_names (list[str], optional) – List of column names of the dataset (default=None). If this is not provided, infers the column_names from the first row of CSV file.
num_samples (int, optional) – Number of samples (rows) to read (default=None, reads the full dataset).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (Union[bool, Shuffle level], optional) –
Perform reshuffling of the data every epoch (default=Shuffle.GLOBAL). If shuffle is False, no shuffling will be performed; If shuffle is True, the behavior is the same as setting shuffle to be Shuffle.GLOBAL Otherwise, there are two levels of shuffling:
Shuffle.GLOBAL: Shuffle both the files and samples.
Shuffle.FILES: Shuffle files only.
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None).
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None which means no cache is used).
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_files = ["/path/to/1", "/path/to/2"] # contains 1 or multiple text files >>> dataset = ds.CSVDataset(dataset_files=dataset_files, column_names=['col1', 'col2', 'col3', 'col4'])
-
class
tinyms.data.GeneratorDataset(source, column_names=None, column_types=None, schema=None, num_samples=None, num_parallel_workers=1, shuffle=None, sampler=None, num_shards=None, shard_id=None, python_multiprocessing=True)[source]¶ A source dataset that generates data from Python by invoking Python data source each epoch.
This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
- Parameters
source (Union[Callable, Iterable, Random Accessible]) – A generator callable object, an iterable Python object or a random accessible Python object. Callable source is required to return a tuple of NumPy arrays as a row of the dataset on source().next(). Iterable source is required to return a tuple of NumPy arrays as a row of the dataset on iter(source).next(). Random accessible source is required to return a tuple of NumPy arrays as a row of the dataset on source[idx].
column_names (Union[str, list[str]], optional) – List of column names of the dataset (default=None). Users are required to provide either column_names or schema.
column_types (list[mindspore.dtype], optional) – List of column data types of the dataset (default=None). If provided, sanity check will be performed on generator output.
schema (Union[Schema, str], optional) – Path to the JSON schema file or schema object (default=None). Users are required to provide either column_names or schema. If both are provided, schema will be used.
num_samples (int, optional) – The number of samples to be included in the dataset (default=None, all images).
num_parallel_workers (int, optional) – Number of subprocesses used to fetch the dataset in parallel (default=1).
shuffle (bool, optional) – Whether or not to perform shuffle on the dataset. Random accessible input is required. (default=None, expected order behavior shown in the table).
sampler (Union[Sampler, Iterable], optional) – Object used to choose samples from the dataset. Random accessible input is required (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, ‘num_samples’ will not used. Random accessible input is required.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument must be specified only when num_shards is also specified. Random accessible input is required.
python_multiprocessing (bool, optional) – Parallelize Python operations with multiple worker process. This option could be beneficial if the Python operation is computational heavy (default=True).
Examples
>>> import mindspore.dataset as ds >>> >>> # 1) Multidimensional generator function as callable input >>> def GeneratorMD(): >>> for i in range(64): >>> yield (np.array([[i, i + 1], [i + 2, i + 3]]),) >>> # Create multi_dimension_generator_dataset with GeneratorMD and column name "multi_dimensional_data" >>> multi_dimension_generator_dataset = ds.GeneratorDataset(GeneratorMD, ["multi_dimensional_data"]) >>> >>> # 2) Multi-column generator function as callable input >>> def GeneratorMC(maxid = 64): >>> for i in range(maxid): >>> yield (np.array([i]), np.array([[i, i + 1], [i + 2, i + 3]])) >>> # Create multi_column_generator_dataset with GeneratorMC and column names "col1" and "col2" >>> multi_column_generator_dataset = ds.GeneratorDataset(GeneratorMC, ["col1", "col2"]) >>> >>> # 3) Iterable dataset as iterable input >>> class MyIterable(): >>> def __iter__(self): >>> return # User implementation >>> # Create iterable_generator_dataset with MyIterable object >>> iterable_generator_dataset = ds.GeneratorDataset(MyIterable(), ["col1"]) >>> >>> # 4) Random accessible dataset as random accessible input >>> class MyRA(): >>> def __getitem__(self, index): >>> return # User implementation >>> # Create ra_generator_dataset with MyRA object >>> ra_generator_dataset = ds.GeneratorDataset(MyRA(), ["col1"]) >>> # List/Dict/Tuple is also random accessible >>> list_generator = ds.GeneratorDataset([(np.array(0),), (np.array(1)), (np.array(2))], ["col1"]) >>> >>> # 5) Built-in Sampler >>> my_generator = ds.GeneratorDataset(my_ds, ["img", "label"], sampler=samplers.RandomSampler())
-
class
tinyms.data.GraphData(dataset_file, num_parallel_workers=None, working_mode='local', hostname='127.0.0.1', port=50051, num_client=1, auto_shutdown=True)[source]¶ Reads the graph dataset used for GNN training from the shared file and database.
- Parameters
dataset_file (str) – One of file names in the dataset.
num_parallel_workers (int, optional) – Number of workers to process the dataset in parallel (default=None).
working_mode (str, optional) –
Set working mode, now supports ‘local’/’client’/’server’ (default=’local’).
’local’, used in non-distributed training scenarios.
’client’, used in distributed training scenarios. The client does not load data, but obtains data from the server.
’server’, used in distributed training scenarios. The server loads the data and is available to the client.
hostname (str, optional) – Hostname of the graph data server. This parameter is only valid when working_mode is set to ‘client’ or ‘server’ (default=’127.0.0.1’).
port (int, optional) – Port of the graph data server. The range is 1024-65535. This parameter is only valid when working_mode is set to ‘client’ or ‘server’ (default=50051).
num_client (int, optional) – Maximum number of clients expected to connect to the server. The server will allocate resources according to this parameter. This parameter is only valid when working_mode is set to ‘server’ (default=1).
auto_shutdown (bool, optional) – Valid when working_mode is set to ‘server’, when the number of connected clients reaches num_client and no client is being connected, the server automatically exits (default=True).
Examples
>>> import mindspore.dataset as ds >>> >>> data_graph = ds.GraphData('dataset_file', 2) >>> nodes = data_graph.get_all_nodes(0) >>> features = data_graph.get_node_feature(nodes, [1])
-
get_all_edges(edge_type)[source]¶ Get all edges in the graph.
- Parameters
edge_type (int) – Specify the type of edge.
- Returns
numpy.ndarray, array of edges.
Examples
>>> import mindspore.dataset as ds >>> >>> data_graph = ds.GraphData('dataset_file', 2) >>> nodes = data_graph.get_all_edges(0)
- Raises
TypeError – If edge_type is not integer.
-
get_all_neighbors(node_list, neighbor_type)[source]¶ Get neighbor_type neighbors of the nodes in node_list.
- Parameters
node_list (Union[list, numpy.ndarray]) – The given list of nodes.
neighbor_type (int) – Specify the type of neighbor.
- Returns
numpy.ndarray, array of neighbors.
Examples
>>> import mindspore.dataset as ds >>> >>> data_graph = ds.GraphData('dataset_file', 2) >>> nodes = data_graph.get_all_nodes(0) >>> neighbors = data_graph.get_all_neighbors(nodes, 0)
-
get_all_nodes(node_type)[source]¶ Get all nodes in the graph.
- Parameters
node_type (int) – Specify the type of node.
- Returns
numpy.ndarray, array of nodes.
Examples
>>> import mindspore.dataset as ds >>> >>> data_graph = ds.GraphData('dataset_file', 2) >>> nodes = data_graph.get_all_nodes(0)
- Raises
TypeError – If node_type is not integer.
-
get_edge_feature(edge_list, feature_types)[source]¶ Get feature_types feature of the edges in edge_list.
- Parameters
edge_list (Union[list, numpy.ndarray]) – The given list of edges.
feature_types (Union[list, numpy.ndarray]) – The given list of feature types.
- Returns
numpy.ndarray, array of features.
Examples
>>> import mindspore.dataset as ds >>> >>> data_graph = ds.GraphData('dataset_file', 2) >>> edges = data_graph.get_all_edges(0) >>> features = data_graph.get_edge_feature(edges, [1])
-
get_neg_sampled_neighbors(node_list, neg_neighbor_num, neg_neighbor_type)[source]¶ Get neg_neighbor_type negative sampled neighbors of the nodes in node_list.
- Parameters
node_list (Union[list, numpy.ndarray]) – The given list of nodes.
neg_neighbor_num (int) – Number of neighbors sampled.
neg_neighbor_type (int) – Specify the type of negative neighbor.
- Returns
numpy.ndarray, array of neighbors.
Examples
>>> import mindspore.dataset as ds >>> >>> data_graph = ds.GraphData('dataset_file', 2) >>> nodes = data_graph.get_all_nodes(0) >>> neg_neighbors = data_graph.get_neg_sampled_neighbors(nodes, 5, 0)
-
get_node_feature(node_list, feature_types)[source]¶ Get feature_types feature of the nodes in node_list.
- Parameters
node_list (Union[list, numpy.ndarray]) – The given list of nodes.
feature_types (Union[list, numpy.ndarray]) – The given list of feature types.
- Returns
numpy.ndarray, array of features.
Examples
>>> import mindspore.dataset as ds >>> >>> data_graph = ds.GraphData('dataset_file', 2) >>> nodes = data_graph.get_all_nodes(0) >>> features = data_graph.get_node_feature(nodes, [1])
-
get_nodes_from_edges(edge_list)[source]¶ Get nodes from the edges.
- Parameters
edge_list (Union[list, numpy.ndarray]) – The given list of edges.
- Returns
numpy.ndarray, array of nodes.
- Raises
TypeError – If edge_list is not list or ndarray.
-
get_sampled_neighbors(node_list, neighbor_nums, neighbor_types)[source]¶ Get sampled neighbor information.
The api supports multi-hop neighbor sampling. That is, the previous sampling result is used as the input of next-hop sampling. A maximum of 6-hop are allowed.
The sampling result is tiled into a list in the format of [input node, 1-hop sampling result, 2-hop samling result …]
- Parameters
node_list (Union[list, numpy.ndarray]) – The given list of nodes.
neighbor_nums (Union[list, numpy.ndarray]) – Number of neighbors sampled per hop.
neighbor_types (Union[list, numpy.ndarray]) – Neighbor type sampled per hop.
- Returns
numpy.ndarray, array of neighbors.
Examples
>>> import mindspore.dataset as ds >>> >>> data_graph = ds.GraphData('dataset_file', 2) >>> nodes = data_graph.get_all_nodes(0) >>> neighbors = data_graph.get_sampled_neighbors(nodes, [2, 2], [0, 0])
-
graph_info()[source]¶ Get the meta information of the graph, including the number of nodes, the type of nodes, the feature information of nodes, the number of edges, the type of edges, and the feature information of edges.
- Returns
dict, meta information of the graph. The key is node_type, edge_type, node_num, edge_num, node_feature_type and edge_feature_type.
-
random_walk(target_nodes, meta_path, step_home_param=1.0, step_away_param=1.0, default_node=-1)[source]¶ Random walk in nodes.
- Parameters
step_home_param (float, optional) – return hyper parameter in node2vec algorithm (Default = 1.0).
step_away_param (float, optional) – inout hyper parameter in node2vec algorithm (Default = 1.0).
default_node (int, optional) – default node if no more neighbors found (Default = -1). A default value of -1 indicates that no node is given.
- Returns
numpy.ndarray, array of nodes.
Examples
>>> import mindspore.dataset as ds >>> >>> data_graph = ds.GraphData('dataset_file', 2) >>> nodes = data_graph.random_walk([1,2], [1,2,1,2,1])
-
class
tinyms.data.ImageFolderDataset(dataset_dir, num_samples=None, num_parallel_workers=None, shuffle=None, sampler=None, extensions=None, class_indexing=None, decode=False, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset that reads images from a tree of directories.
All images within one folder have the same label. The generated dataset has two columns [‘image’, ‘label’]. The shape of the image column is [image_size] if decode flag is False, or [H,W,C] otherwise. The type of the image tensor is uint8. The label is a scalar int32 tensor. This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
num_samples (int, optional) – The number of images to be included in the dataset (default=None, all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, set in the config).
shuffle (bool, optional) – Whether or not to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
extensions (list[str], optional) – List of file extensions to be included in the dataset (default=None).
class_indexing (dict, optional) – A str-to-int mapping from folder name to index (default=None, the folder names will be sorted alphabetically and each class will be given a unique index starting from 0).
decode (bool, optional) – Decode the images after reading (default=False).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None).
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None which means no cache is used).
- Raises
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
RuntimeError – If class_indexing is not a dictionary.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> import mindspore.dataset as ds >>> >>> # Set path to the imagefolder directory. >>> # This directory needs to contain sub-directories which contain the images >>> dataset_dir = "/path/to/imagefolder_directory" >>> >>> # 1) Read all samples (image files) in dataset_dir with 8 threads >>> imagefolder_dataset = ds.ImageFolderDataset(dataset_dir, num_parallel_workers=8) >>> >>> # 2) Read all samples (image files) from folder cat and folder dog with label 0 and 1 >>> imagefolder_dataset = ds.ImageFolderDataset(dataset_dir, class_indexing={"cat":0, "dog":1}) >>> >>> # 3) Read all samples (image files) in dataset_dir with extensions .JPEG and .png (case sensitive) >>> imagefolder_dataset = ds.ImageFolderDataset(dataset_dir, extensions=[".JPEG", ".png"])
-
class
tinyms.data.ManifestDataset(dataset_file, usage='train', num_samples=None, num_parallel_workers=None, shuffle=None, sampler=None, class_indexing=None, decode=False, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset that reads images from a manifest file.
The generated dataset has two columns [‘image’, ‘label’]. The shape of the image column is [image_size] if decode flag is False, or [H,W,C] otherwise. The type of the image tensor is uint8. The label is a scalar uint64 tensor. This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
- Parameters
dataset_file (str) – File to be read.
usage (str, optional) – acceptable usages include train, eval and inference (default=”train”).
num_samples (int, optional) – The number of images to be included in the dataset. (default=None, all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (bool, optional) – Whether to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
class_indexing (dict, optional) – A str-to-int mapping from label name to index (default=None, the folder names will be sorted alphabetically and each class will be given a unique index starting from 0).
decode (bool, optional) – decode the images after reading (default=False).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None).
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None which means no cache is used).
- Raises
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
RuntimeError – If class_indexing is not a dictionary.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_file = "/path/to/manifest_file.manifest" >>> >>> # 1) Read all samples specified in manifest_file dataset with 8 threads for training >>> manifest_dataset = ds.ManifestDataset(dataset_file, usage="train", num_parallel_workers=8) >>> >>> # 2) Read samples (specified in manifest_file.manifest) for shard 0 >>> # in a 2-way distributed training setup >>> manifest_dataset = ds.ManifestDataset(dataset_file, num_shards=2, shard_id=0)
-
class
tinyms.data.MindDataset(dataset_file, columns_list=None, num_parallel_workers=None, shuffle=None, num_shards=None, shard_id=None, sampler=None, padded_sample=None, num_padded=None, num_samples=None)[source]¶ A source dataset that reads MindRecord files.
- Parameters
dataset_file (Union[str, list[str]]) – If dataset_file is a str, it represents for a file name of one component of a mindrecord source, other files with identical source in the same path will be found and loaded automatically. If dataset_file is a list, it represents for a list of dataset files to be read directly.
columns_list (list[str], optional) – List of columns to be read (default=None).
num_parallel_workers (int, optional) – The number of readers (default=None).
shuffle (bool, optional) – Whether or not to perform shuffle on the dataset (default=None, performs shuffle).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None).
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, sampler is exclusive with shuffle and block_reader). Support list: SubsetRandomSampler, PkSampler, RandomSampler, SequentialSampler, DistributedSampler.
padded_sample (dict, optional) – Samples will be appended to dataset, where keys are the same as column_list.
num_padded (int, optional) – Number of padding samples. Dataset size plus num_padded should be divisible by num_shards.
num_samples (int, optional) – The number of samples to be included in the dataset (default=None, all samples).
- Raises
ValueError – If num_shards is specified but shard_id is None.
ValueError – If shard_id is specified but num_shards is None.
-
class
tinyms.data.MnistDataset(dataset_dir, usage=None, num_samples=None, num_parallel_workers=None, shuffle=None, sampler=None, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset for reading and parsing the MNIST dataset.
The generated dataset has two columns [‘image’, ‘label’]. The type of the image tensor is uint8. The label is a scalar uint32 tensor. This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
Citation of Mnist dataset.
@article{lecun2010mnist, title = {MNIST handwritten digit database}, author = {LeCun, Yann and Cortes, Corinna and Burges, CJ}, journal = {ATT Labs [Online]}, volume = {2}, year = {2010}, howpublished = {http://yann.lecun.com/exdb/mnist}, description = {The MNIST database of handwritten digits has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image.} }
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
usage (str, optional) – Usage of this dataset, can be “train”, “test” or “all” . “train” will read from 60,000 train samples, “test” will read from 10,000 test samples, “all” will read from all 70,000 samples. (default=None, all samples)
num_samples (int, optional) – The number of images to be included in the dataset (default=None, all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, set in the config).
shuffle (bool, optional) – Whether or not to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None).
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None which means no cache is used).
- Raises
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_dir = "/path/to/mnist_folder" >>> # Read 3 samples from MNIST dataset >>> mnist_dataset = ds.MnistDataset(dataset_dir=dataset_dir, num_samples=3) >>> # Note: In mnist_dataset dataset, each dictionary has keys "image" and "label"
-
class
tinyms.data.NumpySlicesDataset(data, column_names=None, num_samples=None, num_parallel_workers=1, shuffle=None, sampler=None, num_shards=None, shard_id=None)[source]¶ Create a dataset with given data slices, mainly for loading Python data into dataset.
This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
- Parameters
data (Union[list, tuple, dict]) – list, tuple, dict and other NumPy formats. Input data will be sliced along the first dimension and generate additional rows, if input is list, there will be one column in each row, otherwise there tends to be multi columns. Large data is not recommended to be loaded in this way as data is loading into memory.
column_names (list[str], optional) – List of column names of the dataset (default=None). If column_names is not provided, when data is dict, column_names will be its keys, otherwise it will be like column_0, column_1 …
num_samples (int, optional) – The number of samples to be included in the dataset (default=None, all images).
num_parallel_workers (int, optional) – Number of subprocesses used to fetch the dataset in parallel (default=1).
shuffle (bool, optional) – Whether or not to perform shuffle on the dataset. Random accessible input is required. (default=None, expected order behavior shown in the table).
sampler (Union[Sampler, Iterable], optional) – Object used to choose samples from the dataset. Random accessible input is required (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, ‘num_samples’ will not used. Random accessible input is required.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument must be specified only when num_shards is also specified. Random accessible input is required.
Examples
>>> import mindspore.dataset as ds >>> >>> # 1) Input data can be a list >>> data = [1, 2, 3] >>> dataset1 = ds.NumpySlicesDataset(data, column_names=["column_1"]) >>> >>> # 2) Input data can be a dictionary, and column_names will be its keys >>> data = {"a": [1, 2], "b": [3, 4]} >>> dataset2 = ds.NumpySlicesDataset(data) >>> >>> # 3) Input data can be a tuple of lists (or NumPy arrays), each tuple element refers to data in each column >>> data = ([1, 2], [3, 4], [5, 6]) >>> dataset3 = ds.NumpySlicesDataset(data, column_names=["column_1", "column_2", "column_3"]) >>> >>> # 4) Load data from CSV file >>> import pandas as pd >>> df = pd.read_csv("file.csv") >>> dataset4 = ds.NumpySlicesDataset(dict(df), shuffle=False)
-
class
tinyms.data.PaddedDataset(padded_samples)[source]¶ Create a dataset with fake data provided by user. Mainly used to add to the original data set and assign it to the corresponding shard.
- Parameters
- Raises
TypeError – If padded_samples is not an instance of list.
TypeError – If the element of padded_samples is not an instance of dict.
ValueError – If the padded_samples is empty.
Examples
>>> import mindspore.dataset as ds >>> data1 = [{'image': np.zeros(1, np.uint8)}, {'image': np.zeros(2, np.uint8)}] >>> ds1 = ds.PaddedDataset(data1)
-
class
tinyms.data.TextFileDataset(dataset_files, num_samples=None, num_parallel_workers=None, shuffle=<Shuffle.GLOBAL: 'global'>, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset that reads and parses datasets stored on disk in text format. The generated dataset has one column [‘text’].
- Parameters
dataset_files (Union[str, list[str]]) – String or list of files to be read or glob strings to search for a pattern of files. The list will be sorted in a lexicographical order.
num_samples (int, optional) – Number of samples (rows) to read (default=None, reads the full dataset).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (Union[bool, Shuffle level], optional) –
Perform reshuffling of the data every epoch (default=Shuffle.GLOBAL). If shuffle is False, no shuffling will be performed; If shuffle is True, the behavior is the same as setting shuffle to be Shuffle.GLOBAL Otherwise, there are two levels of shuffling:
Shuffle.GLOBAL: Shuffle both the files and samples.
Shuffle.FILES: Shuffle files only.
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None).
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None which means no cache is used).
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_files = ["/path/to/1", "/path/to/2"] # contains 1 or multiple text files >>> dataset = ds.TextFileDataset(dataset_files=dataset_files)
-
class
tinyms.data.TFRecordDataset(dataset_files, schema=None, columns_list=None, num_samples=None, num_parallel_workers=None, shuffle=<Shuffle.GLOBAL: 'global'>, num_shards=None, shard_id=None, shard_equal_rows=False, cache=None)[source]¶ A source dataset that reads and parses datasets stored on disk in TFData format.
- Parameters
dataset_files (Union[str, list[str]]) – String or list of files to be read or glob strings to search for a pattern of files. The list will be sorted in a lexicographical order.
schema (Union[str, Schema], optional) – Path to the JSON schema file or schema object (default=None). If the schema is not provided, the meta data from the TFData file is considered the schema.
columns_list (list[str], optional) – List of columns to be read (default=None, read all columns)
num_samples (int, optional) – Number of samples (rows) to read (default=None). If num_samples is None and numRows(parsed from schema) does not exist, read the full dataset; If num_samples is None and numRows(parsed from schema) is greater than 0, read numRows rows; If both num_samples and numRows(parsed from schema) are greater than 0, read num_samples rows.
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (Union[bool, Shuffle level], optional) –
Perform reshuffling of the data every epoch (default=Shuffle.GLOBAL). If shuffle is False, no shuffling will be performed; If shuffle is True, the behavior is the same as setting shuffle to be Shuffle.GLOBAL Otherwise, there are two levels of shuffling:
Shuffle.GLOBAL: Shuffle both the files and samples.
Shuffle.FILES: Shuffle files only.
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None).
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
shard_equal_rows (bool, optional) – Get equal rows for all shards(default=False). If shard_equal_rows is false, number of rows of each shard may be not equal.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None which means no cache is used).
Examples
>>> import mindspore.dataset as ds >>> import mindspore.common.dtype as mstype >>> >>> dataset_files = ["/path/to/1", "/path/to/2"] # contains 1 or multiple tf data files >>> >>> # 1) Get all rows from dataset_files with no explicit schema >>> # The meta-data in the first row will be used as a schema. >>> tfdataset = ds.TFRecordDataset(dataset_files=dataset_files) >>> >>> # 2) Get all rows from dataset_files with user-defined schema >>> schema = ds.Schema() >>> schema.add_column('col_1d', de_type=mindspore.int64, shape=[2]) >>> tfdataset = ds.TFRecordDataset(dataset_files=dataset_files, schema=schema) >>> >>> # 3) Get all rows from dataset_files with schema file "./schema.json" >>> tfdataset = ds.TFRecordDataset(dataset_files=dataset_files, schema="./schema.json")
-
class
tinyms.data.VOCDataset(dataset_dir, task='Segmentation', usage='train', class_indexing=None, num_samples=None, num_parallel_workers=None, shuffle=None, decode=False, sampler=None, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset for reading and parsing VOC dataset.
The generated dataset has multiple columns :
task=’Detection’, column: [[‘image’, dtype=uint8], [‘bbox’, dtype=float32], [‘label’, dtype=uint32], [‘difficult’, dtype=uint32], [‘truncate’, dtype=uint32]].
task=’Segmentation’, column: [[‘image’, dtype=uint8], [‘target’,dtype=uint8]].
This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
Citation of VOC dataset.
@article{Everingham10, author = {Everingham, M. and Van~Gool, L. and Williams, C. K. I. and Winn, J. and Zisserman, A.}, title = {The Pascal Visual Object Classes (VOC) Challenge}, journal = {International Journal of Computer Vision}, volume = {88}, year = {2010}, number = {2}, month = {jun}, pages = {303--338}, biburl = {http://host.robots.ox.ac.uk/pascal/VOC/pubs/everingham10.html#bibtex}, howpublished = {http://host.robots.ox.ac.uk/pascal/VOC/voc{year}/index.html}, description = {The PASCAL Visual Object Classes (VOC) challenge is a benchmark in visual object category recognition and detection, providing the vision and machine learning communities with a standard dataset of images and annotation, and standard evaluation procedures.} }
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
task (str) – Set the task type of reading voc data, now only support “Segmentation” or “Detection” (default=”Segmentation”).
usage (str) – The type of data list text file to be read (default=”train”).
class_indexing (dict, optional) – A str-to-int mapping from label name to index, only valid in “Detection” task (default=None, the folder names will be sorted alphabetically and each class will be given a unique index starting from 0).
num_samples (int, optional) – The number of images to be included in the dataset (default=None, all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (bool, optional) – Whether to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
decode (bool, optional) – Decode the images after reading (default=False).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None).
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None which means no cache is used).
- Raises
RuntimeError – If xml of Annotations is an invalid format.
RuntimeError – If xml of Annotations loss attribution of “object”.
RuntimeError – If xml of Annotations loss attribution of “bndbox”.
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
ValueError – If task is not equal ‘Segmentation’ or ‘Detection’.
ValueError – If task equal ‘Segmentation’ but class_indexing is not None.
ValueError – If txt related to mode is not exist.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_dir = "/path/to/voc_dataset_directory" >>> >>> # 1) Read VOC data for segmentatation training >>> voc_dataset = ds.VOCDataset(dataset_dir, task="Segmentation", usage="train") >>> >>> # 2) Read VOC data for detection training >>> voc_dataset = ds.VOCDataset(dataset_dir, task="Detection", usage="train") >>> >>> # 3) Read all VOC dataset samples in dataset_dir with 8 threads in random order >>> voc_dataset = ds.VOCDataset(dataset_dir, task="Detection", usage="train", num_parallel_workers=8) >>> >>> # 4) Read then decode all VOC dataset samples in dataset_dir in sequence >>> voc_dataset = ds.VOCDataset(dataset_dir, task="Detection", usage="train", decode=True, shuffle=False) >>> >>> # In VOC dataset, if task='Segmentation', each dictionary has keys "image" and "target" >>> # In VOC dataset, if task='Detection', each dictionary has keys "image" and "annotation"
-
class
tinyms.data.DistributedSampler(dataset_size, num_replicas=None, rank=None, shuffle=True)[source] Distributed sampler.
-
class
tinyms.data.PKSampler(num_val, num_class=None, shuffle=False, class_column='label', num_samples=None)[source]¶ Samples K elements for each P class in the dataset.
- Parameters
num_val (int) – Number of elements to sample for each class.
num_class (int, optional) – Number of classes to sample (default=None, all classes). The parameter does not supported to specify currently.
shuffle (bool, optional) – If True, the class IDs are shuffled (default=False).
class_column (str, optional) – Name of column with class labels for MindDataset (default=’label’).
num_samples (int, optional) – The number of samples to draw (default=None, all elements).
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_dir = "path/to/imagefolder_directory" >>> >>> # creates a PKSampler that will get 3 samples from every class. >>> sampler = ds.PKSampler(3) >>> data = ds.ImageFolderDataset(dataset_dir, num_parallel_workers=8, sampler=sampler)
- Raises
ValueError – If num_val is not positive.
NotImplementedError – If num_class is not None.
ValueError – If shuffle is not boolean.
-
class
tinyms.data.RandomSampler(replacement=False, num_samples=None)[source]¶ Samples the elements randomly.
- Parameters
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_dir = "path/to/imagefolder_directory" >>> >>> # creates a RandomSampler >>> sampler = ds.RandomSampler() >>> data = ds.ImageFolderDataset(dataset_dir, num_parallel_workers=8, sampler=sampler)
- Raises
ValueError – If replacement is not boolean.
ValueError – If num_samples is not positive.
-
class
tinyms.data.SequentialSampler(start_index=None, num_samples=None)[source]¶ Samples the dataset elements sequentially, same as not having a sampler.
- Parameters
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_dir = "path/to/imagefolder_directory" >>> >>> # creates a SequentialSampler >>> sampler = ds.SequentialSampler() >>> data = ds.ImageFolderDataset(dataset_dir, num_parallel_workers=8, sampler=sampler)
-
class
tinyms.data.SubsetRandomSampler(indices, num_samples=None)[source]¶ Samples the elements randomly from a sequence of indices.
- Parameters
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_dir = "path/to/imagefolder_directory" >>> >>> indices = [0, 1, 2, 3, 7, 88, 119] >>> >>> # creates a SubsetRandomSampler, will sample from the provided indices >>> sampler = ds.SubsetRandomSampler(indices) >>> data = ds.ImageFolderDataset(dataset_dir, num_parallel_workers=8, sampler=sampler)
-
class
tinyms.data.WeightedRandomSampler(weights, num_samples=None, replacement=True)[source]¶ Samples the elements from [0, len(weights) - 1] randomly with the given weights (probabilities).
- Parameters
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_dir = "path/to/imagefolder_directory" >>> >>> weights = [0.9, 0.01, 0.4, 0.8, 0.1, 0.1, 0.3] >>> >>> # creates a WeightedRandomSampler that will sample 4 elements without replacement >>> sampler = ds.WeightedRandomSampler(weights, 4) >>> data = ds.ImageFolderDataset(dataset_dir, num_parallel_workers=8, sampler=sampler)
- Raises
ValueError – If num_samples is not positive.
ValueError – If replacement is not boolean.
-
class
tinyms.data.DatasetCache(session_id, size=0, spilling=False, hostname=None, port=None, num_connections=None, prefetch_size=None)[source]¶ A client to interface with tensor caching service.
For details, please check Chinese tutorial, Chinese programming guide.
- Parameters
session_id (int) – A user assigned session id for the current pipeline.
size (int, optional) – Size of the memory set aside for the row caching (default=0 which means unlimited, note that it might bring in the risk of running out of memory on the machine).
spilling (bool, optional) – Whether or not spilling to disk if out of memory (default=False).
hostname (str, optional) – Host name (default=”127.0.0.1”).
port (int, optional) – Port to connect to server (default=50052).
num_connections (int, optional) – Number of tcp/ip connections (default=12).
prefetch_size (int, optional) – Prefetch size (default=20).
-
class
tinyms.data.Schema(schema_file=None)[source]¶ Class to represent a schema of a dataset.
- Parameters
schema_file (str) – Path of schema file (default=None).
- Returns
Schema object, schema info about dataset.
- Raises
RuntimeError – If schema file failed to load.
Example
>>> import mindspore.dataset as ds >>> import mindspore.common.dtype as mstype >>> >>> # Create schema; specify column name, mindspore.dtype and shape of the column >>> schema = ds.Schema() >>> schema.add_column('col1', de_type=mindspore.int64, shape=[2])
-
add_column(name, de_type, shape=None)[source]¶ Add new column to the schema.
- Parameters
- Raises
ValueError – If column type is unknown.
-
from_json(json_obj)[source]¶ Get schema file from JSON object.
- Parameters
json_obj (dictionary) – Object of JSON parsed.
- Raises
RuntimeError – if there is unknown item in the object.
RuntimeError – if dataset type is missing in the object.
RuntimeError – if columns are missing in the object.
-
parse_columns(columns)[source]¶ Parse the columns and add it to self.
- Parameters
columns (Union[dict, list[dict], tuple[dict]]) –
Dataset attribute information, decoded from schema file.
list[dict], ‘name’ and ‘type’ must be in keys, ‘shape’ optional.
dict, columns.keys() as name, columns.values() is dict, and ‘type’ inside, ‘shape’ optional.
- Raises
RuntimeError – If failed to parse columns.
RuntimeError – If column’s name field is missing.
RuntimeError – If column’s type field is missing.
Example
>>> schema = Schema() >>> columns1 = [{'name': 'image', 'type': 'int8', 'shape': [3, 3]}, >>> {'name': 'label', 'type': 'int8', 'shape': [1]}] >>> schema.parse_columns(columns1) >>> columns2 = {'image': {'shape': [3, 3], 'type': 'int8'}, 'label': {'shape': [1], 'type': 'int8'}} >>> schema.parse_columns(columns2)
-
tinyms.data.zip(datasets)[source]¶ Zip the datasets in the input tuple of datasets.
- Parameters
datasets (tuple of class Dataset) – A tuple of datasets to be zipped together. The number of datasets must be more than 1.
- Returns
ZipDataset, dataset zipped.
- Raises
ValueError – If the number of datasets is 1.
TypeError – If datasets is not a tuple.
Examples
>>> import mindspore.dataset as ds >>> >>> dataset_dir1 = "path/to/imagefolder_directory1" >>> dataset_dir2 = "path/to/imagefolder_directory2" >>> ds1 = ds.ImageFolderDataset(dataset_dir1, num_parallel_workers=8) >>> ds2 = ds.ImageFolderDataset(dataset_dir2, num_parallel_workers=8) >>> >>> # Create a dataset which is the combination of ds1 and ds2 >>> data = ds.zip((ds1, ds2))
-
tinyms.data.download_dataset(dataset_name, local_path='.')[source]¶ This function is defined to easily download any public dataset without specifing much details.
- Parameters
- Returns
str, the source location of dataset downloaded.
Examples
>>> from tinyms.data import download_dataset >>> >>> ds_path = download_dataset('mnist')
-
tinyms.data.generate_image_list(dir_path, max_dataset_size=inf)[source]¶ Traverse the directory to generate a list of images path.
tinyms.vision¶
This module is to support vision augmentations. transforms is a high performance image augmentation module which is developed with C++ OpenCV.
-
class
tinyms.vision.ImageViewer(image, label=None)[source]¶ ImageViewer is a class defined for visualizing the input image.
- Parameters
image (Union[PIL.Image, numpy.ndarray]) – image input.
label (str, optional) – specifies the label of this image. Default: None.
- Raises
TypeError – When image input is not numpy.ndarray or PIL.Image.
-
draw(pred_res, labels)[source]¶ Draw the predicting boxes on the picture and show the visualized picture.
- Parameters
Note
This function is only valid when being called in interactive environment, such like Jupyter notebook.
Examples
>>> form PIL import Image >>> >>> img = Image.open('example.jpg') >>> img_viewer = ImageViewer(img) >>> labels = ['1', '2', '3'] >>> img_viewer.draw(pred_res, labels)
-
show()[source]¶ Directly show the visualized picture.
Note
This function is only valid when being called in interactive environment, such like Jupyter notebook.
Examples
>>> form PIL import Image >>> >>> img = Image.open('example.jpg') >>> img_viewer = ImageViewer(img, label='cat') >>> img_viewer.show()
-
tinyms.vision.ssd_bboxes_encode(boxes)[source]¶ Labels anchors with ground truth inputs.
- Parameters
boxes (numpy.ndarray) – Ground truth with shape [N, 5], for each row, it stores [ymin, xmin, ymax, xmax, cls].
- Returns
numpy.ndarray, location ground truth with shape [num_anchors, 4]. numpy.ndarray, class ground truth with shape [num_anchors, 1]. numpy.ndarray, number of positives in an image.
-
tinyms.vision.ssd_bboxes_filter(boxes, box_scores, image_shape)[source]¶ Filter predict boxes with minimum score and nms threshold.
- Parameters
boxes (numpy.ndarray) – Ground truth with shape [N, 4], for each row, it stores
xmin, ymax, xmax] ([ymin,) –
box_scores (numpy.ndarray) – Class scores with shape [N, 21].
image_shape (tuple) – Shape of original image with the format [h, w].
- Returns
list[list[float]], ground truth with shape [N, 4], for each row, it stores [ymin, xmin, ymax, xmax]. list[list[float]], class scores with shape [N, 21]. list[list[int]], class label with shape [N, 21].
-
class
tinyms.vision.MnistTransform[source]¶ Mnist dataset transform class.
- Inputs:
img (Union[numpy.ndarray, PIL.Image]): Image to be transformed in Mnist-style.
- Outputs:
numpy.ndarray, transformed image.
Examples
>>> from PIL import Image >>> from tinyms.vision import MnistTransform >>> >>> mnist_transform = MnistTransform() >>> img = Image.open('example.jpg') >>> img = mnist_transform(img)
-
apply_ds(mnist_ds, repeat_size=1, batch_size=32, num_parallel_workers=None)[source]¶ Apply preprocess operation on MnistDataset instance.
- Parameters
mnist_ds (data.MnistDataset) – MnistDataset instance.
repeat_size (int) – The repeat size of dataset. Default: 1.
batch_size (int) – Batch size. Default: 32.
num_parallel_workers (int) – The number of concurrent workers. Default: None.
- Returns
data.MnistDataset, the preprocessed MnistDataset instance.
Examples
>>> from tinyms.vision import MnistTransform >>> >>> mnist_transform = MnistTransform() >>> mnist_ds = mnist_transform.apply_ds(mnist_ds)
-
postprocess(input, strategy='TOP1_CLASS')¶ Apply postprocess operation for prediction result.
- Parameters
input (numpy.ndarray) – Prediction result.
strategy (str) – Specifies the postprocess strategy. Default: TOP1_CLASS.
- Returns
str, the postprocess result.
-
class
tinyms.vision.Cifar10Transform[source]¶ Cifar10 dataset transform class.
- Inputs:
img (Union[numpy.ndarray, PIL.Image]): Image to be transformed in Cifar10-style.
- Outputs:
numpy.ndarray, Transformed image.
Examples
>>> from PIL import Image >>> from tinyms.vision import Cifar10Transform >>> >>> cifar10_transform = Cifar10Transform() >>> img = Image.open('example.jpg') >>> img = cifar10_transform(img)
“””
-
apply_ds(cifar10_ds, repeat_size=1, batch_size=32, num_parallel_workers=None, is_training=True)[source]¶ Apply preprocess operation on Cifar10Dataset instance.
- Parameters
cifar10_ds (data.Cifar10Dataset) – Cifar10Dataset instance.
repeat_size (int) – The repeat size of dataset. Default: 1.
batch_size (int) – Batch size. Default: 32.
num_parallel_workers (int) – The number of concurrent workers. Default: None.
is_training (bool) – Specifies if is in training step. Default: True.
- Returns
data.Cifar10Dataset, the preprocessed Cifar10Dataset instance.
Examples
>>> from tinyms.vision import Cifar10Transform >>> >>> cifar10_transform = Cifar10Transform() >>> cifar10_ds = cifar10_transform.apply_ds(cifar10_ds)
-
postprocess(input, strategy='TOP1_CLASS')¶ Apply postprocess operation for prediction result.
- Parameters
input (numpy.ndarray) – Prediction result.
strategy (str) – Specifies the postprocess strategy. Default: TOP1_CLASS.
- Returns
str, the postprocess result.
-
class
tinyms.vision.ImageFolderTransform[source]¶ ImageFolder dataset transform class.
- Inputs:
img(Union[numpy.ndarray, PIL.Image]): Image to be transformed in ImageFolder-style.
- Outputs:
numpy.ndarray, transformed image.
Examples
>>> from PIL import Image >>> from tinyms.vision import ImageFolderTransform >>> >>> imagefolder_transform = ImageFolderTransform() >>> img = Image.open('example.jpg') >>> img = imagefolder_transform(img)
-
apply_ds(imagefolder_ds, repeat_size=1, batch_size=32, num_parallel_workers=None, is_training=True)[source]¶ Apply preprocess operation on ImageFolderDataset instance.
- Parameters
cifar10_ds (data.ImageFolderDataset) – ImageFolderDataset instance.
repeat_size (int) – The repeat size of dataset. Default: 1.
batch_size (int) – Batch size. Default: 32.
num_parallel_workers (int) – The number of concurrent workers. Default: None.
is_training (bool) – Specifies if is in training step. Default: True.
- Returns
data.ImageFolderDataset, the preprocessed ImageFolderDataset instance.
Examples
>>> from tinyms.vision import ImageFolderTransform >>> >>> imagefolder_transform = ImageFolderTransform() >>> imagefolder_ds = imagefolder_transform.apply_ds(imagefolder_ds)
-
postprocess(input, strategy='TOP1_CLASS')¶ Apply postprocess operation for prediction result.
- Parameters
input (numpy.ndarray) – Prediction result.
strategy (str) – Specifies the postprocess strategy. Default: TOP1_CLASS.
- Returns
str, the postprocess result.
-
class
tinyms.vision.VOCTransform[source]¶ VOC dataset transform class.
- Inputs:
img(Union[numpy.ndarray, PIL.Image]): Image to be transformed in VOC-style.
- Outputs:
numpy.ndarray, transformed image.
Examples
>>> from PIL import Image >>> from tinyms.vision import VOCTransform >>> >>> voc_transform = VOCTransform() >>> img = Image.open('example.jpg') >>> img = voc_transform(img)
-
apply_ds(voc_ds, repeat_size=1, batch_size=32, num_parallel_workers=None, is_training=True)[source]¶ Apply preprocess operation on VOCDataset instance.
- Parameters
cifar10_ds (data.VOCDataset) – VOCDataset instance.
repeat_size (int) – The repeat size of dataset. Default: 1.
batch_size (int) – Batch size. Default: 32.
num_parallel_workers (int) – The number of concurrent workers. Default: None.
is_training (bool) – Specifies if is in training step. Default: True.
- Returns
data.VOCDataset, the preprocessed VOCDataset instance.
Examples
>>> from tinyms.vision import VOCTransform >>> >>> VOC_transform = VOCTransform() >>> voc_ds = voc_transform.apply_ds(voc_ds)
-
postprocess(input, image_shape, strategy='TOP1_CLASS')[source]¶ Apply postprocess operation for prediction result.
- Parameters
input (numpy.ndarray) – Prediction result.
image_shape (tuple) – Image shape.
strategy (str) – Specifies the postprocess strategy. Default: TOP1_CLASS.
- Returns
dict, the postprocess result.
-
class
tinyms.vision.CycleGanDatasetTransform[source]¶ CycleGan dataset transform class.
- Inputs:
img(Union[numpy.ndarray, PIL.Image]): Image to be transformed in city_scape.
- Outputs:
numpy.ndarray, transformed image.
Examples
>>> from PIL import Image >>> from tinyms.vision import CycleGanDatasetTransform >>> >>> cyclegan_transform = CycleGanDatasetTransform() >>> img = Image.open('example.jpg') >>> img = cyclegan_transform(img)
-
apply_ds(gan_generator_ds, repeat_size=1, batch_size=1, num_parallel_workers=1, shuffle=True, phase='train')[source]¶ Apply preprocess operation on GeneratorDataset instance.
- Parameters
gan_generator_ds (data.GeneratorDataset) – GeneratorDataset instance.
repeat_size (int) – The repeat size of dataset. Default: 1.
batch_size (int) – Batch size. Default: 32.
num_parallel_workers (int) – The number of concurrent workers. Default: 1.
shuffle (bool) – Specifies if applying shuffle operation. Default: True.
phase (str) – Specifies the current phase. Default: train.
- Returns
data.GeneratorDataset, the preprocessed GeneratorDataset instance.
Examples
>>> from tinyms.vision import CycleGanDatasetTransform >>> >>> cyclegan_transform = CycleGanDatasetTransform() >>> gan_generator_ds = cyclegan_transform.apply_ds(gan_generator_ds)
- Raises
TypeError – If gan_generator_ds is not instance of GeneratorDataset.
-
class
tinyms.vision.AutoContrast(cutoff=0.0, ignore=None)[source]¶ Apply automatic contrast on input image.
- Parameters
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.AutoContrast(cutoff=10.0, ignore=[10, 20])] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.BoundingBoxAugment(transform, ratio=0.3)[source]¶ Apply a given image transform on a random selection of bounding box regions of a given image.
- Parameters
transform – C++ transformation function to be applied on random selection of bounding box regions of a given image.
ratio (float, optional) – Ratio of bounding boxes to apply augmentation on. Range: [0, 1] (default=0.3).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> # set bounding box operation with ratio of 1 to apply rotation on all bounding boxes >>> bbox_aug_op = c_vision.BoundingBoxAugment(c_vision.RandomRotation(90), 1) >>> # map to apply ops >>> data3 = data3.map(operations=[bbox_aug_op], >>> input_columns=["image", "bbox"], >>> output_columns=["image", "bbox"], >>> column_order=["image", "bbox"])
-
class
tinyms.vision.CenterCrop(size)[source]¶ Crops the input image at the center to the given size.
- Parameters
size (Union[int, sequence]) – The output size of the cropped image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> # crop image to a square >>> transforms_list1 = [c_vision.Decode(), c_vision.CenterCrop(50)] >>> data1 = data1.map(operations=transforms_list1, input_columns=["image"]) >>> # crop image to portrait style >>> transforms_list2 = [c_vision.Decode(), c_vision.CenterCrop((60, 40))] >>> data2 = data2.map(operations=transforms_list2, input_columns=["image"])
-
class
tinyms.vision.CutMixBatch(image_batch_format, alpha=1.0, prob=1.0)[source]¶ Apply CutMix transformation on input batch of images and labels. Note that you need to make labels into one-hot format and batch before calling this function.
- Parameters
image_batch_format (Image Batch Format) – The method of padding. Can be any of [ImageBatchFormat.NHWC, ImageBatchFormat.NCHW]
alpha (float, optional) – hyperparameter of beta distribution (default = 1.0).
prob (float, optional) – The probability by which CutMix is applied to each image (default = 1.0).
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> import mindspore.dataset.vision.c_transforms as c_vision >>> from mindspore.dataset.transforms.vision import ImageBatchFormat >>> >>> onehot_op = c_transforms.OneHot(num_classes=10) >>> data1 = data1.map(operations=onehot_op, input_columns=["label"]) >>> cutmix_batch_op = c_vision.CutMixBatch(ImageBatchFormat.NHWC, 1.0, 0.5) >>> data1 = data1.batch(5) >>> data1 = data1.map(operations=cutmix_batch_op, input_columns=["image", "label"])
-
class
tinyms.vision.CutOut(length, num_patches=1)[source]¶ Randomly cut (mask) out a given number of square patches from the input NumPy image array.
- Parameters
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.CutOut(80, num_patches=10)] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.Decode(rgb=True)[source]¶ Decode the input image in RGB mode.
- Parameters
rgb (bool, optional) – Mode of decoding input image (default=True). If True means format of decoded image is RGB else BGR(deprecated).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.RandomHorizontalFlip()] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.Equalize[source]¶ Apply histogram equalization on input image.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.Equalize()] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.Grayscale(num_output_channels=1)[source]¶ Convert the input PIL image to grayscale image.
- Parameters
num_output_channels (int) – Number of channels of the output grayscale image (1 or 3). Default is 1. If set to 3, the returned image has 3 identical RGB channels.
Examples
>>> import mindspore.dataset.vision.py_transforms as py_vision >>> from mindspore.dataset.transforms.py_transforms import Compose >>> >>> Compose([py_vision.Decode(), >>> py_vision.Grayscale(3), >>> py_vision.ToTensor()])
-
class
tinyms.vision.HWC2CHW[source]¶ Transpose the input image; shape (H, W, C) to shape (C, H, W).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.RandomHorizontalFlip(0.75), c_vision.RandomCrop(512), >>> c_vision.HWC2CHW()] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.Invert[source]¶ Apply invert on input image in RGB mode.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.Invert()] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.MixUpBatch(alpha=1.0)[source]¶ Apply MixUp transformation on input batch of images and labels. Each image is multiplied by a random weight (lambda) and then added to a randomly selected image from the batch multiplied by (1 - lambda). The same formula is also applied to the one-hot labels. Note that you need to make labels into one-hot format and batch before calling this function.
- Parameters
alpha (float, optional) – Hyperparameter of beta distribution (default = 1.0).
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> onehot_op = c_transforms.OneHot(num_classes=10) >>> data1 = data1.map(operations=onehot_op, input_columns=["label"]) >>> mixup_batch_op = c_vision.MixUpBatch(alpha=0.9) >>> data1 = data1.batch(5) >>> data1 = data1.map(operations=mixup_batch_op, input_columns=["image", "label"])
-
class
tinyms.vision.Normalize(mean, std)[source]¶ Normalize the input image with respect to mean and standard deviation.
- Parameters
mean (sequence) – List or tuple of mean values for each channel, with respect to channel order. The mean values must be in range [0.0, 255.0].
std (sequence) – List or tuple of standard deviations for each channel, with respect to channel order. The standard deviation values must be in range (0.0, 255.0].
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> decode_op = c_vision.Decode() >>> normalize_op = c_vision.Normalize(mean=[121.0, 115.0, 100.0], std=[70.0, 68.0, 71.0]) >>> transforms_list = [decode_op, normalize_op] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.Pad(padding, fill_value=0, padding_mode=<Border.CONSTANT: 'constant'>)[source]¶ Pads the image according to padding parameters.
- Parameters
padding (Union[int, sequence]) – The number of pixels to pad the image. If a single number is provided, it pads all borders with this value. If a tuple or list of 2 values are provided, it pads the (left and top) with the first value and (right and bottom) with the second value. If 4 values are provided as a list or tuple, it pads the left, top, right and bottom respectively.
fill_value (Union[int, tuple], optional) – The pixel intensity of the borders, only valid for padding_mode Border.CONSTANT (default=0). If it is an integer, it is used for all RGB channels. If it is a 3-tuple, it is used to fill R, G, B channels respectively. The fill_value values must be in range [0, 255].
padding_mode (Border mode, optional) –
The method of padding (default=Border.CONSTANT). Can be any of [Border.CONSTANT, Border.EDGE, Border.REFLECT, Border.SYMMETRIC].
Border.CONSTANT, means it fills the border with constant values.
Border.EDGE, means it pads with the last value on the edge.
Border.REFLECT, means it reflects the values on the edge omitting the last value of edge.
Border.SYMMETRIC, means it reflects the values on the edge repeating the last value of edge.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> from mindspore.dataset.transforms.vision import Border >>> >>> transforms_list = [c_vision.Decode(), c_vision.Pad([100, 100, 100, 100])] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
tinyms.vision.PILRandomHorizontalFlip¶ alias of
mindspore.dataset.vision.py_transforms.RandomHorizontalFlip
-
class
tinyms.vision.RandomAffine(degrees, translate=None, scale=None, shear=None, resample=<Inter.NEAREST: 0>, fill_value=0)[source]¶ Apply Random affine transformation to the input image.
- Parameters
degrees (int or float or sequence) – Range of the rotation degrees. If degrees is a number, the range will be (-degrees, degrees). If degrees is a sequence, it should be (min, max).
translate (sequence, optional) – Sequence (tx_min, tx_max, ty_min, ty_max) of minimum/maximum translation in x(horizontal) and y(vertical) directions (default=None). The horizontal and vertical shift is selected randomly from the range: (tx_min*width, tx_max*width) and (ty_min*height, ty_max*height), respectively. If a tuple or list of size 2, then a translate parallel to the X axis in the range of (translate[0], translate[1]) is applied. If a tuple of list of size 4, then a translate parallel to the X axis in the range of (translate[0], translate[1]) and a translate parallel to the Y axis in the range of (translate[2], translate[3]) are applied. If None, no translation is applied.
scale (sequence, optional) – Scaling factor interval (default=None, original scale is used).
shear (int or float or sequence, optional) – Range of shear factor (default=None). If a number, then a shear parallel to the X axis in the range of (-shear, +shear) is applied. If a tuple or list of size 2, then a shear parallel to the X axis in the range of (shear[0], shear[1]) is applied. If a tuple of list of size 4, then a shear parallel to X axis in the range of (shear[0], shear[1]) and a shear parallel to Y axis in the range of (shear[2], shear[3]) is applied. If None, no shear is applied.
resample (Inter mode, optional) –
An optional resampling filter (default=Inter.NEAREST). If omitted, or if the image has mode “1” or “P”, it is set to be Inter.NEAREST. It can be any of [Inter.BILINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.BILINEAR, means resample method is bilinear interpolation.
Inter.NEAREST, means resample method is nearest-neighbor interpolation.
Inter.BICUBIC, means resample method is bicubic interpolation.
fill_value (tuple or int, optional) – Optional fill_value to fill the area outside the transform in the output image. There must be three elements in tuple and the value of single element is [0, 255]. Used only in Pillow versions > 5.0.0 (default=0, filling is performed).
- Raises
ValueError – If degrees is negative.
ValueError – If translation value is not between -1 and 1.
ValueError – If scale is not positive.
ValueError – If shear is a number but is not positive.
TypeError – If degrees is not a number or a list or a tuple. If degrees is a list or tuple, its length is not 2.
TypeError – If translate is specified but is not list or a tuple of length 2 or 4.
TypeError – If scale is not a list or tuple of length 2.’’
TypeError – If shear is not a list or tuple of length 2 or 4.
TypeError – If fill_value is not a single integer or a 3-tuple.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> from mindspore.dataset.transforms.vision import Inter >>> >>> decode_op = c_vision.Decode() >>> random_affine_op = c_vision.RandomAffine(degrees=15, translate=(-0.1, 0.1, 0, 0), scale=(0.9, 1.1), >>> resample=Inter.NEAREST) >>> transforms_list = [decode_op, random_affine_op] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomColor(degrees=(0.1, 1.9))[source]¶ Adjust the color of the input image by a fixed or random degree. This operation works only with 3-channel color images.
- Parameters
degrees (sequence, optional) – Range of random color adjustment degrees. It should be in (min, max) format. If min=max, then it is a single fixed magnitude operation (default=(0.1, 1.9)).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.RandomColor((0.5, 2.0))] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomColorAdjust(brightness=(1, 1), contrast=(1, 1), saturation=(1, 1), hue=(0, 0))[source]¶ Randomly adjust the brightness, contrast, saturation, and hue of the input image.
- Parameters
brightness (Union[float, tuple], optional) – Brightness adjustment factor (default=(1, 1)). Cannot be negative. If it is a float, the factor is uniformly chosen from the range [max(0, 1-brightness), 1+brightness]. If it is a sequence, it should be [min, max] for the range.
contrast (Union[float, tuple], optional) – Contrast adjustment factor (default=(1, 1)). Cannot be negative. If it is a float, the factor is uniformly chosen from the range [max(0, 1-contrast), 1+contrast]. If it is a sequence, it should be [min, max] for the range.
saturation (Union[float, tuple], optional) – Saturation adjustment factor (default=(1, 1)). Cannot be negative. If it is a float, the factor is uniformly chosen from the range [max(0, 1-saturation), 1+saturation]. If it is a sequence, it should be [min, max] for the range.
hue (Union[float, tuple], optional) – Hue adjustment factor (default=(0, 0)). If it is a float, the range will be [-hue, hue]. Value should be 0 <= hue <= 0.5. If it is a sequence, it should be [min, max] where -0.5 <= min <= max <= 0.5.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> decode_op = c_vision.Decode() >>> transform_op = c_vision.RandomColorAdjust(brightness=(0.5, 1), contrast=(0.4, 1), saturation=(0.3, 1)) >>> transforms_list = [decode_op, transform_op] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomCrop(size, padding=None, pad_if_needed=False, fill_value=0, padding_mode=<Border.CONSTANT: 'constant'>)[source]¶ Crop the input image at a random location.
- Parameters
size (Union[int, sequence]) – The output size of the cropped image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
padding (Union[int, sequence], optional) – The number of pixels to pad the image (default=None). If padding is not None, pad image firstly with padding values. If a single number is provided, pad all borders with this value. If a tuple or list of 2 values are provided, pad the (left and top) with the first value and (right and bottom) with the second value. If 4 values are provided as a list or tuple, pad the left, top, right and bottom respectively.
pad_if_needed (bool, optional) – Pad the image if either side is smaller than the given output size (default=False).
fill_value (Union[int, tuple], optional) – The pixel intensity of the borders if the padding_mode is Border.CONSTANT (default=0). If it is a 3-tuple, it is used to fill R, G, B channels respectively.
padding_mode (Border mode, optional) –
The method of padding (default=Border.CONSTANT). It can be any of [Border.CONSTANT, Border.EDGE, Border.REFLECT, Border.SYMMETRIC].
Border.CONSTANT, means it fills the border with constant values.
Border.EDGE, means it pads with the last value on the edge.
Border.REFLECT, means it reflects the values on the edge omitting the last value of edge.
Border.SYMMETRIC, means it reflects the values on the edge repeating the last value of edge.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> decode_op = c_vision.Decode() >>> random_crop_op = c_vision.RandomCrop(512, [200, 200, 200, 200], padding_mode=Border.EDGE) >>> transforms_list = [decode_op, random_crop_op] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomCropDecodeResize(size, scale=(0.08, 1.0), ratio=(0.75, 1.3333333333333333), interpolation=<Inter.BILINEAR: 2>, max_attempts=10)[source]¶ Equivalent to RandomResizedCrop, but crops before decodes.
- Parameters
size (Union[int, sequence]) – The size of the output image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
scale (tuple, optional) – Range [min, max) of respective size of the original size to be cropped (default=(0.08, 1.0)).
ratio (tuple, optional) – Range [min, max) of aspect ratio to be cropped (default=(3. / 4., 4. / 3.)).
interpolation (Inter mode, optional) –
Image interpolation mode (default=Inter.BILINEAR). It can be any of [Inter.BILINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.BILINEAR, means interpolation method is bilinear interpolation.
Inter.NEAREST, means interpolation method is nearest-neighbor interpolation.
Inter.BICUBIC, means interpolation method is bicubic interpolation.
max_attempts (int, optional) – The maximum number of attempts to propose a valid crop_area (default=10). If exceeded, fall back to use center_crop instead.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> from mindspore.dataset.transforms.vision import Inter >>> >>> resize_crop_decode_op = c_vision.RandomCropDecodeResize(size=(50, 75), scale=(0.25, 0.5), >>> interpolation=Inter.NEAREST, max_attempts=5) >>> transforms_list = [resize_crop_decode_op] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomCropWithBBox(size, padding=None, pad_if_needed=False, fill_value=0, padding_mode=<Border.CONSTANT: 'constant'>)[source]¶ Crop the input image at a random location and adjust bounding boxes accordingly.
- Parameters
size (Union[int, sequence]) – The output size of the cropped image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
padding (Union[int, sequence], optional) – The number of pixels to pad the image (default=None). If padding is not None, first pad image with padding values. If a single number is provided, pad all borders with this value. If a tuple or list of 2 values are provided, pad the (left and top) with the first value and (right and bottom) with the second value. If 4 values are provided as a list or tuple, pad the left, top, right and bottom respectively.
pad_if_needed (bool, optional) – Pad the image if either side is smaller than the given output size (default=False).
fill_value (Union[int, tuple], optional) – The pixel intensity of the borders if the padding_mode is Border.CONSTANT (default=0). If it is a 3-tuple, it is used to fill R, G, B channels respectively.
padding_mode (Border mode, optional) –
The method of padding (default=Border.CONSTANT). It can be any of [Border.CONSTANT, Border.EDGE, Border.REFLECT, Border.SYMMETRIC].
Border.CONSTANT, means it fills the border with constant values.
Border.EDGE, means it pads with the last value on the edge.
Border.REFLECT, means it reflects the values on the edge omitting the last value of edge.
Border.SYMMETRIC, means it reflects the values on the edge repeating the last value of edge.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> decode_op = c_vision.Decode() >>> random_crop_with_bbox_op = c_vision.RandomCrop([512, 512], [200, 200, 200, 200]) >>> transforms_list = [decode_op, random_crop_with_bbox_op] >>> data3 = data3.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomHorizontalFlip(prob=0.5)[source]¶ Flip the input image horizontally, randomly with a given probability.
- Parameters
prob (float, optional) – Probability of the image being flipped (default=0.5).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.RandomHorizontalFlip(0.75)] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomHorizontalFlipWithBBox(prob=0.5)[source]¶ Flip the input image horizontally, randomly with a given probability and adjust bounding boxes accordingly.
- Parameters
prob (float, optional) – Probability of the image being flipped (default=0.5).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.RandomHorizontalFlipWithBBox(0.70)] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomPosterize(bits=(8, 8))[source]¶ Reduce the number of bits for each color channel.
- Parameters
bits (sequence or int, optional) – Range of random posterize to compress image. Bits values must be in range of [1,8], and include at least one integer value in the given range. It must be in (min, max) or integer format. If min=max, then it is a single fixed magnitude operation (default=(8, 8)).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.RandomPosterize((6, 8))] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomResize(size)[source]¶ Tensor operation to resize the input image using a randomly selected interpolation mode.
- Parameters
size (Union[int, sequence]) – The output size of the resized image. If size is an integer, smaller edge of the image will be resized to this value with the same image aspect ratio. If size is a sequence of length 2, it should be (height, width).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> # randomly resize image, keeping aspect ratio >>> transforms_list1 = [c_vision.Decode(), c_vision.RandomResize(50)] >>> data1 = data1.map(operations=transforms_list1, input_columns=["image"]) >>> # randomly resize image to landscape style >>> transforms_list2 = [c_vision.Decode(), c_vision.RandomResize((40, 60))] >>> data2 = data2.map(operations=transforms_list2, input_columns=["image"])
-
class
tinyms.vision.RandomResizedCrop(size, scale=(0.08, 1.0), ratio=(0.75, 1.3333333333333333), interpolation=<Inter.BILINEAR: 2>, max_attempts=10)[source]¶ Crop the input image to a random size and aspect ratio.
- Parameters
size (Union[int, sequence]) – The size of the output image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
scale (tuple, optional) – Range [min, max) of respective size of the original size to be cropped (default=(0.08, 1.0)).
ratio (tuple, optional) – Range [min, max) of aspect ratio to be cropped (default=(3. / 4., 4. / 3.)).
interpolation (Inter mode, optional) –
Image interpolation mode (default=Inter.BILINEAR). It can be any of [Inter.BILINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.BILINEAR, means interpolation method is bilinear interpolation.
Inter.NEAREST, means interpolation method is nearest-neighbor interpolation.
Inter.BICUBIC, means interpolation method is bicubic interpolation.
max_attempts (int, optional) – The maximum number of attempts to propose a valid crop_area (default=10). If exceeded, fall back to use center_crop instead.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> from mindspore.dataset.transforms.vision import Inter >>> >>> decode_op = c_vision.Decode() >>> resize_crop_op = c_vision.RandomResizedCrop(size=(50, 75), scale=(0.25, 0.5), >>> interpolation=Inter.BILINEAR) >>> transforms_list = [decode_op, resize_crop_op] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomResizedCropWithBBox(size, scale=(0.08, 1.0), ratio=(0.75, 1.3333333333333333), interpolation=<Inter.BILINEAR: 2>, max_attempts=10)[source]¶ Crop the input image to a random size and aspect ratio and adjust bounding boxes accordingly.
- Parameters
size (Union[int, sequence]) – The size of the output image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
scale (tuple, optional) – Range (min, max) of respective size of the original size to be cropped (default=(0.08, 1.0)).
ratio (tuple, optional) – Range (min, max) of aspect ratio to be cropped (default=(3. / 4., 4. / 3.)).
interpolation (Inter mode, optional) –
Image interpolation mode (default=Inter.BILINEAR). It can be any of [Inter.BILINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.BILINEAR, means interpolation method is bilinear interpolation.
Inter.NEAREST, means interpolation method is nearest-neighbor interpolation.
Inter.BICUBIC, means interpolation method is bicubic interpolation.
max_attempts (int, optional) – The maximum number of attempts to propose a valid crop area (default=10). If exceeded, fall back to use center crop instead.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> from mindspore.dataset.transforms.vision import Inter >>> >>> decode_op = c_vision.Decode() >>> bbox_op = c_vision.RandomResizedCropWithBBox(size=50, interpolation=Inter.NEAREST) >>> transforms_list = [decode_op, bbox_op] >>> data3 = data3.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomResizeWithBBox(size)[source]¶ Tensor operation to resize the input image using a randomly selected interpolation mode and adjust bounding boxes accordingly.
- Parameters
size (Union[int, sequence]) – The output size of the resized image. If size is an integer, smaller edge of the image will be resized to this value with the same image aspect ratio. If size is a sequence of length 2, it should be (height, width).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> # randomly resize image with bounding boxes, keeping aspect ratio >>> transforms_list1 = [c_vision.Decode(), c_vision.RandomResizeWithBBox(60)] >>> data1 = data1.map(operations=transforms_list1, input_columns=["image"]) >>> # randomly resize image with bounding boxes to portrait style >>> transforms_list2 = [c_vision.Decode(), c_vision.RandomResizeWithBBox((80, 60))] >>> data2 = data2.map(operations=transforms_list2, input_columns=["image"])
-
class
tinyms.vision.RandomRotation(degrees, resample=<Inter.NEAREST: 0>, expand=False, center=None, fill_value=0)[source]¶ Rotate the input image by a random angle.
- Parameters
degrees (Union[int, float, sequence) – Range of random rotation degrees. If degrees is a number, the range will be converted to (-degrees, degrees). If degrees is a sequence, it should be (min, max).
resample (Inter mode, optional) –
An optional resampling filter (default=Inter.NEAREST). If omitted, or if the image has mode “1” or “P”, it is set to be Inter.NEAREST. It can be any of [Inter.BILINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.BILINEAR, means resample method is bilinear interpolation.
Inter.NEAREST, means resample method is nearest-neighbor interpolation.
Inter.BICUBIC, means resample method is bicubic interpolation.
expand (bool, optional) – Optional expansion flag (default=False). If set to True, expand the output image to make it large enough to hold the entire rotated image. If set to False or omitted, make the output image the same size as the input. Note that the expand flag assumes rotation around the center and no translation.
center (tuple, optional) – Optional center of rotation (a 2-tuple) (default=None). Origin is the top left corner. None sets to the center of the image.
fill_value (Union[int, tuple], optional) – Optional fill color for the area outside the rotated image (default=0). If it is a 3-tuple, it is used for R, G, B channels respectively. If it is an integer, it is used for all RGB channels.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> from mindspore.dataset.transforms.vision import Inter >>> >>> transforms_list = [c_vision.Decode(), >>> c_vision.RandomRotation(degrees=5.0, resample=Inter.NEAREST, expand=True)] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomSelectSubpolicy(policy)[source]¶ Choose a random sub-policy from a list to be applied on the input image. A sub-policy is a list of tuples (op, prob), where op is a TensorOp operation and prob is the probability that this op will be applied. Once a sub-policy is selected, each op within the subpolicy with be applied in sequence according to its probability.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> policy = [[(c_vision.RandomRotation((45, 45)), 0.5), (c_vision.RandomVerticalFlip(), 1), >>> (c_vision.RandomColorAdjust(), 0.8)], >>> [(c_vision.RandomRotation((90, 90)), 1), (c_vision.RandomColorAdjust(), 0.2)]] >>> data_policy = data1.map(operations=c_vision.RandomSelectSubpolicy(policy), input_columns=["image"])
-
class
tinyms.vision.RandomSharpness(degrees=(0.1, 1.9))[source]¶ Adjust the sharpness of the input image by a fixed or random degree. Degree of 0.0 gives a blurred image, degree of 1.0 gives the original image, and degree of 2.0 gives a sharpened image.
- Parameters
degrees (tuple, optional) – Range of random sharpness adjustment degrees. It should be in (min, max) format. If min=max, then it is a single fixed magnitude operation (default = (0.1, 1.9)).
- Raises
TypeError – If degrees is not a list or tuple.
ValueError – If degrees is negative.
ValueError – If degrees is in (max, min) format instead of (min, max).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.RandomSharpness(degrees=(0.2, 1.9))] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomSolarize(threshold=(0, 255))[source]¶ Invert all pixel values above a threshold.
- Parameters
threshold (tuple, optional) – Range of random solarize threshold. Threshold values should always be in the range (0, 255), include at least one integer value in the given range and be in (min, max) format. If min=max, then it is a single fixed magnitude operation (default=(0, 255)).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.RandomSolarize(threshold=(10,100))] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomVerticalFlip(prob=0.5)[source]¶ Flip the input image vertically, randomly with a given probability.
- Parameters
prob (float, optional) – Probability of the image being flipped (default=0.5).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.RandomVerticalFlip(0.25)] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.RandomVerticalFlipWithBBox(prob=0.5)[source]¶ Flip the input image vertically, randomly with a given probability and adjust bounding boxes accordingly.
- Parameters
prob (float, optional) – Probability of the image being flipped (default=0.5).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.RandomVerticalFlipWithBBox(0.20)] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.Rescale(rescale, shift)[source]¶ Tensor operation to rescale the input image.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> transforms_list = [c_vision.Decode(), c_vision.Rescale(1.0 / 255.0, -1.0)] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.Resize(size, interpolation=<Inter.BILINEAR: 2>)[source]¶ Resize the input image to the given size.
- Parameters
size (Union[int, sequence]) – The output size of the resized image. If size is an integer, the smaller edge of the image will be resized to this value with the same image aspect ratio. If size is a sequence of length 2, it should be (height, width).
interpolation (Inter mode, optional) –
Image interpolation mode (default=Inter.LINEAR). It can be any of [Inter.LINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.LINEAR, means interpolation method is bilinear interpolation.
Inter.NEAREST, means interpolation method is nearest-neighbor interpolation.
Inter.BICUBIC, means interpolation method is bicubic interpolation.
Inter.AREA, means interpolation method is pixel area interpolation.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> from mindspore.dataset.transforms.vision import Inter >>> >>> decode_op = c_vision.Decode() >>> resize_op = c_vision.Resize([100, 75], Inter.BICUBIC) >>> transforms_list = [decode_op, resize_op] >>> data1 = data1.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.ResizeWithBBox(size, interpolation=<Inter.BILINEAR: 2>)[source]¶ Resize the input image to the given size and adjust bounding boxes accordingly.
- Parameters
size (Union[int, sequence]) – The output size of the resized image. If size is an integer, smaller edge of the image will be resized to this value with the same image aspect ratio. If size is a sequence of length 2, it should be (height, width).
interpolation (Inter mode, optional) –
Image interpolation mode (default=Inter.LINEAR). It can be any of [Inter.LINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.LINEAR, means interpolation method is bilinear interpolation.
Inter.NEAREST, means interpolation method is nearest-neighbor interpolation.
Inter.BICUBIC, means interpolation method is bicubic interpolation.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> from mindspore.dataset.transforms.vision import Inter >>> >>> decode_op = c_vision.Decode() >>> bbox_op = c_vision.ResizeWithBBox(50, Inter.NEAREST) >>> transforms_list = [decode_op, bbox_op] >>> data3 = data3.map(operations=transforms_list, input_columns=["image"])
-
class
tinyms.vision.SoftDvppDecodeRandomCropResizeJpeg(size, scale=(0.08, 1.0), ratio=(0.75, 1.3333333333333333), max_attempts=10)[source]¶ Tensor operation to decode, random crop and resize JPEG image using the simulation algorithm of Ascend series chip DVPP module.
The usage scenario is consistent with SoftDvppDecodeResizeJpeg. The input image size should be in range [32*32, 8192*8192]. The zoom-out and zoom-in multiples of the image length and width should in the range [1/32, 16]. Only images with an even resolution can be output. The output of odd resolution is not supported.
- Parameters
size (Union[int, sequence]) – The size of the output image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
scale (tuple, optional) – Range [min, max) of respective size of the original size to be cropped (default=(0.08, 1.0)).
ratio (tuple, optional) – Range [min, max) of aspect ratio to be cropped (default=(3. / 4., 4. / 3.)).
max_attempts (int, optional) – The maximum number of attempts to propose a valid crop_area (default=10). If exceeded, fall back to use center_crop instead.
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> # decode, randomly crop and resize image, keeping aspect ratio >>> transforms_list1 = [c_vision.Decode(), c_vision.SoftDvppDecodeRandomCropResizeJpeg(90)] >>> data1 = data1.map(operations=transforms_list1, input_columns=["image"]) >>> # decode, randomly crop and resize to landscape style >>> transforms_list2 = [c_vision.Decode(), c_vision.SoftDvppDecodeRandomCropResizeJpeg((80, 100))] >>> data2 = data2.map(operations=transforms_list2, input_columns=["image"])
-
class
tinyms.vision.SoftDvppDecodeResizeJpeg(size)[source]¶ Tensor operation to decode and resize JPEG image using the simulation algorithm of Ascend series chip DVPP module.
It is recommended to use this algorithm in the following scenarios: When training, the DVPP of the Ascend chip is not used, and the DVPP of the Ascend chip is used during inference, and the accuracy of inference is lower than the accuracy of training; and the input image size should be in range [32*32, 8192*8192]. The zoom-out and zoom-in multiples of the image length and width should in the range [1/32, 16]. Only images with an even resolution can be output. The output of odd resolution is not supported.
- Parameters
size (Union[int, sequence]) – The output size of the resized image. If size is an integer, smaller edge of the image will be resized to this value with the same image aspect ratio. If size is a sequence of length 2, it should be (height, width).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> # decode and resize image, keeping aspect ratio >>> transforms_list1 = [c_vision.Decode(), c_vision.SoftDvppDecodeResizeJpeg(70)] >>> data1 = data1.map(operations=transforms_list1, input_columns=["image"]) >>> # decode and resize to portrait style >>> transforms_list2 = [c_vision.Decode(), c_vision.SoftDvppDecodeResizeJpeg((80, 60))] >>> data2 = data2.map(operations=transforms_list2, input_columns=["image"])
-
class
tinyms.vision.UniformAugment(transforms, num_ops=2)[source]¶ Tensor operation to perform randomly selected augmentation.
- Parameters
transforms – List of C++ operations (Python operations are not accepted).
num_ops (int, optional) – Number of operations to be selected and applied (default=2).
Examples
>>> import mindspore.dataset.vision.c_transforms as c_vision >>> import mindspore.dataset.vision.py_transforms as py_vision >>> >>> transforms_list = [c_vision.RandomHorizontalFlip(), >>> c_vision.RandomVerticalFlip(), >>> c_vision.RandomColorAdjust(), >>> c_vision.RandomRotation(degrees=45)] >>> uni_aug_op = c_vision.UniformAugment(transforms=transforms_list, num_ops=2) >>> transforms_all = [c_vision.Decode(), c_vision.Resize(size=[224, 224]), >>> uni_aug_op, py_vision.ToTensor()] >>> data_aug = data1.map(operations=transforms_all, input_columns="image", >>> num_parallel_workers=1)
-
class
tinyms.vision.Compose(transforms)[source]¶ Compose a list of transforms into a single transform.
- Parameters
transforms (list) – List of transformations to be applied.
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> compose = c_transforms.Compose([c_vision.Decode(), c_vision.RandomCrop(512)]) >>> data1 = data1.map(operations=compose)
-
class
tinyms.vision.Concatenate(axis=0, prepend=None, append=None)[source]¶ Tensor operation that concatenates all columns into a single tensor.
- Parameters
axis (int, optional) – Concatenate the tensors along given axis (Default=0).
prepend (numpy.array, optional) – NumPy array to be prepended to the already concatenated tensors (Default=None).
append (numpy.array, optional) – NumPy array to be appended to the already concatenated tensors (Default=None).
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> >>> # concatenate string >>> prepend_tensor = np.array(["dw", "df"], dtype='S') >>> append_tensor = np.array(["dwsdf", "df"], dtype='S') >>> concatenate_op = c_transforms.Concatenate(0, prepend_tensor, append_tensor)
-
class
tinyms.vision.Duplicate[source]¶ Duplicate the input tensor to a new output tensor. The input tensor is carried over to the output list.
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> >>> # Data before >>> # | x | >>> # +---------+ >>> # | [1,2,3] | >>> # +---------+ >>> data1 = data1.map(operations=c_transforms.Duplicate(), input_columns=["x"], >>> output_columns=["x", "y"], column_order=["x", "y"]) >>> # Data after >>> # | x | y | >>> # +---------+---------+ >>> # | [1,2,3] | [1,2,3] | >>> # +---------+---------+
-
class
tinyms.vision.Fill(fill_value)[source]¶ Tensor operation to create a tensor filled with input scalar value. The output tensor will have the same shape and type as the input tensor.
- Parameters
fill_value (Union[str, bytes, int, float, bool])) – scalar value to fill created tensor with.
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> >>> fill_op = c_transforms.Fill(3)
-
class
tinyms.vision.Mask(operator, constant, dtype=mindspore.bool)[source]¶ Mask content of the input tensor with the given predicate. Any element of the tensor that matches the predicate will be evaluated to True, otherwise False.
- Parameters
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> >>> # Data before >>> # | col1 | >>> # +---------+ >>> # | [1,2,3] | >>> # +---------+ >>> data1 = data1.map(operations=c_transforms.Mask(Relational.EQ, 2)) >>> # Data after >>> # | col1 | >>> # +--------------------+ >>> # | [False,True,False] | >>> # +--------------------+
-
class
tinyms.vision.OneHot(num_classes)[source]¶ Tensor operation to apply one hot encoding.
- Parameters
num_classes (int) – Number of classes of the label. It should be larger than the largest label number in the dataset.
- Raises
RuntimeError – feature size is bigger than num_classes.
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> onehot_op = c_transforms.OneHot(num_classes=10) >>> data1 = data1.map(operations=onehot_op, input_columns=["label"]) >>> mixup_batch_op = c_vision.MixUpBatch(alpha=0.8) >>> data1 = data1.batch(4) >>> data1 = data1.map(operations=mixup_batch_op, input_columns=["image", "label"])
-
class
tinyms.vision.PadEnd(pad_shape, pad_value=None)[source]¶ Pad input tensor according to pad_shape, need to have same rank.
- Parameters
pad_shape (list(int)) – List of integers representing the shape needed. Dimensions that set to None will not be padded (i.e., original dim will be used). Shorter dimensions will truncate the values.
pad_value (Union[str, bytes, int, float, bool]), optional) – Value used to pad. Default to 0 or empty string in case of tensors of strings.
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> >>> # Data before >>> # | col | >>> # +---------+ >>> # | [1,2,3] | >>> # +---------| >>> data1 = data1.map(operations=c_transforms.PadEnd(pad_shape=[4], pad_value=10)) >>> # Data after >>> # | col | >>> # +------------+ >>> # | [1,2,3,10] | >>> # +------------|
-
class
tinyms.vision.RandomApply(transforms, prob=0.5)[source]¶ Randomly perform a series of transforms with a given probability.
- Parameters
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> rand_apply = c_transforms.RandomApply([c_vision.RandomCrop(512)]) >>> data1 = data1.map(operations=rand_apply)
-
class
tinyms.vision.RandomChoice(transforms)[source]¶ Randomly selects one transform from a list of transforms to perform operation.
- Parameters
transforms (list) – List of transformations to be chosen from to apply.
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> import mindspore.dataset.vision.c_transforms as c_vision >>> >>> rand_choice = c_transforms.RandomChoice([c_vision.CenterCrop(50), c_vision.RandomCrop(512)]) >>> data1 = data1.map(operations=rand_choice)
-
class
tinyms.vision.Slice(*slices)[source]¶ Slice operation to extract a tensor out using the given n slices.
The functionality of Slice is similar to NumPy’s indexing feature. (Currently only rank-1 tensors are supported).
- Parameters
*slices (Union[int, list(int), slice, None, Ellipses]) –
Maximum n number of arguments to slice a tensor of rank n. One object in slices can be one of:
int: Slice this index only along the first dimension. Negative index is supported.list(int): Slice these indices along the first dimension. Negative indices are supported.slice: Slice the generated indices from the slice object along the first dimension. Similar to start:stop:step.None: Slice the whole dimension. Similar to : in Python indexing.Ellipses: Slice the whole dimension. Similar to : in Python indexing.
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> >>> # Data before >>> # | col | >>> # +---------+ >>> # | [1,2,3] | >>> # +---------| >>> data1 = data1.map(operations=c_transforms.Slice(slice(1,3))) # slice indices 1 and 2 only >>> # Data after >>> # | col | >>> # +---------+ >>> # | [2,3] | >>> # +---------|
-
class
tinyms.vision.TypeCast(data_type)[source]¶ Tensor operation to cast to a given MindSpore data type.
- Parameters
data_type (mindspore.dtype) – mindspore.dtype to be cast to.
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> import mindspore.common.dtype as mstype >>> >>> type_cast_op = c_transforms.TypeCast(mstype.int32)
-
class
tinyms.vision.Unique[source]¶ Return an output tensor containing all the unique elements of the input tensor in the same order that they occur in the input tensor.
Also return an index tensor that contains the index of each element of the input tensor in the Unique output tensor.
Finally, return a count tensor that constains the count of each element of the output tensor in the input tensor.
Note
Call batch op before calling this function.
Examples
>>> import mindspore.dataset.transforms.c_transforms as c_transforms >>> >>> # Data before >>> # | x | >>> # +--------------------+ >>> # | [[0,1,2], [1,2,3]] | >>> # +--------------------+ >>> data1 = data1.map(operations=c_transforms.Unique(), input_columns=["x"], >>> output_columns=["x", "y", "z"], column_order=["x", "y", "z"]) >>> # Data after >>> # | x | y |z | >>> # +---------+-----------------+---------+ >>> # | [0,1,2,3] | [0,1,2,1,2,3] | [1,2,2,1] >>> # +---------+-----------------+---------+
tinyms.primitives¶
Primitives module. Operators can be used in the construct function of Layer.
Examples
>>> import tinyms as ts
>>> from tinyms.primitives import tensor_add
>>>
>>> x = ts.ones([2, 3])
>>> y = ts.ones([2, 3])
>>> print(tensor_add(x, y))
[[2. 2. 2.]
[2. 2. 2.]]
-
tinyms.primitives.core(fn=None, **flags)[source]¶ A decorator that adds a flag to the function.
By default, the function is marked as True, enabling to use this decorator to set flag to a graph.
- Parameters
fn (Function) – Function to add flag. Default: None.
flags (dict) – The following flags can be set core, which indicates that this is a core function or other flag. Default: None.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> net = core(net, predit=True) >>> print(hasattr(net, '_mindspore_flags')) True
-
tinyms.primitives.add_flags(fn=None, **flags)[source]¶ A decorator that adds a flag to the function.
Note
Only supports bool value.
- Parameters
fn (Function) – Function or cell to add flag. Default: None.
flags (dict) – Flags use kwargs. Default: None.
- Returns
Function, the function with added flags.
Examples
>>> net = Net(); >>> net = add_flags(net, predit=True) >>> print(hasattr(net, '_mindspore_flags')) True
-
class
tinyms.primitives.MultitypeFuncGraph(name, read_value=False)[source]¶ Generates overloaded functions.
MultitypeFuncGraph is a class used to generate overloaded functions, considering different types as inputs. Initialize an MultitypeFuncGraph object with name, and use register with input types as the decorator for the function to be registed. And the object can be called with different types of inputs, and work with HyperMap and Map.
- Parameters
- Raises
ValueError – If failed to find find a matching function for the given arguments.
Examples
>>> # `add` is a metagraph object which will add two objects according to >>> # input type using ".register" decorator. >>> from mindspore import Tensor >>> from mindspore.ops import Primitive, operations as P >>> from mindspore import dtype as mstype >>> >>> tensor_add = P.Add() >>> add = MultitypeFuncGraph('add') >>> @add.register("Number", "Number") ... def add_scala(x, y): ... return x + y >>> @add.register("Tensor", "Tensor") ... def add_tensor(x, y): ... return tensor_add(x, y) >>> output = add(1, 2) >>> print(output) 3 >>> output = add(Tensor([0.1, 0.6, 1.2], dtype=mstype.float32), Tensor([0.1, 0.6, 1.2], dtype=mstype.float32)) >>> print(output) [0.2 1.2 2.4]
-
class
tinyms.primitives.GradOperation(get_all=False, get_by_list=False, sens_param=False)[source]¶ A higher-order function which is used to generate the gradient function for the input function.
The gradient function generated by GradOperation higher-order function can be customized by construction arguments.
Given an input function net = Net() that takes x and y as inputs, and has a parameter z, see Net in Examples.
To generate a gradient function that returns gradients with respect to the first input (see GradNetWrtX in Examples).
Construct a GradOperation higher-order function with default arguments: grad_op = GradOperation().
Call it with input function as argument to get the gradient function: gradient_function = grad_op(net).
Call the gradient function with input function’s inputs to get the gradients with respect to the first input: grad_op(net)(x, y).
To generate a gradient function that returns gradients with respect to all inputs (see GradNetWrtXY in Examples).
Construct a GradOperation higher-order function with get_all=True which indicates getting gradients with respect to all inputs, they are x and y in example function Net(): grad_op = GradOperation(get_all=True).
Call it with input function as argument to get the gradient function: gradient_function = grad_op(net).
Call the gradient function with input function’s inputs to get the gradients with respect to all inputs: gradient_function(x, y).
To generate a gradient function that returns gradients with respect to given parameters (see GradNetWithWrtParams in Examples).
Construct a GradOperation higher-order function with get_by_list=True: grad_op = GradOperation(get_by_list=True).
Construct a ParameterTuple that will be passed to the input function when constructing GradOperation higher-order function, it will be used as a parameter filter that determine which gradient to return: params = ParameterTuple(net.trainable_params()).
Call it with input function and params as arguments to get the gradient function: gradient_function = grad_op(net, params).
4. Call the gradient function with input function’s inputs to get the gradients with respect to given parameters: gradient_function(x, y).
To generate a gradient function that returns gradients with respect to all inputs and given parameters in the format of ((dx, dy), (dz))(see GradNetWrtInputsAndParams in Examples).
Construct a GradOperation higher-order function with get_all=True and get_by_list=True: grad_op = GradOperation(get_all=True, get_by_list=True).
Construct a ParameterTuple that will be passed along input function when constructing GradOperation higher-order function: params = ParameterTuple(net.trainable_params()).
Call it with input function and params as arguments to get the gradient function: gradient_function = grad_op(net, params).
Call the gradient function with input function’s inputs to get the gradients with respect to all inputs and given parameters: gradient_function(x, y).
We can configure the sensitivity(gradient with respect to output) by setting sens_param as True and passing an extra sensitivity input to the gradient function, the sensitivity input should has the same shape and type with input function’s output(see GradNetWrtXYWithSensParam in Examples).
Construct a GradOperation higher-order function with get_all=True and sens_param=True: grad_op = GradOperation(get_all=True, sens_param=True).
Define grad_wrt_output as sens_param which works as the gradient with respect to output: grad_wrt_output = Tensor(np.ones([2, 2]).astype(np.float32)).
Call it with input function as argument to get the gradient function: gradient_function = grad_op(net).
Call the gradient function with input function’s inputs and sens_param to get the gradients with respect to all inputs: gradient_function(x, y, grad_wrt_output).
- Parameters
get_all (bool) – If True, get all the gradients with respect to inputs. Default: False.
get_by_list (bool) – If True, get all the gradients with respect to Parameter variables. If get_all and get_by_list are both False, get the gradient with respect to first input. If get_all and get_by_list are both True, get the gradients with respect to inputs and Parameter variables at the same time in the form of ((gradients with respect to inputs), (gradients with respect to parameters)). Default: False.
sens_param (bool) – Whether to append sensitivity (gradient with respect to output) as input. If sens_param is False, a ‘ones_like(outputs)’ sensitivity will be attached automatically. Default: False. If the sensor_param is True, a sensitivity (gradient with respect to output) needs to be transferred through the location parameter or key-value pair parameter. If the value is transferred through the key-value pair parameter, the key must be sens.
- Returns
The higher-order function which takes a function as argument and returns gradient function for it.
Examples
>>> from mindspore.common import ParameterTuple >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.matmul = P.MatMul() ... self.z = Parameter(Tensor(np.array([1.0], np.float32)), name='z') ... def construct(self, x, y): ... x = x * self.z ... out = self.matmul(x, y) ... return out ... >>> class GradNetWrtX(nn.Cell): ... def __init__(self, net): ... super(GradNetWrtX, self).__init__() ... self.net = net ... self.grad_op = GradOperation() ... def construct(self, x, y): ... gradient_function = self.grad_op(self.net) ... return gradient_function(x, y) ... >>> x = Tensor([[0.5, 0.6, 0.4], [1.2, 1.3, 1.1]], dtype=mstype.float32) >>> y = Tensor([[0.01, 0.3, 1.1], [0.1, 0.2, 1.3], [2.1, 1.2, 3.3]], dtype=mstype.float32) >>> output = GradNetWrtX(Net())(x, y) >>> print(output) [[1.4100001 1.5999999 6.6 ] [1.4100001 1.5999999 6.6 ]] >>> >>> class GradNetWrtXY(nn.Cell): ... def __init__(self, net): ... super(GradNetWrtXY, self).__init__() ... self.net = net ... self.grad_op = GradOperation(get_all=True) ... def construct(self, x, y): ... gradient_function = self.grad_op(self.net) ... return gradient_function(x, y) >>> >>> x = Tensor([[0.8, 0.6, 0.2], [1.8, 1.3, 1.1]], dtype=mstype.float32) >>> y = Tensor([[0.11, 3.3, 1.1], [1.1, 0.2, 1.4], [1.1, 2.2, 0.3]], dtype=mstype.float32) >>> output = GradNetWrtXY(Net())(x, y) >>> print(output) (Tensor(shape=[2, 3], dtype=Float32, value= [[ 4.50999975e+00, 2.70000005e+00, 3.60000014e+00], [ 4.50999975e+00, 2.70000005e+00, 3.60000014e+00]]), Tensor(shape=[3, 3], dtype=Float32, value= [[ 2.59999990e+00, 2.59999990e+00, 2.59999990e+00], [ 1.89999998e+00, 1.89999998e+00, 1.89999998e+00], [ 1.30000007e+00, 1.30000007e+00, 1.30000007e+00]])) >>> >>> class GradNetWrtXYWithSensParam(nn.Cell): ... def __init__(self, net): ... super(GradNetWrtXYWithSensParam, self).__init__() ... self.net = net ... self.grad_op = GradOperation(get_all=True, sens_param=True) ... self.grad_wrt_output = Tensor([[0.1, 0.6, 0.2], [0.8, 1.3, 1.1]], dtype=mstype.float32) ... def construct(self, x, y): ... gradient_function = self.grad_op(self.net) ... return gradient_function(x, y, self.grad_wrt_output) >>> >>> x = Tensor([[0.8, 0.6, 0.2], [1.8, 1.3, 1.1]], dtype=mstype.float32) >>> y = Tensor([[0.11, 3.3, 1.1], [1.1, 0.2, 1.4], [1.1, 2.2, 0.3]], dtype=mstype.float32) >>> output = GradNetWrtXYWithSensParam(Net())(x, y) >>> print(output) (Tensor(shape=[2, 3], dtype=Float32, value= [[ 2.21099997e+00, 5.09999990e-01, 1.49000001e+00], [ 5.58799982e+00, 2.68000007e+00, 4.07000017e+00]]), Tensor(shape=[3, 3], dtype=Float32, value= [[ 1.51999998e+00, 2.81999993e+00, 2.14000010e+00], [ 1.09999990e+00, 2.04999971e+00, 1.54999995e+00], [ 9.00000036e-01, 1.54999995e+00, 1.25000000e+00]])) >>> >>> class GradNetWithWrtParams(nn.Cell): ... def __init__(self, net): ... super(GradNetWithWrtParams, self).__init__() ... self.net = net ... self.params = ParameterTuple(net.trainable_params()) ... self.grad_op = GradOperation(get_by_list=True) ... def construct(self, x, y): ... gradient_function = self.grad_op(self.net, self.params) ... return gradient_function(x, y) >>> >>> x = Tensor([[0.8, 0.6, 0.2], [1.8, 1.3, 1.1]], dtype=mstype.float32) >>> y = Tensor([[0.11, 3.3, 1.1], [1.1, 0.2, 1.4], [1.1, 2.2, 0.3]], dtype=mstype.float32) >>> output = GradNetWithWrtParams(Net())(x, y) >>> print(output) (Tensor(shape=[1], dtype=Float32, value= [ 2.15359993e+01]),) >>> >>> class GradNetWrtInputsAndParams(nn.Cell): ... def __init__(self, net): ... super(GradNetWrtInputsAndParams, self).__init__() ... self.net = net ... self.params = ParameterTuple(net.trainable_params()) ... self.grad_op = GradOperation(get_all=True, get_by_list=True) ... def construct(self, x, y): ... gradient_function = self.grad_op(self.net, self.params) ... return gradient_function(x, y) >>> >>> x = Tensor([[0.1, 0.6, 1.2], [0.5, 1.3, 0.1]], dtype=mstype.float32) >>> y = Tensor([[0.12, 2.3, 1.1], [1.3, 0.2, 2.4], [0.1, 2.2, 0.3]], dtype=mstype.float32) >>> output = GradNetWrtInputsAndParams(Net())(x, y) >>> print(output) ((Tensor(shape=[2, 3], dtype=Float32, value= [[ 3.51999998e+00, 3.90000010e+00, 2.59999990e+00], [ 3.51999998e+00, 3.90000010e+00, 2.59999990e+00]]), Tensor(shape=[3, 3], dtype=Float32, value= [[ 6.00000024e-01, 6.00000024e-01, 6.00000024e-01], [ 1.89999998e+00, 1.89999998e+00, 1.89999998e+00], [ 1.30000007e+00, 1.30000007e+00, 1.30000007e+00]])), (Tensor(shape=[1], dtype=Float32, value= [ 1.29020004e+01]),))
-
class
tinyms.primitives.HyperMap(ops=None)[source]¶ Hypermap will apply the set operation to input sequences.
Apply the operations to every elements of the sequence or nested sequence. Different from Map, the HyperMap supports to apply on nested structure.
- Parameters
ops (Union[MultitypeFuncGraph, None]) – ops is the operation to apply. If ops is None, the operations should be put in the first input of the instance.
- Inputs:
args (Tuple[sequence]) - If ops is None, all the inputs should be sequences with the same length. And each row of the sequences will be the inputs of the operation.
If ops is not None, the first input is the operation, and the others are inputs.
- Outputs:
Sequence or nested sequence, the sequence of output after applying the function. e.g. operation(args[0][i], args[1][i]).
Examples
>>> from mindspore import dtype as mstype >>> nest_tensor_list = ((Tensor(1, mstype.float32), Tensor(2, mstype.float32)), ... (Tensor(3, mstype.float32), Tensor(4, mstype.float32))) >>> # square all the tensor in the nested list >>> >>> square = MultitypeFuncGraph('square') >>> @square.register("Tensor") ... def square_tensor(x): ... return F.square(x) >>> >>> common_map = HyperMap() >>> output = common_map(square, nest_tensor_list) >>> print(output) ((Tensor(shape=[], dtype=Float32, value= 1), Tensor(shape=[], dtype=Float32, value= 4)), (Tensor(shape=[], dtype=Float32, value= 9), Tensor(shape=[], dtype=Float32, value= 16))) >>> square_map = HyperMap(square) >>> output = square_map(nest_tensor_list) >>> print(output) ((Tensor(shape=[], dtype=Float32, value= 1), Tensor(shape=[], dtype=Float32, value= 4)), (Tensor(shape=[], dtype=Float32, value= 9), Tensor(shape=[], dtype=Float32, value= 16)))
-
tinyms.primitives.normal(shape, mean, stddev, seed=None)[source]¶ Generates random numbers according to the Normal (or Gaussian) random number distribution.
- Parameters
shape (tuple) – The shape of random tensor to be generated.
mean (Tensor) – The mean μ distribution parameter, which specifies the location of the peak, with data type in [int8, int16, int32, int64, float16, float32].
stddev (Tensor) – The deviation σ distribution parameter. It should be greater than 0, with data type in [int8, int16, int32, int64, float16, float32].
seed (int) – Seed is used as entropy source for the Random number engines to generate pseudo-random numbers. must be non-negative. Default: None, which will be treated as 0.
- Returns
Tensor. The shape should be equal to the broadcasted shape between the input shape and shapes of mean and stddev. The dtype is float32.
- Supported Platforms:
AscendGPUCPU
Examples
>>> shape = (3, 1, 2) >>> mean = Tensor(np.array([[3, 4], [5, 6]]), mstype.float32) >>> stddev = Tensor(1.0, mstype.float32) >>> output = C.normal(shape, mean, stddev, seed=5) >>> result = output.shape >>> print(result) (3, 2, 2)
-
tinyms.primitives.laplace(shape, mean, lambda_param, seed=None)[source]¶ Generates random numbers according to the Laplace random number distribution. It is defined as:
\[\text{f}(x;μ,λ) = \frac{1}{2λ}\exp(-\frac{|x-μ|}{λ}),\]- Parameters
shape (tuple) – The shape of random tensor to be generated.
mean (Tensor) – The mean μ distribution parameter, which specifies the location of the peak. With float32 data type.
lambda_param (Tensor) – The parameter used for controlling the variance of this random distribution. The variance of Laplace distribution is equal to twice the square of lambda_param. With float32 data type.
seed (int) – Seed is used as entropy source for Random number engines generating pseudo-random numbers. Default: None, which will be treated as 0.
- Returns
Tensor. The shape should be the broadcasted shape of Input “shape” and shapes of mean and lambda_param. The dtype is float32.
- Supported Platforms:
AscendGPUCPU
Examples
>>> from mindspore import Tensor >>> from mindspore.ops import composite as C >>> import mindspore.common.dtype as mstype >>> shape = (2, 3) >>> mean = Tensor(1.0, mstype.float32) >>> lambda_param = Tensor(1.0, mstype.float32) >>> output = C.laplace(shape, mean, lambda_param, seed=5) >>> print(output.shape) (2, 3)
-
tinyms.primitives.uniform(shape, minval, maxval, seed=None, dtype=mindspore.float32)[source]¶ Generates random numbers according to the Uniform random number distribution.
Note
The number in tensor minval should be strictly less than maxval at any position after broadcasting.
- Parameters
shape (tuple) – The shape of random tensor to be generated.
minval (Tensor) – The distribution parameter a. It defines the minimum possible generated value, with int32 or float32 data type. If dtype is int32, only one number is allowed.
maxval (Tensor) – The distribution parameter b. It defines the maximum possible generated value, with int32 or float32 data type. If dtype is int32, only one number is allowed.
seed (int) – Seed is used as entropy source for the random number engines to generate pseudo-random numbers, must be non-negative. Default: None, which will be treated as 0.
dtype (mindspore.dtype) – type of the Uniform distribution. If it is int32, it generates numbers from discrete uniform distribution; if it is float32, it generates numbers from continuous uniform distribution. It only supports these two data types. Default: mstype.float32.
- Returns
Tensor. The shape should be equal to the broadcasted shape between the input shape and shapes of minval and maxval. The dtype is designated as the input dtype.
- Supported Platforms:
AscendGPU
Examples
>>> # For discrete uniform distribution, only one number is allowed for both minval and maxval: >>> shape = (4, 2) >>> minval = Tensor(1, mstype.int32) >>> maxval = Tensor(2, mstype.int32) >>> output = C.uniform(shape, minval, maxval, seed=5, dtype=mstype.int32) >>> >>> # For continuous uniform distribution, minval and maxval can be multi-dimentional: >>> shape = (3, 1, 2) >>> minval = Tensor(np.array([[3, 4], [5, 6]]), mstype.float32) >>> maxval = Tensor([8.0, 10.0], mstype.float32) >>> output = C.uniform(shape, minval, maxval, seed=5) >>> result = output.shape >>> print(result) (3, 2, 2)
-
tinyms.primitives.gamma(shape, alpha, beta, seed=None)[source]¶ Generates random numbers according to the Gamma random number distribution.
- Parameters
shape (tuple) – The shape of random tensor to be generated.
alpha (Tensor) – The alpha α distribution parameter. It should be greater than 0 with float32 data type.
beta (Tensor) – The beta β distribution parameter. It should be greater than 0 with float32 data type.
seed (int) – Seed is used as entropy source for the random number engines to generate pseudo-random numbers, must be non-negative. Default: None, which will be treated as 0.
- Returns
Tensor. The shape should be equal to the broadcasted shape between the input “shape” and shapes of alpha and beta. The dtype is float32.
- Supported Platforms:
AscendGPUCPU
Examples
>>> shape = (3, 1, 2) >>> alpha = Tensor(np.array([[3, 4], [5, 6]]), mstype.float32) >>> beta = Tensor(np.array([1.0]), mstype.float32) >>> output = C.gamma(shape, alpha, beta, seed=5) >>> result = output.shape >>> print(result) (3, 2, 2)
-
tinyms.primitives.poisson(shape, mean, seed=None)[source]¶ Generates random numbers according to the Poisson random number distribution.
\[\text{P}(i|μ) = \frac{\exp(-μ)μ^{i}}{i!}\]- Parameters
shape (tuple) – The shape of random tensor to be generated.
mean (Tensor) – The mean μ distribution parameter. It should be greater than 0 with float32 data type.
seed (int) – Seed is used as entropy source for the random number engines to generate pseudo-random numbers and must be non-negative. Default: None, which will be treated as 0.
- Returns
Tensor. The shape should be equal to the broadcasted shape between the input “shape” and shapes of mean. The dtype is float32.
- Supported Platforms:
AscendGPUCPU
Examples
>>> shape = (4, 1) >>> mean = Tensor(np.array([5.0, 10.0]), mstype.float32) >>> output = C.poisson(shape, mean, seed=5) >>> result = output.shape >>> print(result) (4, 2)
-
tinyms.primitives.multinomial(inputs, num_sample, replacement=True, seed=None)[source]¶ Returns a tensor sampled from the multinomial probability distribution located in the corresponding row of the input tensor.
Note
The rows of input do not need to sum to one (in which case we use the values as weights), but must be non-negative, finite and have a non-zero sum.
- Parameters
inputs (Tensor) – The input tensor containing probabilities, must be 1 or 2 dimensions, with float32 data type.
num_sample (int) – Number of samples to draw.
replacement (bool, optional) – Whether to draw with replacement or not, default True.
seed (int, optional) – Seed is used as entropy source for the random number engines to generate pseudo-random numbers, must be non-negative. Default: 0.
- Outputs:
Tensor, has the same rows with input. The number of sampled indices of each row is num_samples. The dtype is float32.
- Supported Platforms:
GPU
Examples
>>> input = Tensor([0, 9, 4, 0], mstype.float32) >>> output = C.multinomial(input, 2, True) >>> print(output) [1 1]
-
tinyms.primitives.clip_by_value(x, clip_value_min, clip_value_max)[source]¶ Clips tensor values to a specified min and max.
Limits the value of \(x\) to a range, whose lower limit is ‘clip_value_min’ and upper limit is ‘clip_value_max’.
\[\begin{split}out_i= \left\{ \begin{array}{align} clip\_value_{max} & \text{ if } x_i\ge clip\_value_{max} \\ x_i & \text{ if } clip\_value_{min} \lt x_i \lt clip\_value_{max} \\ clip\_value_{min} & \text{ if } x_i \le clip\_value_{min} \\ \end{array}\right.\end{split}\]Note
‘clip_value_min’ needs to be less than or equal to ‘clip_value_max’.
- Parameters
- Returns
Tensor, a clipped Tensor.
- Supported Platforms:
AscendGPU
Examples
>>> import numpy as np >>> from mindspore import Tensor >>> from mindspore.ops import composite as C >>> import mindspore.common.dtype as mstype >>> min_value = Tensor(5, mstype.float32) >>> max_value = Tensor(20, mstype.float32) >>> x = Tensor(np.array([[1., 25., 5., 7.], [4., 11., 6., 21.]]), mstype.float32) >>> output = C.clip_by_value(x, min_value, max_value) >>> print(output) [[ 5. 20. 5. 7.] [ 5. 11. 6. 20.]]
-
tinyms.primitives.clip_by_global_norm(x, clip_norm=1.0, use_norm=None)[source]¶ Clips tensor values by the ratio of the sum of their norms.
Note
Input ‘x’ should be a tuple or list of tensors. Otherwise, it will raise an error.
- Parameters
- Returns
tuple[Tensor], a clipped Tensor.
- Supported Platforms:
AscendGPU
Examples
>>> x1 = np.array([[2., 3.], [1., 2.]]).astype(np.float32) >>> x2 = np.array([[1., 4.], [3., 1.]]).astype(np.float32) >>> input_x = (Tensor(x1), Tensor(x2)) >>> out = clip_by_global_norm(input_x, 1.0) >>> print(out) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 2.98142403e-01, 4.47213590e-01], [ 1.49071202e-01, 2.98142403e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 1.49071202e-01, 5.96284807e-01], [ 4.47213590e-01, 1.49071202e-01]]))
-
tinyms.primitives.count_nonzero(x, axis=(), keep_dims=False, dtype=mindspore.int32)[source]¶ Count number of nonzero elements across axis of input tensor
- Parameters
x (Tensor) – Input data is used to count non-zero numbers.
axis (Union[int, tuple(int), list(int)]) – The dimensions to reduce. Only constant value is allowed. Default: (), reduce all dimensions.
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default: False.
dtype (Union[Number, mstype.bool_]) – The data type of the output tensor. Only constant value is allowed. Default: mstype.int32
- Returns
Tensor, number of nonzero element. The data type is dtype.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[0, 1, 0], [1, 1, 0]]).astype(np.float32)) >>> nonzero_num = count_nonzero(x=input_x, axis=[0, 1], keep_dims=True, dtype=mstype.int32) >>> print(nonzero_num) [[3]]
-
tinyms.primitives.tensor_dot(x1, x2, axes)[source]¶ Computation of Tensor contraction on arbitrary axes between tensors a and b.
Contraction allows for the summation of products of elements of a and b on specified axes. The same number of axes must be specified for both x1 and x2, and values must be within range of number of dims of both a and b.
Selected dims in both inputs must also match.
axes = 0 leads to outer product axes = 1 leads to normal matrix multiplication when inputs both 2D. axes = 1 is the same as axes = ((1,),(0,) where both a and b are 2D. axes = 2 is the same as axes = ((1,2),(0,1)) where both a and b are 3D.
- Inputs:
x1 (Tensor) - First tensor in tensor_dot with datatype float16 or float32
x2 (Tensor) - Second tensor in tensor_dot with datatype float16 or float32
axes (Union[int, tuple(int), tuple(tuple(int)), list(list(int))]) - Single value or tuple/list of length 2 with dimensions specified for a and b each. If single value N passed, automatically picks up last N dims from a input shape and first N dims from b input shape in order as axes for each respectively.
- Outputs:
Tensor, the shape of the output tensor is \((N + M)\). Where \(N\) and \(M\) are the free axes not contracted in both inputs
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x1 = Tensor(np.ones(shape=[1, 2, 3]), mindspore.float32) >>> input_x2 = Tensor(np.ones(shape=[3, 1, 2]), mindspore.float32) >>> output = C.tensor_dot(input_x1, input_x2, ((0,1),(1,2))) >>> print(output) [[2. 2. 2] [2. 2. 2] [2. 2. 2]]
-
tinyms.primitives.repeat_elements(x, rep, axis=0)[source]¶ Repeat elements of a tensor along an axis, like np.repeat.
- Parameters
- Outputs:
One tensor with values repeated along the specified axis. If x has shape (s1, s2, …, sn) and axis is i, the output will have shape (s1, s2, …, si * rep, …, sn). The output type will be the same as the type of x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> x = Tensor(np.array([[0, 1, 2], [3, 4, 5]]), mindspore.int32) >>> output = C.repeat_elements(x, rep = 2, axis = 0) >>> print(output) [[0 1 2] [0 1 2] [3 4 5] [3 4 5]]
-
tinyms.primitives.sequence_mask(lengths, maxlen)[source]¶ Returns a mask tensor representing the first N positions of each cell.
If lengths has shape [d_1, d_2, …, d_n], then the resulting tensor mask has type dtype and shape [d_1, d_2, …, d_n, maxlen], with mask[i_1, i_2, …, i_n, j] = (j < lengths[i_1, i_2, …, i_n])
- Inputs:
lengths (Tensor) - Tensor to calculate the mask for. All values in this tensor should be less than or equal to maxlen. Values greater than maxlen will be treated as maxlen. Must be type int32 or int64.
maxlen (int) - size of the last dimension of returned tensor. Must be positive and same type as elements in lengths.
- Outputs:
One mask tensor of shape lengths.shape + (maxlen,).
- Supported Platforms:
GPU
Examples
>>> x = Tensor(np.array([[1, 3], [2, 0]])) >>> output = C.sequence_mask(x, 3) >>> print(output) [[[True, False, False], [True, True, True]], [[True, True, False], [False, False, False]]]
-
class
tinyms.primitives.ACos(*args, **kwargs)[source]¶ Computes arccosine of input tensors element-wise.
\[out_i = cos^{-1}(x_i)\]- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Supported Platforms:
AscendGPU
Examples
>>> acos = ops.ACos() >>> input_x = Tensor(np.array([0.74, 0.04, 0.30, 0.56]), mindspore.float32) >>> output = acos(input_x) >>> print(output) [0.7377037 1.5307858 1.2661037 0.97641146]
-
class
tinyms.primitives.Abs(*args, **kwargs)[source]¶ Returns absolute value of a tensor element-wise.
\[out_i = |x_i|\]- Inputs:
input_x (Tensor) - The input tensor. The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as the input_x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([-1.0, 1.0, 0.0]), mindspore.float32) >>> abs = ops.Abs() >>> output = abs(input_x) >>> print(output) [1. 1. 0.]
-
class
tinyms.primitives.AccumulateNV2(*args, **kwargs)[source]¶ Computes accumulation of all input tensors element-wise.
AccumulateNV2 is similar to AddN, but there is a significant difference among them: AccumulateNV2 will not wait for all of its inputs to be ready before summing. That is to say, AccumulateNV2 is able to save memory when inputs are ready at different time since the minimum temporary storage is proportional to the output size rather than the input size.
- Inputs:
input_x (Union(tuple[Tensor], list[Tensor])) - The input tuple or list is made up of multiple tensors whose dtype is number to be added together.
- Outputs:
Tensor, has the same shape and dtype as each entry of the input_x.
- Supported Platforms:
Ascend
Examples
>>> class NetAccumulateNV2(nn.Cell): ... def __init__(self): ... super(NetAccumulateNV2, self).__init__() ... self.accumulateNV2 = ops.AccumulateNV2() ... ... def construct(self, *z): ... return self.accumulateNV2(z) ... >>> net = NetAccumulateNV2() >>> input_x = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> input_y = Tensor(np.array([4, 5, 6]), mindspore.float32) >>> output = net(input_x, input_y, input_x, input_y) >>> print(output) [10. 14. 18.]
-
class
tinyms.primitives.Acosh(*args, **kwargs)[source]¶ Computes inverse hyperbolic cosine of the input element-wise.
\[out_i = cosh^{-1}(input_i)\]- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). The data type should be one of the following types: float16, float32.
- Outputs:
Tensor, has the same shape and type as input_x.
- Supported Platforms:
AscendGPU
Examples
>>> acosh = ops.Acosh() >>> input_x = Tensor(np.array([1.0, 1.5, 3.0, 100.0]), mindspore.float32) >>> output = acosh(input_x) >>> print(output) [0. 0.9624236 1.7627472 5.298292]
-
class
tinyms.primitives.Adam(*args, **kwargs)[source]¶ Updates gradients by the Adaptive Moment Estimation (Adam) algorithm.
The Adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ w = w - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector, \(v\) represents the 2nd moment vector, \(g\) represents gradient, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents var, \(\epsilon\) represents epsilon.
- Parameters
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
- Inputs:
var (Tensor) - Weights to be updated.
m (Tensor) - The 1st moment vector in the updating formula, has the same type as var.
v (Tensor) - the 2nd moment vector in the updating formula. Mean square gradients with the same type as var.
beta1_power (float) - \(beta_1^t\) in the updating formula.
beta2_power (float) - \(beta_2^t\) in the updating formula.
lr (float) - \(l\) in the updating formula.
beta1 (float) - The exponential decay rate for the 1st moment estimations.
beta2 (float) - The exponential decay rate for the 2nd moment estimations.
epsilon (float) - Term added to the denominator to improve numerical stability.
gradient (Tensor) - Gradient, has the same type as var.
- Outputs:
Tuple of 3 Tensor, the updated parameters.
var (Tensor) - The same shape and data type as var.
m (Tensor) - The same shape and data type as m.
v (Tensor) - The same shape and data type as v.
- Supported Platforms:
AscendGPU
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_adam = ops.Adam() ... self.var = Parameter(Tensor(np.ones([2, 2]).astype(np.float32)), name="var") ... self.m = Parameter(Tensor(np.ones([2, 2]).astype(np.float32)), name="m") ... self.v = Parameter(Tensor(np.ones([2, 2]).astype(np.float32)), name="v") ... def construct(self, beta1_power, beta2_power, lr, beta1, beta2, epsilon, grad): ... out = self.apply_adam(self.var, self.m, self.v, beta1_power, beta2_power, lr, beta1, beta2, ... epsilon, grad) ... ... return out >>> np.random.seed(0) >>> net = Net() >>> gradient = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(0.9, 0.999, 0.001, 0.9, 0.999, 1e-8, gradient) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 9.99697924e-01, 9.99692678e-01], [ 9.99696255e-01, 9.99698043e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 9.54881310e-01, 9.71518934e-01], [ 9.60276306e-01, 9.54488277e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 9.99301195e-01, 9.99511480e-01], [ 9.99363303e-01, 9.99296904e-01]]))
-
class
tinyms.primitives.AdamNoUpdateParam(*args, **kwargs)[source]¶ Updates gradients by Adaptive Moment Estimation (Adam) algorithm. This operator do not update the parameter, but calculate the value that should be added to the parameter instead.
The Adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ \Delta{w} = - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector, \(v\) represents the 2nd moment vector, \(g\) represents gradient, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents the parameter to be updated, \(\epsilon\).
- Parameters
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
- Inputs:
m (Tensor) - The 1st moment vector in the updating formula. The data type must be float32.
v (Tensor) - the 2nd moment vector in the updating formula. The shape must be the same as m. The data type must be float32.
beta1_power (Tensor) - \(beta_1^t\) in the updating formula. The data type must be float32.
beta2_power (Tensor) - \(beta_2^t\) in the updating formula. The data type must be float32.
lr (Tensor) - \(l\) in the updating formula. The data type must be float32.
beta1 (Tensor) - The exponential decay rate for the 1st moment estimations. The data type must be float32.
beta2 (Tensor) - The exponential decay rate for the 2nd moment estimations. The data type must be float32.
epsilon (Tensor) - Term added to the denominator to improve numerical stability. The data type must be float32.
gradient (Tensor) - Gradient, the shape must be the same as m, the data type must be float32.
- Outputs:
Tensor, whose shape and data type are the same with gradient, is a value that should be added to the parameter to be updated.
- Supported Platforms:
CPU
Examples
>>> import numpy as np >>> import mindspore as ms >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> >>> class Net(nn.Cell): >>> def __init__(self): >>> super(Net, self).__init__() >>> self.adam = ops.AdamNoUpdateParam() >>> self.m = Parameter(Tensor(np.array([[0.1, 0.1, 0.1], [0.2, 0.2, 0.2]]).astype(np.float32)), >>> name="m") >>> self.v = Parameter(Tensor(np.array([[0.1, 0.1, 0.1], [0.2, 0.2, 0.2]]).astype(np.float32)), >>> name="v") >>> def construct(self, beta1_power, beta2_power, lr, beta1, beta2, epsilon, grad): >>> out = self.adam(self.m, self.v, beta1_power, beta2_power, lr, beta1, beta2, epsilon, grad) >>> return out >>> net = Net() >>> beta1_power = Tensor(0.9, ms.float32) >>> beta2_power = Tensor(0.999, ms.float32) >>> lr = Tensor(0.001, ms.float32) >>> beta1 = Tensor(0.9, ms.float32) >>> beta2 = Tensor(0.999, ms.float32) >>> epsilon = Tensor(1e-8, ms.float32) >>> gradient = Tensor(np.array([[0.1, 0.1, 0.1], [0.1, 0.1, 0.1]]).astype(np.float32)) >>> >>> result = net(beta1_power, beta2_power, lr, beta1, beta2, epsilon, gradient) >>> print(result) [[-0.00010004 -0.00010004 -0.00010004] [-0.00013441 -0.00013441 -0.00013441]]
-
class
tinyms.primitives.Add(*args, **kwargs)[source]¶ Adds two input tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
\[out_{i} = x_{i} + y_{i}\]- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number, or a bool, or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number, or a bool when the first input is a tensor, or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
AscendGPUCPU
Examples
>>> add = ops.Add() >>> input_x = Tensor(np.array([1, 2, 3]).astype(np.float32)) >>> input_y = Tensor(np.array([4, 5, 6]).astype(np.float32)) >>> output = add(input_x, input_y) >>> print(output) [5. 7. 9.]
-
class
tinyms.primitives.AddN(*args, **kwargs)[source]¶ Computes addition of all input tensors element-wise.
All input tensors must have the same shape.
- Inputs:
input_x (Union(tuple[Tensor], list[Tensor])) - The input tuple or list is made up of multiple tensors whose dtype is number or bool to be added together.
- Outputs:
Tensor, has the same shape and dtype as each entry of the input_x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> class NetAddN(nn.Cell): ... def __init__(self): ... super(NetAddN, self).__init__() ... self.addN = ops.AddN() ... ... def construct(self, *z): ... return self.addN(z) ... >>> net = NetAddN() >>> input_x = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> input_y = Tensor(np.array([4, 5, 6]), mindspore.float32) >>> output = net(input_x, input_y, input_x, input_y) >>> print(output) [10. 14. 18.]
-
class
tinyms.primitives.AllGather(*args, **kwargs)[source]¶ Gathers tensors from the specified communication group.
Note
The tensors must have the same shape and format in all processes of the collection.
- Parameters
group (str) – The communication group to work on. Default: “hccl_world_group”.
- Raises
TypeError – If group is not a string.
ValueError – If the local rank id of the calling process in the group is larger than the group’s rank size.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor. If the number of devices in the group is N, then the shape of output is \((N, x_1, x_2, ..., x_R)\).
- Supported Platforms:
AscendGPU
Examples
>>> # This example should be run with two devices. Refer to the tutorial > Distributed Training on mindspore.cn. >>> import numpy as np >>> import mindspore.ops.operations as ops >>> import mindspore.nn as nn >>> from mindspore.communication import init >>> from mindspore import Tensor, context >>> >>> context.set_context(mode=context.GRAPH_MODE) >>> init() ... class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.allgather = ops.AllGather() ... ... def construct(self, x): ... return self.allgather(x) ... >>> input_ = Tensor(np.ones([2, 8]).astype(np.float32)) >>> net = Net() >>> output = net(input_) >>> print(output) [[1. 1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1. 1.]]
-
class
tinyms.primitives.AllReduce(*args, **kwargs)[source]¶ Reduces the tensor data across all devices in such a way that all devices will get the same final result.
Note
The operation of AllReduce does not support “prod” currently. The tensors must have the same shape and format in all processes of the collection.
- Parameters
- Raises
TypeError – If any of operation and group is not a string, or fusion is not an integer, or the input’s dtype is bool.
ValueError – If the operation is “prod”.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape of the input, i.e., \((x_1, x_2, ..., x_R)\). The contents depend on the specified operation.
- Supported Platforms:
AscendGPU
Examples
>>> from mindspore.communication import init >>> from mindspore import Tensor >>> from mindspore.ops.operations.comm_ops import ReduceOp >>> import mindspore.nn as nn >>> import mindspore.ops.operations as ops >>> >>> init() >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.allreduce_sum = ops.AllReduce(ReduceOp.SUM, group="nccl_world_group") ... ... def construct(self, x): ... return self.allreduce_sum(x) ... >>> input_ = Tensor(np.ones([2, 8]).astype(np.float32)) >>> net = Net() >>> output = net(input_) >>> print(output) [[4. 5. 6. 0. 0. 0. 0. 0.] [0. 0. 0. 0. 0. 0. 0. 0.]]
-
class
tinyms.primitives.AllSwap(*args, **kwargs)[source]¶ AllSwap is a collective operation.
AllSwap sends data from the all processes to the all processes in the specified group. It has two phases:
The scatter phase: On each process, the operand is split into the send size of blocks along the 0-th axis, and the blocks are scattered to all processes, e.g., the ith block is send to the ith process.
The gather phase: Each process concatenates the received blocks along the 0-th axis.
Note
The tensors must have the same format in all processes of the collection.
- Parameters
group (str) – The communication group name.
- Inputs:
tensor_in (tensor): A 2-D tensor. On each process, divide blocks into number of the send size. send_size (tensor): A 1-D int64 tensor. The element is the send data size for each process. recv_size (tensor): A 1-D int64 tensor. The element is the receive data size for each process.
- Returns
The result tensor.
- Return type
tensor_out (tensor)
- Raises
TypeError – If group is not a string.
-
class
tinyms.primitives.ApplyAdaMax(*args, **kwargs)[source]¶ Updates relevant entries according to the adamax scheme.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m_{t} = \beta_1 * m_{t-1} + (1 - \beta_1) * g \\ v_{t} = \max(\beta_2 * v_{t-1}, \left| g \right|) \\ var = var - \frac{l}{1 - \beta_1^t} * \frac{m_{t}}{v_{t} + \epsilon} \end{array}\end{split}\]\(t\) represents updating step while \(m\) represents the 1st moment vector, \(m_{t-1}\) is the last momentent of \(m_{t}\), \(v\) represents the 2nd moment vector, \(v_{t-1}\) is the last momentent of \(v_{t}\), \(l\) represents scaling factor lr, \(g\) represents grad, \(\beta_1, \beta_2\) represent beta1 and beta2, \(beta_1^t\) represents beta1_power, \(var\) represents the variable to be updated, \(\epsilon\) represents epsilon.
Inputs of var, m, v and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
var (Parameter) - Variable to be updated. With float32 or float16 data type.
m (Parameter) - The 1st moment vector in the updating formula, has the same shape and type as var. With float32 or float16 data type.
v (Parameter) - The 2nd moment vector in the updating formula. Mean square gradients with the same shape and type as var. With float32 or float16 data type.
beta1_power (Union[Number, Tensor]) - \(beta_1^t\) in the updating formula, must be scalar. With float32 or float16 data type.
lr (Union[Number, Tensor]) - Learning rate, \(l\) in the updating formula, must be scalar. With float32 or float16 data type.
beta1 (Union[Number, Tensor]) - The exponential decay rate for the 1st moment estimations, must be scalar. With float32 or float16 data type.
beta2 (Union[Number, Tensor]) - The exponential decay rate for the 2nd moment estimations, must be scalar. With float32 or float16 data type.
epsilon (Union[Number, Tensor]) - A small value added for numerical stability, must be scalar. With float32 or float16 data type.
grad (Tensor) - A tensor for gradient, has the same shape and type as var. With float32 or float16 data type.
- Outputs:
Tuple of 3 Tensor, the updated parameters.
var (Tensor) - The same shape and data type as var.
m (Tensor) - The same shape and data type as m.
v (Tensor) - The same shape and data type as v.
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_ada_max = ops.ApplyAdaMax() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.m = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="m") ... self.v = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="v") ... def construct(self, beta1_power, lr, beta1, beta2, epsilon, grad): ... out = self.apply_ada_max(self.var, self.m, self.v, beta1_power, lr, beta1, beta2, epsilon, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> beta1_power =Tensor(0.9, mstype.float32) >>> lr = Tensor(0.001, mstype.float32) >>> beta1 = Tensor(0.9, mstype.float32) >>> beta2 = Tensor(0.99, mstype.float32) >>> epsilon = Tensor(1e-10, mstype.float32) >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(beta1_power, lr, beta1, beta2, epsilon, grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.44221461e-01, 7.07908988e-01], [ 5.97648144e-01, 5.29388547e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 4.38093781e-01, 6.73864365e-01], [ 4.00932074e-01, 8.11308622e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 9.54026103e-01, 9.25596654e-01], [ 7.83807814e-01, 5.23605943e-01]]))
-
class
tinyms.primitives.ApplyAdadelta(*args, **kwargs)[source]¶ Updates relevant entries according to the adadelta scheme.
\[accum = \rho * accum + (1 - \rho) * grad^2\]\[\text{update} = \sqrt{\text{accum_update} + \epsilon} * \frac{grad}{\sqrt{accum + \epsilon}}\]\[\text{accum_update} = \rho * \text{accum_update} + (1 - \rho) * update^2\]\[var -= lr * update\]Inputs of var, accum, accum_update and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
var (Parameter) - Weights to be updated. With float32 or float16 data type.
accum (Parameter) - Accumulation to be updated, has the same shape and type as var. With float32 or float16 data type.
accum_update (Parameter) - Accum_update to be updated, has the same shape and type as var. With float32 or float16 data type.
lr (Union[Number, Tensor]) - Learning rate, must be scalar. With float32 or float16 data type.
rho (Union[Number, Tensor]) - Decay rate, must be scalar. With float32 or float16 data type.
epsilon (Union[Number, Tensor]) - A small value added for numerical stability, must be scalar. With float32 or float16 data type.
grad (Tensor) - Gradients, has the same shape and type as var. With float32 or float16 data type.
- Outputs:
Tuple of 3 Tensor, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as accum.
accum_update (Tensor) - The same shape and data type as accum_update.
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_adadelta = ops.ApplyAdadelta() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="accum") ... self.accum_update = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="accum_update") ... def construct(self, lr, rho, epsilon, grad): ... out = self.apply_adadelta(self.var, self.accum, self.accum_update, lr, rho, epsilon, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> lr = Tensor(0.001, mstype.float32) >>> rho = Tensor(0.0, mstype.float32) >>> epsilon = Tensor(1e-6, mstype.float32) >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(lr, rho, epsilon, grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.47831833e-01, 7.14570105e-01], [ 6.01873636e-01, 5.44156015e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 3.22674602e-01, 8.56729150e-01], [ 5.04612131e-03, 7.59151531e-03]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 9.63660717e-01, 3.83442074e-01], [ 7.91569054e-01, 5.28826237e-01]]))
-
class
tinyms.primitives.ApplyAdagrad(*args, **kwargs)[source]¶ Updates relevant entries according to the adagrad scheme.
\[accum += grad * grad\]\[var -= lr * grad * \frac{1}{\sqrt{accum}}\]Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent.. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
update_slots (bool) – If True, accum will be updated. Default: True.
- Inputs:
var (Parameter) - Variable to be updated. With float32 or float16 data type.
accum (Parameter) - Accumulation to be updated. The shape and dtype must be the same as var. With float32 or float16 data type.
lr (Union[Number, Tensor]) - The learning rate value, must be scalar. With float32 or float16 data type.
grad (Tensor) - A tensor for gradient. The shape and dtype must be the same as var. With float32 or float16 data type.
- Outputs:
Tuple of 2 Tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as accum.
- Supported Platforms:
AscendCPUGPU
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_adagrad = ops.ApplyAdagrad() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="accum") ... def construct(self, lr, grad): ... out = self.apply_adagrad(self.var, self.accum, lr, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> lr = Tensor(0.001, mstype.float32) >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(lr, grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.47984838e-01, 7.14758754e-01], [ 6.01995945e-01, 5.44394553e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 1.35230064e+00, 7.92921484e-01], [ 1.06441569e+00, 1.17150283e+00]]))
-
class
tinyms.primitives.ApplyAdagradV2(*args, **kwargs)[source]¶ Updates relevant entries according to the adagradv2 scheme.
\[accum += grad * grad\]\[var -= lr * grad * \frac{1}{\sqrt{accum} + \epsilon}\]Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
- Inputs:
var (Parameter) - Variable to be updated. With float16 or float32 data type.
accum (Parameter) - Accumulation to be updated. The shape and dtype must be the same as var. With float16 or float32 data type.
lr (Union[Number, Tensor]) - The learning rate value, must be a float number or a scalar tensor with float16 or float32 data type.
grad (Tensor) - A tensor for gradient. The shape and dtype must be the same as var. With float16 or float32 data type.
- Outputs:
Tuple of 2 Tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as m.
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_adagrad_v2 = ops.ApplyAdagradV2(epsilon=1e-6) ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="accum") ... def construct(self, lr, grad): ... out = self.apply_adagrad_v2(self.var, self.accum, lr, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> lr = Tensor(0.001, mstype.float32) >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(lr, grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.47984838e-01, 7.14758754e-01], [ 6.01995945e-01, 5.44394553e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 1.35230064e+00, 7.92921484e-01], [ 1.06441569e+00, 1.17150283e+00]]))
-
class
tinyms.primitives.ApplyAddSign(*args, **kwargs)[source]¶ Updates relevant entries according to the AddSign algorithm.
\[\begin{split}\begin{array}{ll} \\ m_{t} = \beta * m_{t-1} + (1 - \beta) * g \\ \text{update} = (\alpha + \text{sign_decay} * sign(g) * sign(m)) * g \\ var = var - lr_{t} * \text{update} \end{array}\end{split}\]\(t\) represents updating step while \(m\) represents the 1st moment vector, \(m_{t-1}\) is the last momentent of \(m_{t}\), \(lr\) represents scaling factor lr, \(g\) represents grad.
Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
var (Parameter) - Variable tensor to be updated. With float32 or float16 data type.
m (Parameter) - Variable tensor to be updated, has the same dtype as var.
lr (Union[Number, Tensor]) - The learning rate value, must be a scalar. With float32 or float16 data type.
alpha (Union[Number, Tensor]) - Must be a scalar. With float32 or float16 data type.
sign_decay (Union[Number, Tensor]) - Must be a scalar. With float32 or float16 data type.
beta (Union[Number, Tensor]) - The exponential decay rate, must be a scalar. With float32 or float16 data type.
grad (Tensor) - A tensor of the same type as var, for the gradient.
- Outputs:
Tuple of 2 Tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
m (Tensor) - The same shape and data type as m.
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_add_sign = ops.ApplyAddSign() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.m = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="m") ... self.lr = 0.001 ... self.alpha = 1.0 ... self.sign_decay = 0.99 ... self.beta = 0.9 ... def construct(self, grad): ... out = self.apply_add_sign(self.var, self.m, self.lr, self.alpha, self.sign_decay, self.beta, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.46895862e-01, 7.14426279e-01], [ 6.01187825e-01, 5.43830693e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 4.77655590e-01, 6.19648814e-01], [ 4.73001003e-01, 8.55485201e-01]]))
-
class
tinyms.primitives.ApplyCenteredRMSProp(*args, **kwargs)[source]¶ Optimizer that implements the centered RMSProp algorithm. Please refer to the usage in source code of nn.RMSProp.
Note
Update var according to the centered RMSProp algorithm.
\[g_{t} = \rho g_{t-1} + (1 - \rho)\nabla Q_{i}(w)\]\[s_{t} = \rho s_{t-1} + (1 - \rho)(\nabla Q_{i}(w))^2\]\[m_{t} = \beta m_{t-1} + \frac{\eta} {\sqrt{s_{t} - g_{t}^2 + \epsilon}} \nabla Q_{i}(w)\]\[w = w - m_{t}\]where \(w\) represents var, which will be updated. \(g_{t}\) represents mean_gradient, \(g_{t-1}\) is the last momentent of \(g_{t}\). \(s_{t}\) represents mean_square, \(s_{t-1}\) is the last momentent of \(s_{t}\), \(m_{t}\) represents moment, \(m_{t-1}\) is the last momentent of \(m_{t}\). \(\rho\) represents decay. \(\beta\) is the momentum term, represents momentum. \(\epsilon\) is a smoothing term to avoid division by zero, represents epsilon. \(\eta\) represents learning_rate. \(\nabla Q_{i}(w)\) represents grad.
- Parameters
use_locking (bool) – Whether to enable a lock to protect the variable and accumlation tensors from being updated. Default: False.
- Inputs:
var (Tensor) - Weights to be update.
mean_gradient (Tensor) - Mean gradients, must have the same type as var.
mean_square (Tensor) - Mean square gradients, must have the same type as var.
moment (Tensor) - Delta of var, must have the same type as var.
grad (Tensor) - Gradient, must have the same type as var.
learning_rate (Union[Number, Tensor]) - Learning rate. Must be a float number or a scalar tensor with float16 or float32 data type.
decay (float) - Decay rate.
momentum (float) - Momentum.
epsilon (float) - Ridge term.
- Outputs:
Tensor, parameters to be update.
- Supported Platforms:
AscendGPU
Examples
>>> centered_rms_prop = ops.ApplyCenteredRMSProp() >>> input_x = Tensor(np.arange(-2, 2).astype(np.float32).reshape(2, 2), mindspore.float32) >>> mean_grad = Tensor(np.arange(4).astype(np.float32).reshape(2, 2), mindspore.float32) >>> mean_square = Tensor(np.arange(-3, 1).astype(np.float32).reshape(2, 2), mindspore.float32) >>> moment = Tensor(np.arange(4).astype(np.float32).reshape(2, 2), mindspore.float32) >>> grad = Tensor(np.arange(4).astype(np.float32).reshape(2, 2), mindspore.float32) >>> learning_rate = Tensor(0.9, mindspore.float32) >>> decay = 0.0 >>> momentum = 1e-10 >>> epsilon = 0.05 >>> output = centered_rms_prop(input_x, mean_grad, mean_square, moment, grad, ... learning_rate, decay, momentum, epsilon) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[-2.00000000e+00, -5.02492237e+00], [-8.04984474e+00, -1.10747662e+01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 0.00000000e+00, 1.00000000e+00], [ 2.00000000e+00, 3.00000000e+00]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 0.00000000e+00, 1.00000000e+00], [ 4.00000000e+00, 9.00000000e+00]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 0.00000000e+00, 4.02492237e+00], [ 8.04984474e+00, 1.20747662e+01]]))
-
class
tinyms.primitives.ApplyFtrl(*args, **kwargs)[source]¶ Updates relevant entries according to the FTRL scheme.
- Parameters
use_locking (bool) – Use locks for updating operation if true . Default: False.
- Inputs:
var (Parameter) - The variable to be updated. The data type must be float16 or float32.
accum (Parameter) - The accumulation to be updated, must be same type and shape as var.
linear (Parameter) - the linear coefficient to be updated, must be same type and shape as var.
grad (Tensor) - Gradient. The data type must be float16 or float32.
lr (Union[Number, Tensor]) - The learning rate value, must be positive. Default: 0.001. It must be a float number or a scalar tensor with float16 or float32 data type.
l1 (Union[Number, Tensor]) - l1 regularization strength, must be greater than or equal to zero. Default: 0.0. It must be a float number or a scalar tensor with float16 or float32 data type.
l2 (Union[Number, Tensor]) - l2 regularization strength, must be greater than or equal to zero. Default: 0.0. It must be a float number or a scalar tensor with float16 or float32 data type.
lr_power (Union[Number, Tensor]) - Learning rate power controls how the learning rate decreases during training, must be less than or equal to zero. Use fixed learning rate if lr_power is zero. Default: -0.5. It must be a float number or a scalar tensor with float16 or float32 data type.
- Outputs:
There are three outputs for Ascend environment.
var (Tensor) - represents the updated var.
accum (Tensor) - represents the updated accum.
linear (Tensor) - represents the updated linear.
There is only one output for GPU environment.
var (Tensor) - This value is always zero and the input parameters has been updated in-place.
- Supported Platforms:
AscendGPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Parameter, Tensor >>> import mindspore.context as context >>> from mindspore.ops import operations as ops >>> class ApplyFtrlNet(nn.Cell): ... def __init__(self): ... super(ApplyFtrlNet, self).__init__() ... self.apply_ftrl = ops.ApplyFtrl() ... self.lr = 0.001 ... self.l1 = 0.0 ... self.l2 = 0.0 ... self.lr_power = -0.5 ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="accum") ... self.linear = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="linear") ... ... def construct(self, grad): ... out = self.apply_ftrl(self.var, self.accum, self.linear, grad, self.lr, self.l1, self.l2, ... self.lr_power) ... return out ... >>> np.random.seed(0) >>> net = ApplyFtrlNet() >>> input_x = Tensor(np.random.randint(-4, 4, (2, 2)), mindspore.float32) >>> output = net(input_x) >>> is_tbe = context.get_context("device_target") == "Ascend" >>> if is_tbe: ... print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 4.61418092e-01, 5.30964255e-01], [ 2.68715084e-01, 3.82065028e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 1.64236546e+01, 9.64589405e+00], [ 1.43758726e+00, 9.89177322e+00]]), Tensor(shape=[2, 2], dtype=Float32, value= [[-1.86994812e+03, -1.64906018e+03], [-3.22187836e+02, -1.20163989e+03]])) ... else: ... print(net.var.asnumpy()) [[0.4614181 0.5309642 ] [0.2687151 0.38206503]] ... print(net.accum.asnumpy()) [[16.423655 9.645894 ] [ 1.4375873 9.891773 ]] ... print(net.linear.asnumpy()) [[-1869.9479 -1649.0599] [ -322.1879 -1201.6399]]
-
class
tinyms.primitives.ApplyGradientDescent(*args, **kwargs)[source]¶ Updates relevant entries according to the following.
\[var = var - \alpha * \delta\]Inputs of var and delta comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
var (Parameter) - Variable tensor to be updated. With float32 or float16 data type.
alpha (Union[Number, Tensor]) - Scaling factor, must be a scalar. With float32 or float16 data type.
delta (Tensor) - A tensor for the change, has the same type as var.
- Outputs:
Tensor, represents the updated var.
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_gradient_descent = ops.ApplyGradientDescent() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.alpha = 0.001 ... def construct(self, delta): ... out = self.apply_gradient_descent(self.var, self.alpha, delta) ... return out ... >>> net = Net() >>> delta = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(delta) >>> print(output.shape) (2, 2)
-
class
tinyms.primitives.ApplyMomentum(*args, **kwargs)[source]¶ Optimizer that implements the Momentum algorithm.
Refer to the paper On the importance of initialization and momentum in deep learning for more details.
Refer to
mindspore.nn.Momentumfor more details about the formula and usage.Inputs of variable, accumulation and gradient comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. Data type conversion of Parameter is not supported. RuntimeError exception will be thrown.
- Parameters
- Inputs:
variable (Parameter) - Weights to be updated. data type must be float.
accumulation (Parameter) - Accumulated gradient value by moment weight. Has the same data type with variable.
learning_rate (Union[Number, Tensor]) - The learning rate value, must be a float number or a scalar tensor with float data type.
gradient (Tensor) - Gradient, has the same data type as variable.
momentum (Union[Number, Tensor]) - Momentum, must be a float number or a scalar tensor with float data type.
- Outputs:
Tensor, parameters to be updated.
- Raises
TypeError – If the use_locking or use_nesterov is not a bool or gradient_scale is not a float.
- Supported Platforms:
AscendGPUCPU
Examples
Please refer to the usage in
mindspore.nn.Momentum.
-
class
tinyms.primitives.ApplyPowerSign(*args, **kwargs)[source]¶ Updates relevant entries according to the AddSign algorithm.
\[\begin{split}\begin{array}{ll} \\ m_{t} = \beta * m_{t-1} + (1 - \beta) * g \\ \text{update} = \exp(\text{logbase} * \text{sign_decay} * sign(g) * sign(m)) * g \\ var = var - lr_{t} * \text{update} \end{array}\end{split}\]\(t\) represents updating step while \(m\) represents the 1st moment vector, \(m_{t-1}\) is the last momentent of \(m_{t}\), \(lr\) represents scaling factor lr, \(g\) represents grad.
All of inputs comply with the implicit type conversion rules to make the data types consistent. If lr, logbase, sign_decay or beta is a number, the number is automatically converted to Tensor, and the data type is consistent with the Tensor data type involved in the operation. If inputs are tensors and have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
var (Parameter) - Variable tensor to be updated. With float32 or float16 data type. If data type of var is float16, all inputs must have the same data type as var.
m (Parameter) - Variable tensor to be updated, has the same dtype as var.
lr (Union[Number, Tensor]) - The learning rate value, must be a scalar. With float32 or float16 data type.
logbase (Union[Number, Tensor]) - Must be a scalar. With float32 or float16 data type.
sign_decay (Union[Number, Tensor]) - Must be a scalar. With float32 or float16 data type.
beta (Union[Number, Tensor]) - The exponential decay rate, must be a scalar. With float32 or float16 data type.
grad (Tensor) - A tensor of the same type as var, for the gradient.
- Outputs:
Tuple of 2 Tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
m (Tensor) - The same shape and data type as m.
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_power_sign = ops.ApplyPowerSign() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.m = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="m") ... self.lr = 0.001 ... self.logbase = np.e ... self.sign_decay = 0.99 ... self.beta = 0.9 ... def construct(self, grad): ... out = self.apply_power_sign(self.var, self.m, self.lr, self.logbase, ... self.sign_decay, self.beta, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.34601569e-01, 7.09534407e-01], [ 5.91087103e-01, 5.37083089e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 4.77655590e-01, 6.19648814e-01], [ 4.73001003e-01, 8.55485201e-01]]))
-
class
tinyms.primitives.ApplyProximalAdagrad(*args, **kwargs)[source]¶ Updates relevant entries according to the proximal adagrad algorithm.
\[accum += grad * grad\]\[\text{prox_v} = var - lr * grad * \frac{1}{\sqrt{accum}}\]\[var = \frac{sign(\text{prox_v})}{1 + lr * l2} * \max(\left| \text{prox_v} \right| - lr * l1, 0)\]Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – If true, the var and accumulation tensors will be protected from being updated. Default: False.
- Inputs:
var (Parameter) - Variable to be updated. The data type must be float16 or float32.
accum (Parameter) - Accumulation to be updated. Must has the same shape and dtype as var.
lr (Union[Number, Tensor]) - The learning rate value, must be scalar. The data type must be float16 or float32.
l1 (Union[Number, Tensor]) - l1 regularization strength, must be scalar. The data type must be float16 or float32.
l2 (Union[Number, Tensor]) - l2 regularization strength, must be scalar. The data type must be float16 or float32.
grad (Tensor) - Gradient with the same shape and dtype as var.
- Outputs:
Tuple of 2 Tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as accum.
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_proximal_adagrad = ops.ApplyProximalAdagrad() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="accum") ... self.lr = 0.01 ... self.l1 = 0.0 ... self.l2 = 0.0 ... def construct(self, grad): ... out = self.apply_proximal_adagrad(self.var, self.accum, self.lr, self.l1, self.l2, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.40526688e-01, 7.10883260e-01], [ 5.95089436e-01, 5.39996684e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 1.35230064e+00, 7.92921484e-01], [ 1.06441569e+00, 1.17150283e+00]]))
-
class
tinyms.primitives.ApplyProximalGradientDescent(*args, **kwargs)[source]¶ Updates relevant entries according to the FOBOS(Forward Backward Splitting) algorithm.
\[\text{prox_v} = var - \alpha * \delta\]\[var = \frac{sign(\text{prox_v})}{1 + \alpha * l2} * \max(\left| \text{prox_v} \right| - alpha * l1, 0)\]Inputs of var and delta comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
var (Parameter) - Variable tensor to be updated. With float32 or float16 data type.
alpha (Union[Number, Tensor]) - Scaling factor, must be a scalar. With float32 or float16 data type.
l1 (Union[Number, Tensor]) - l1 regularization strength, must be scalar. With float32 or float16 data type.
l2 (Union[Number, Tensor]) - l2 regularization strength, must be scalar. With float32 or float16 data type.
delta (Tensor) - A tensor for the change, has the same type as var.
- Outputs:
Tensor, represents the updated var.
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_proximal_gradient_descent = ops.ApplyProximalGradientDescent() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.alpha = 0.001 ... self.l1 = 0.0 ... self.l2 = 0.0 ... def construct(self, delta): ... out = self.apply_proximal_gradient_descent(self.var, self.alpha, self.l1, self.l2, delta) ... return out ... >>> net = Net() >>> delta = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(delta) >>> print(output.shape) (2, 2)
-
class
tinyms.primitives.ApplyRMSProp(*args, **kwargs)[source]¶ Optimizer that implements the Root Mean Square prop(RMSProp) algorithm. Please refer to the usage in source code of nn.RMSProp.
Note
Update var according to the RMSProp algorithm.
\[s_{t} = \rho s_{t-1} + (1 - \rho)(\nabla Q_{i}(w))^2\]\[m_{t} = \beta m_{t-1} + \frac{\eta} {\sqrt{s_{t} + \epsilon}} \nabla Q_{i}(w)\]\[w = w - m_{t}\]where \(w\) represents var, which will be updated. \(s_{t}\) represents mean_square, \(s_{t-1}\) is the last momentent of \(s_{t}\), \(m_{t}\) represents moment, \(m_{t-1}\) is the last momentent of \(m_{t}\). \(\rho\) represents decay. \(\beta\) is the momentum term, represents momentum. \(\epsilon\) is a smoothing term to avoid division by zero, represents epsilon. \(\eta\) represents learning_rate. \(\nabla Q_{i}(w)\) represents grad.
- Parameters
use_locking (bool) – Whether to enable a lock to protect the variable and accumlation tensors from being updated. Default: False.
- Inputs:
var (Tensor) - Weights to be update.
mean_square (Tensor) - Mean square gradients, must have the same type as var.
moment (Tensor) - Delta of var, must have the same type as var.
learning_rate (Union[Number, Tensor]) - Learning rate. Must be a float number or a scalar tensor with float16 or float32 data type.
grad (Tensor) - Gradient, must have the same type as var.
decay (float) - Decay rate. Only constant value is allowed.
momentum (float) - Momentum. Only constant value is allowed.
epsilon (float) - Ridge term. Only constant value is allowed.
- Outputs:
Tensor, parameters to be update.
- Supported Platforms:
AscendGPU
Examples
>>> apply_rms = ops.ApplyRMSProp() >>> input_x = Tensor(1., mindspore.float32) >>> mean_square = Tensor(2., mindspore.float32) >>> moment = Tensor(1., mindspore.float32) >>> grad = Tensor(2., mindspore.float32) >>> learning_rate = Tensor(0.9, mindspore.float32) >>> decay = 0.0 >>> momentum = 1e-10 >>> epsilon = 0.001 >>> output = apply_rms(input_x, mean_square, moment, learning_rate, grad, decay, momentum, epsilon) >>> print(output) (Tensor(shape=[], dtype=Float32, value= 0.100112), Tensor(shape=[], dtype=Float32, value= 4), Tensor(shape=[], dtype=Float32, value= 0.899888))
-
class
tinyms.primitives.ApproximateEqual(*args, **kwargs)[source]¶ Returns True if abs(x1-x2) is smaller than tolerance element-wise, otherwise False.
Inputs of x1 and x2 comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
tolerance (float) – The maximum deviation that two elements can be considered equal. Default: 1e-05.
- Inputs:
x1 (Tensor) - A tensor. Must be one of the following types: float32, float16.
x2 (Tensor) - A tensor of the same type and shape as ‘x1’.
- Outputs:
Tensor, the shape is the same as the shape of ‘x1’, and the data type is bool.
- Supported Platforms:
Ascend
Examples
>>> x1 = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> x2 = Tensor(np.array([2, 4, 6]), mindspore.float32) >>> approximate_equal = ops.ApproximateEqual(2.) >>> output = approximate_equal(x1, x2) >>> print(output) [ True True False]
-
class
tinyms.primitives.ArgMaxWithValue(*args, **kwargs)[source]¶ Calculates the maximum value with the corresponding index.
Calculates the maximum value along with the given axis for the input tensor. It returns the maximum values and indices.
Note
In auto_parallel and semi_auto_parallel mode, the first output index can not be used.
- Parameters
- Inputs:
input_x (Tensor) - The input tensor, can be any dimension. Set the shape of input tensor as \((x_1, x_2, ..., x_N)\).
- Outputs:
tuple (Tensor), tuple of 2 tensors, containing the corresponding index and the maximum value of the input tensor. - index (Tensor) - The index for the maximum value of the input tensor. If keep_dims is true, the shape of output tensors is \((x_1, x_2, ..., x_{axis-1}, 1, x_{axis+1}, ..., x_N)\). Otherwise, the shape is \((x_1, x_2, ..., x_{axis-1}, x_{axis+1}, ..., x_N)\). - output_x (Tensor) - The maximum value of input tensor, with the same shape as index.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]), mindspore.float32) >>> index, output = ops.ArgMaxWithValue()(input_x) >>> print(index, output) 3 0.7
-
class
tinyms.primitives.ArgMinWithValue(*args, **kwargs)[source]¶ Calculates the minimum value with corresponding index, and returns indices and values.
Calculates the minimum value along with the given axis for the input tensor. It returns the minimum values and indices.
Note
In auto_parallel and semi_auto_parallel mode, the first output index can not be used.
- Parameters
- Inputs:
input_x (Tensor) - The input tensor, can be any dimension. Set the shape of input tensor as \((x_1, x_2, ..., x_N)\).
- Outputs:
tuple (Tensor), tuple of 2 tensors, containing the corresponding index and the minimum value of the input tensor. - index (Tensor) - The index for the minimum value of the input tensor. If keep_dims is true, the shape of output tensors is \((x_1, x_2, ..., x_{axis-1}, 1, x_{axis+1}, ..., x_N)\). Otherwise, the shape is \((x_1, x_2, ..., x_{axis-1}, x_{axis+1}, ..., x_N)\). - output_x (Tensor) - The minimum value of input tensor, with the same shape as index.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]), mindspore.float32) >>> output = ops.ArgMinWithValue()(input_x) >>> print(output) (Tensor(shape=[], dtype=Int32, value= 0), Tensor(shape=[], dtype=Float32, value= 0.0))
-
class
tinyms.primitives.Argmax(*args, **kwargs)[source]¶ Returns the indices of the maximum value of a tensor across the axis.
If the shape of input tensor is \((x_1, ..., x_N)\), the shape of the output tensor will be \((x_1, ..., x_{axis-1}, x_{axis+1}, ..., x_N)\).
- Parameters
axis (int) – Axis where the Argmax operation applies to. Default: -1.
output_type (
mindspore.dtype) – An optional data type of mindspore.dtype.int32. Default: mindspore.dtype.int32.
- Inputs:
input_x (Tensor) - Input tensor.
- Outputs:
Tensor, indices of the max value of input tensor across the axis.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[1, 20, 5], [67, 8, 9], [130, 24, 15]]).astype(np.float32)) >>> output = ops.Argmax(output_type=mindspore.int32)(input_x) >>> print(output) [1 0 0]
-
class
tinyms.primitives.Argmin(*args, **kwargs)[source]¶ Returns the indices of the minimum value of a tensor across the axis.
If the shape of input tensor is \((x_1, ..., x_N)\), the shape of the output tensor is \((x_1, ..., x_{axis-1}, x_{axis+1}, ..., x_N)\).
- Parameters
axis (int) – Axis where the Argmin operation applies to. Default: -1.
output_type (
mindspore.dtype) – An optional data type of mindspore.dtype.int32. Default: mindspore.dtype.int32.
- Inputs:
input_x (Tensor) - Input tensor.
- Outputs:
Tensor, indices of the min value of input tensor across the axis.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([2.0, 3.1, 1.2]), mindspore.float32) >>> index = ops.Argmin()(input_x) >>> print(index) 2
-
class
tinyms.primitives.Asin(*args, **kwargs)[source]¶ Computes arcsine of input tensors element-wise.
\[out_i = sin^{-1}(x_i)\]- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Supported Platforms:
AscendGPU
Examples
>>> asin = ops.Asin() >>> input_x = Tensor(np.array([0.74, 0.04, 0.30, 0.56]), mindspore.float32) >>> output = asin(input_x) >>> print(output) [0.8330927 0.04001068 0.30469266 0.59438497]
-
class
tinyms.primitives.Asinh(*args, **kwargs)[source]¶ Computes inverse hyperbolic sine of the input element-wise.
\[out_i = sinh^{-1}(input_i)\]- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). The data type should be one of the following types: float16, float32.
- Outputs:
Tensor, has the same shape and type as input_x.
- Supported Platforms:
AscendGPU
Examples
>>> asinh = ops.Asinh() >>> input_x = Tensor(np.array([-5.0, 1.5, 3.0, 100.0]), mindspore.float32) >>> output = asinh(input_x) >>> print(output) [-2.3124385 1.1947632 1.8184465 5.298342 ]
-
class
tinyms.primitives.Assert(*args, **kwargs)[source]¶ Asserts that the given condition is True. If input condition evaluates to false, print the list of tensor in data.
- Parameters
summarize (int) – Print this many entries of each tensor.
- Inputs:
condition [Union[Tensor[bool], bool]] - The condition to evaluate.
input_data (Union(tuple[Tensor], list[Tensor])) - The tensors to print out when condition is false.
Examples
>>> class AssertDemo(nn.Cell): ... def __init__(self): ... super(AssertDemo, self).__init__() ... self.assert1 = ops.Assert(summarize=10) ... self.add = ops.Add() ... ... def construct(self, x, y): ... data = self.add(x, y) ... self.assert1(True, [data]) ... return data ...
-
class
tinyms.primitives.Assign(*args, **kwargs)[source]¶ Assigns Parameter with a value.
Inputs of variable and value comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
variable (Parameter) - The Parameter.
value (Tensor) - The value to be assigned.
- Outputs:
Tensor, has the same type as original variable.
- Supported Platforms:
AscendGPUCPU
Examples
>>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.y = mindspore.Parameter(Tensor([1.0], mindspore.float32), name="y") ... ... def construct(self, x): ... ops.Assign()(self.y, x) ... return self.y ... >>> x = Tensor([2.0], mindspore.float32) >>> net = Net() >>> output = net(x) >>> print(output) Parameter (name=y)
-
class
tinyms.primitives.AssignAdd(*args, **kwargs)[source]¶ Updates a Parameter by adding a value to it.
Inputs of variable and value comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. If value is a number, the number is automatically converted to Tensor, and the data type is consistent with the Tensor data type involved in the operation. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
variable (Parameter) - The Parameter.
value (Union[numbers.Number, Tensor]) - The value to be added to the variable. It must have the same shape as variable if it is a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.AssignAdd = ops.AssignAdd() ... self.variable = mindspore.Parameter(initializer(1, [1], mindspore.int64), name="global_step") ... ... def construct(self, x): ... self.AssignAdd(self.variable, x) ... return self.variable ... >>> net = Net() >>> value = Tensor(np.ones([1]).astype(np.int64)*100) >>> output = net(value) >>> print(output) Parameter (name=global_step)
-
class
tinyms.primitives.AssignSub(*args, **kwargs)[source]¶ Updates a Parameter by subtracting a value from it.
Inputs of variable and value comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. If value is a number, the number is automatically converted to Tensor, and the data type is consistent with the Tensor data type involved in the operation. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
variable (Parameter) - The Parameter.
value (Union[numbers.Number, Tensor]) - The value to be subtracted from the variable. It must have the same shape as variable if it is a Tensor.
- Supported Platforms:
Ascend
Examples
>>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.AssignSub = ops.AssignSub() ... self.variable = mindspore.Parameter(initializer(1, [1], mindspore.int32), name="global_step") ... ... def construct(self, x): ... self.AssignSub(self.variable, x) ... return self.variable ... >>> net = Net() >>> value = Tensor(np.ones([1]).astype(np.int32)*100) >>> output = net(value) >>> print(output) Parameter (name=global_step)
-
class
tinyms.primitives.Atan(*args, **kwargs)[source]¶ Computes the trigonometric inverse tangent of the input element-wise.
\[out_i = tan^{-1}(x_i)\]- Inputs:
input_x (Tensor): The input tensor. The data type should be one of the following types: float16, float32.
- Outputs:
A Tensor, has the same type as the input.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([1.0, 0.0]), mindspore.float32) >>> atan = ops.Atan() >>> output = atan(input_x) >>> print(output) [0.7853982 0. ]
-
class
tinyms.primitives.Atan2(*args, **kwargs)[source]¶ Returns arctangent of input_x/input_y element-wise.
It returns \(\theta\ \in\ [-\pi, \pi]\) such that \(x = r*\sin(\theta), y = r*\cos(\theta)\), where \(r = \sqrt{x^2 + y^2}\).
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
input_x (Tensor) - The input tensor.
input_y (Tensor) - The input tensor.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is same as input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([0, 1]), mindspore.float32) >>> input_y = Tensor(np.array([1, 1]), mindspore.float32) >>> atan2 = ops.Atan2() >>> output = atan2(input_x, input_y) >>> print(output) [0. 0.7853982]
-
class
tinyms.primitives.Atanh(*args, **kwargs)[source]¶ Computes inverse hyperbolic tangent of the input element-wise.
- Inputs:
input_x (Tensor): The input tensor.
- Outputs:
A Tensor, has the same type as the input.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([1.047, 0.785]), mindspore.float32) >>> atanh = ops.Atanh() >>> output = atanh(input_x) >>> print(output) [1.8869909 1.058268 ]
-
class
tinyms.primitives.AvgPool(*args, **kwargs)[source]¶ Average pooling operation.
Applies a 2D average pooling over an input Tensor which can be regarded as a composition of 2D input planes. Typically the input is of shape \((N_{in}, C_{in}, H_{in}, W_{in})\), AvgPool2d outputs regional average in the \((H_{in}, W_{in})\)-dimension. Given kernel size \(ks = (h_{ker}, w_{ker})\) and stride \(s = (s_0, s_1)\), the operation is as follows.
\[\text{output}(N_i, C_j, h, w) = \frac{1}{h_{ker} * w_{ker}} \sum_{m=0}^{h_{ker}-1} \sum_{n=0}^{w_{ker}-1} \text{input}(N_i, C_j, s_0 \times h + m, s_1 \times w + n)\]- Parameters
kernel_size (Union[int, tuple[int]]) – The size of kernel used to take the average value, is an int number that represents height and width are both kernel_size, or a tuple of two int numbers that represent height and width respectively. Default: 1.
strides (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Default: 1.
pad_mode (str) –
The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
data_format (str) – default is ‘NCHW’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor, with shape \((N, C_{out}, H_{out}, W_{out})\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.avgpool_op = ops.AvgPool(pad_mode="VALID", kernel_size=2, strides=1) ... ... def construct(self, x): ... result = self.avgpool_op(x) ... return result ... >>> input_x = Tensor(np.arange(1 * 3 * 3 * 4).reshape(1, 3, 3, 4), mindspore.float32) >>> net = Net() >>> output = net(input_x) >>> print(output) [[[[ 2.5 3.5 4.5] [ 6.5 7.5 8.5]] [[14.5 15.5 16.5] [18.5 19.5 20.5]] [[26.5 27.5 28.5] [30.5 31.5 32.5]]]]
-
class
tinyms.primitives.BNTrainingReduce(*args, **kwargs)[source]¶ For the BatchNorm operation this operator update the moving averages for training and is used in conjunction with BNTrainingUpdate.
- Inputs:
x (Tensor) - A 4-D Tensor with float16 or float32 data type. Tensor of shape \((N, C, A, B)\).
- Outputs:
sum (Tensor) - A 1-D Tensor with float32 data type. Tensor of shape \((C,)\).
square_sum (Tensor) - A 1-D Tensor with float32 data type. Tensor of shape \((C,)\).
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.ones([128, 3, 32, 3]), mindspore.float32) >>> bn_training_reduce = ops.BNTrainingReduce() >>> output = bn_training_reduce(input_x) >>> print(output) (Tensor(shape=[3], dtype=Float32, value= [ 1.22880000e+04, 1.22880000e+04, 1.22880000e+04]), Tensor(shape=[3], dtype=Float32, value= [ 1.22880000e+04, 1.22880000e+04, 1.22880000e+04]))
-
class
tinyms.primitives.BNTrainingUpdate(*args, **kwargs)[source]¶ For the BatchNorm operation, this operator update the moving averages for training and is used in conjunction with BNTrainingReduce.
- Parameters
- Inputs:
x (Tensor) - A 4-D Tensor with float16 or float32 data type. Tensor of shape \((N, C, A, B)\).
sum (Tensor) - A 1-D Tensor with float16 or float32 data type for the output of operator BNTrainingReduce. Tensor of shape \((C,)\).
square_sum (Tensor) - A 1-D Tensor with float16 or float32 data type for the output of operator BNTrainingReduce. Tensor of shape \((C,)\).
scale (Tensor) - A 1-D Tensor with float16 or float32, for the scaling factor. Tensor of shape \((C,)\).
offset (Tensor) - A 1-D Tensor with float16 or float32, for the scaling offset. Tensor of shape \((C,)\).
mean (Tensor) - A 1-D Tensor with float16 or float32, for the scaling mean. Tensor of shape \((C,)\).
variance (Tensor) - A 1-D Tensor with float16 or float32, for the update variance. Tensor of shape \((C,)\).
- Outputs:
y (Tensor) - Tensor, has the same shape data type as x.
mean (Tensor) - Tensor for the updated mean, with float32 data type. Has the same shape as variance.
variance (Tensor) - Tensor for the updated variance, with float32 data type. Has the same shape as variance.
batch_mean (Tensor) - Tensor for the mean of x, with float32 data type. Has the same shape as variance.
batch_variance (Tensor) - Tensor for the mean of variance, with float32 data type. Has the same shape as variance.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.ones([1, 2, 2, 2]), mindspore.float32) >>> sum = Tensor(np.ones([2]), mindspore.float32) >>> square_sum = Tensor(np.ones([2]), mindspore.float32) >>> scale = Tensor(np.ones([2]), mindspore.float32) >>> offset = Tensor(np.ones([2]), mindspore.float32) >>> mean = Tensor(np.ones([2]), mindspore.float32) >>> variance = Tensor(np.ones([2]), mindspore.float32) >>> bn_training_update = ops.BNTrainingUpdate() >>> output = bn_training_update(input_x, sum, square_sum, scale, offset, mean, variance) >>> print(output) (Tensor(shape=[1, 2, 2, 2], dtype=Float32, value= [[[[ 2.73200464e+00, 2.73200464e+00], [ 2.73200464e+00, 2.73200464e+00]], [[ 2.73200464e+00, 2.73200464e+00], [ 2.73200464e+00, 2.73200464e+00]]]]), Tensor(shape=[1, 2, 2, 2], dtype=Float32, value= [[[[ 2.73200464e+00, 2.73200464e+00], [ 2.73200464e+00, 2.73200464e+00]], [[ 2.73200464e+00, 2.73200464e+00], [ 2.73200464e+00, 2.73200464e+00]]]]), Tensor(shape=[1, 2, 2, 2], dtype=Float32, value= [[[[ 2.73200464e+00, 2.73200464e+00], [ 2.73200464e+00, 2.73200464e+00]], [[ 2.73200464e+00, 2.73200464e+00], [ 2.73200464e+00, 2.73200464e+00]]]]), Tensor(shape=[2], dtype=Float32, value= [ 2.50000000e-01, 2.50000000e-01]), Tensor(shape=[2], dtype=Float32, value= [ 1.87500000e-01, 1.87500000e-01]))
-
class
tinyms.primitives.BasicLSTMCell(*args, **kwargs)[source]¶ Applies the long short-term memory (LSTM) to the input.
\[\begin{split}\begin{array}{ll} \\ i_t = \sigma(W_{ix} x_t + b_{ix} + W_{ih} h_{(t-1)} + b_{ih}) \\ f_t = \sigma(W_{fx} x_t + b_{fx} + W_{fh} h_{(t-1)} + b_{fh}) \\ \tilde{c}_t = \tanh(W_{cx} x_t + b_{cx} + W_{ch} h_{(t-1)} + b_{ch}) \\ o_t = \sigma(W_{ox} x_t + b_{ox} + W_{oh} h_{(t-1)} + b_{oh}) \\ c_t = f_t * c_{(t-1)} + i_t * \tilde{c}_t \\ h_t = o_t * \tanh(c_t) \\ \end{array}\end{split}\]Here \(\sigma\) is the sigmoid function, and \(*\) is the Hadamard product. \(W, b\) are learnable weights between the output and the input in the formula. For instance, \(W_{ix}, b_{ix}\) are the weight and bias used to transform from input \(x\) to \(i\). Details can be found in paper LONG SHORT-TERM MEMORY and Long Short-Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling.
- Parameters
keep_prob (float) – If not 1.0, append Dropout layer on the outputs of each LSTM layer except the last layer. Default 1.0. The range of dropout is [0.0, 1.0].
forget_bias (float) – Add forget bias to forget gate biases in order to decrease former scale. Default: 1.0.
state_is_tuple (bool) – If true, the state is a tuple of 2 tensors, containing h and c; If false, the state is a tensor and it needs to be split first. Default: True.
activation (str) – Activation. Default: “tanh”. Only “tanh” is currently supported.
- Inputs:
x (Tensor) - Current words. Tensor of shape (batch_size, input_size). The data type must be float16 or float32.
h (Tensor) - Hidden state last moment. Tensor of shape (batch_size, hidden_size). The data type must be float16 or float32.
c (Tensor) - Cell state last moment. Tensor of shape (batch_size, hidden_size). The data type must be float16 or float32.
w (Tensor) - Weight. Tensor of shape (input_size + hidden_size, 4 x hidden_size). The data type must be float16 or float32.
b (Tensor) - Bias. Tensor of shape (4 x hidden_size). The data type must be the same as c.
- Outputs:
ct (Tensor) - Forward \(c_t\) cache at moment t. Tensor of shape (batch_size, hidden_size). Has the same type with input c.
ht (Tensor) - Cell output. Tensor of shape (batch_size, hidden_size). With data type of float16.
it (Tensor) - Forward \(i_t\) cache at moment t. Tensor of shape (batch_size, hidden_size). Has the same type with input c.
jt (Tensor) - Forward \(j_t\) cache at moment t. Tensor of shape (batch_size, hidden_size). Has the same type with input c.
ft (Tensor) - Forward \(f_t\) cache at moment t. Tensor of shape (batch_size, hidden_size). Has the same type with input c.
ot (Tensor) - Forward \(o_t\) cache at moment t. Tensor of shape (batch_size, hidden_size). Has the same type with input c.
tanhct (Tensor) - Forward \(tanh c_t\) cache at moment t. Tensor of shape (batch_size, hidden_size), has the same type with input c.
- Supported Platforms:
Ascend
Examples
>>> np.random.seed(0) >>> x = Tensor(np.random.rand(1, 32).astype(np.float16)) >>> h = Tensor(np.random.rand(1, 2).astype(np.float16)) >>> c = Tensor(np.random.rand(1, 2).astype(np.float16)) >>> w = Tensor(np.random.rand(34, 8).astype(np.float16)) >>> b = Tensor(np.random.rand(8, ).astype(np.float16)) >>> lstm = ops.BasicLSTMCell(keep_prob=1.0, forget_bias=1.0, state_is_tuple=True, activation='tanh') >>> output = lstm(x, h, c, w, b) >>> print(output) (Tensor(shape=[1, 2], dtype=Float16, value= ([[7.6953e-01, 9.2432e-01]]), Tensor(shape=[1, 2], dtype=Float16, value= [[1.0000e+00, 1.0000e+00]]), Tensor(shape=[1, 2], dtype=Float16, value= [[1.0000e+00, 1.0000e+00]]), Tensor(shape=[1, 2], dtype=Float16, value= [[1.0000e+00, 1.0000e+00]]), Tensor(shape=[1, 2], dtype=Float16, value= [[1.0000e+00, 1.0000e+00]]), Tensor(shape=[1, 2], dtype=Float16, value= [[7.6953e-01, 9.2432e-01]]), Tensor(shape=[1, 2], dtype=Float16, value= [[0.0000e+00, 0.0000e+00]]))
-
class
tinyms.primitives.BatchMatMul(*args, **kwargs)[source]¶ Computes matrix multiplication between two tensors by batch
result[…, :, :] = tensor(a[…, :, :]) * tensor(b[…, :, :]).
The two input tensors must have the same rank and the rank must be not less than 3.
- Parameters
- Inputs:
input_x (Tensor) - The first tensor to be multiplied. The shape of the tensor is \((*B, N, C)\), where \(*B\) represents the batch size which can be multidimensional, \(N\) and \(C\) are the size of the last two dimensions. If transpose_a is True, its shape must be \((*B, C, N)\).
input_y (Tensor) - The second tensor to be multiplied. The shape of the tensor is \((*B, C, M)\). If transpose_b is True, its shape must be \((*B, M, C)\).
- Outputs:
Tensor, the shape of the output tensor is \((*B, N, M)\).
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.ones(shape=[2, 4, 1, 3]), mindspore.float32) >>> input_y = Tensor(np.ones(shape=[2, 4, 3, 4]), mindspore.float32) >>> batmatmul = ops.BatchMatMul() >>> output = batmatmul(input_x, input_y) >>> print(output) [[[[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]]], [[[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]]]] >>> >>> input_x = Tensor(np.ones(shape=[2, 4, 3, 1]), mindspore.float32) >>> input_y = Tensor(np.ones(shape=[2, 4, 3, 4]), mindspore.float32) >>> batmatmul = ops.BatchMatMul(transpose_a=True) >>> output = batmatmul(input_x, input_y) >>> print(output) [[[[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]]], [[[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]]]]
-
class
tinyms.primitives.BatchNorm(*args, **kwargs)[source]¶ Batch Normalization for input data and updated parameters.
Batch Normalization is widely used in convolutional neural networks. This operation applies Batch Normalization over input to avoid internal covariate shift as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. It rescales and recenters the features using a mini-batch of data and the learned parameters which can be described in the following formula,
\[y = \frac{x - mean}{\sqrt{variance + \epsilon}} * \gamma + \beta\]where \(\gamma\) is scale, \(\beta\) is bias, \(\epsilon\) is epsilon.
- Parameters
is_training (bool) – If is_training is True, mean and variance are computed during training. If is_training is False, they’re loaded from checkpoint during inference. Default: False.
epsilon (float) – A small value added for numerical stability. Default: 1e-5.
data_format (str) – The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: “NCHW”.
- Inputs:
input_x (Tensor) - Tensor of shape \((N, C)\), with float16 or float32 data type.
scale (Tensor) - Tensor of shape \((C,)\), with float16 or float32 data type.
bias (Tensor) - Tensor of shape \((C,)\), has the same data type with scale.
mean (Tensor) - Tensor of shape \((C,)\), with float16 or float32 data type.
variance (Tensor) - Tensor of shape \((C,)\), has the same data type with mean.
- Outputs:
Tuple of 5 Tensor, the normalized inputs and the updated parameters.
output_x (Tensor) - The same type and shape as the input_x. The shape is \((N, C)\).
updated_scale (Tensor) - Tensor of shape \((C,)\).
updated_bias (Tensor) - Tensor of shape \((C,)\).
reserve_space_1 (Tensor) - Tensor of shape \((C,)\).
reserve_space_2 (Tensor) - Tensor of shape \((C,)\).
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.ones([2, 2]), mindspore.float32) >>> scale = Tensor(np.ones([2]), mindspore.float32) >>> bias = Tensor(np.ones([2]), mindspore.float32) >>> mean = Tensor(np.ones([2]), mindspore.float32) >>> variance = Tensor(np.ones([2]), mindspore.float32) >>> batch_norm = ops.BatchNorm() >>> output = batch_norm(input_x, scale, bias, mean, variance) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 1.00000000e+00, 1.00000000e+00], [ 1.00000000e+00, 1.00000000e+00]]), Tensor(shape=[2], dtype=Float32, value= [ 1.00000000e+00, 1.00000000e+00]), Tensor(shape=[2], dtype=Float32, value= [ 1.00000000e+00, 1.00000000e+00]), Tensor(shape=[2], dtype=Float32, value= [ 1.00000000e+00, 1.00000000e+00]), Tensor(shape=[2], dtype=Float32, value= [ 1.00000000e+00, 1.00000000e+00]))
-
class
tinyms.primitives.BatchToSpace(*args, **kwargs)[source]¶ Divides batch dimension with blocks and interleaves these blocks back into spatial dimensions.
This operation will divide batch dimension N into blocks with block_size, the output tensor’s N dimension is the corresponding number of blocks after division. The output tensor’s H, W dimension is product of original H, W dimension and block_size with given amount to crop from dimension, respectively.
- Parameters
block_size (int) – The block size of division, has the value not less than 2.
crops (Union[list(int), tuple(int)]) – The crop value for H and W dimension, containing 2 subtraction lists. Each list contains 2 integers. All values must be not less than 0. crops[i] specifies the crop values for the spatial dimension i, which corresponds to the input dimension i+2. It is required that input_shape[i+2]*block_size >= crops[i][0]+crops[i][1].
- Inputs:
input_x (Tensor) - The input tensor. It must be a 4-D tensor, dimension 0 must be divisible by product of block_shape.
- Outputs:
Tensor, the output tensor with the same type as input. Assume input shape is (n, c, h, w) with block_size and crops. The output shape will be (n’, c’, h’, w’), where
\(n' = n//(block\_size*block\_size)\)
\(c' = c\)
\(h' = h*block\_size-crops[0][0]-crops[0][1]\)
\(w' = w*block\_size-crops[1][0]-crops[1][1]\)
- Supported Platforms:
Ascend
Examples
>>> block_size = 2 >>> crops = [[0, 0], [0, 0]] >>> batch_to_space = ops.BatchToSpace(block_size, crops) >>> input_x = Tensor(np.array([[[[1]]], [[[2]]], [[[3]]], [[[4]]]]), mindspore.float32) >>> output = batch_to_space(input_x) >>> print(output) [[[[1. 2.] [3. 4.]]]]
-
class
tinyms.primitives.BatchToSpaceND(*args, **kwargs)[source]¶ Divides batch dimension with blocks and interleaves these blocks back into spatial dimensions.
This operation will divide batch dimension N into blocks with block_shape, the output tensor’s N dimension is the corresponding number of blocks after division. The output tensor’s H, W dimension is product of original H, W dimension and block_shape with given amount to crop from dimension, respectively.
- Parameters
block_shape (Union[list(int), tuple(int), int]) – The block shape of dividing block with all value greater than 1. If block_shape is a tuple or list, the length of block_shape is M corresponding to the number of spatial dimensions. If block_shape is a int, the block size of M dimendions are the same, equal to block_shape. M must be 2.
crops (Union[list(int), tuple(int)]) – The crop value for H and W dimension, containing 2 subtraction list, each containing 2 int value. All values must be >= 0. crops[i] specifies the crop values for spatial dimension i, which corresponds to input dimension i+2. It is required that input_shape[i+2]*block_shape[i] > crops[i][0]+crops[i][1].
- Inputs:
input_x (Tensor) - The input tensor. It must be a 4-D tensor, dimension 0 must be divisible by product of block_shape.
- Outputs:
Tensor, the output tensor with the same type as input. Assume input shape is (n, c, h, w) with block_shape and crops. The output shape will be (n’, c’, h’, w’), where
\(n' = n//(block\_shape[0]*block\_shape[1])\)
\(c' = c\)
\(h' = h*block\_shape[0]-crops[0][0]-crops[0][1]\)
\(w' = w*block\_shape[1]-crops[1][0]-crops[1][1]\)
- Supported Platforms:
Ascend
Examples
>>> block_shape = [2, 2] >>> crops = [[0, 0], [0, 0]] >>> batch_to_space_nd = ops.BatchToSpaceND(block_shape, crops) >>> input_x = Tensor(np.array([[[[1]]], [[[2]]], [[[3]]], [[[4]]]]), mindspore.float32) >>> output = batch_to_space_nd(input_x) >>> print(output) [[[[1. 2.] [3. 4.]]]]
-
class
tinyms.primitives.BesselI0e(*args, **kwargs)[source]¶ Computes BesselI0e of input element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). Data type must be float16 or float32.
- Outputs:
Tensor, has the same shape as input_x.
- Supported Platforms:
Ascend
Examples
>>> bessel_i0e = ops.BesselI0e() >>> input_x = Tensor(np.array([0.24, 0.83, 0.31, 0.09]), mindspore.float32) >>> output = bessel_i0e(input_x) >>> print(output) [0.7979961 0.5144438 0.75117415 0.9157829 ]
-
class
tinyms.primitives.BesselI1e(*args, **kwargs)[source]¶ Computes BesselI1e of input element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). Data type must be float16 or float32.
- Outputs:
Tensor, has the same shape as input_x.
- Supported Platforms:
Ascend
Examples
>>> bessel_i1e = ops.BesselI1e() >>> input_x = Tensor(np.array([0.24, 0.83, 0.31, 0.09]), mindspore.float32) >>> output = bessel_i1e(input_x) >>> print(output) [0.09507662 0.19699717 0.11505538 0.04116856]
-
class
tinyms.primitives.BiasAdd(*args, **kwargs)[source]¶ Returns sum of input and bias tensor.
Adds the 1-D bias tensor to the input tensor, and broadcasts the shape on all axis except for the channel axis.
- Inputs:
input_x (Tensor) - The input tensor. The shape can be 2-4 dimensions.
bias (Tensor) - The bias tensor, with shape \((C)\).
data_format (str) - The format of input and output data. It should be ‘NHWC’ or ‘NCHW’，default is ‘NCHW’. The shape of bias must be the same as input_x in the second dimension.
- Outputs:
Tensor, with the same shape and type as input_x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.arange(6).reshape((2, 3)), mindspore.float32) >>> bias = Tensor(np.random.random(3).reshape((3,)), mindspore.float32) >>> bias_add = ops.BiasAdd() >>> output = bias_add(input_x, bias) >>> print(output.shape) (2, 3)
-
class
tinyms.primitives.BinaryCrossEntropy(*args, **kwargs)[source]¶ Computes the binary cross entropy between the target and the output.
Sets input as \(x\), input label as \(y\), output as \(\ell(x, y)\). Let,
\[L = \{l_1,\dots,l_N\}^\top, \quad l_n = - w_n \left[ y_n \cdot \log x_n + (1 - y_n) \cdot \log (1 - x_n) \right]\]Then,
\[\begin{split}\ell(x, y) = \begin{cases} L, & \text{if reduction} = \text{'none';}\\ \operatorname{mean}(L), & \text{if reduction} = \text{'mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]- Parameters
reduction (str) – Specifies the reduction to be applied to the output. Its value must be one of ‘none’, ‘mean’, ‘sum’. Default: ‘mean’.
- Inputs:
input_x (Tensor) - The input Tensor. The data type must be float16 or float32.
input_y (Tensor) - The label Tensor which has same shape and data type as input_x.
weight (Tensor, optional) - A rescaling weight applied to the loss of each batch element. And it must have same shape and data type as input_x. Default: None.
- Outputs:
Tensor or Scalar, if reduction is ‘none’, then output is a tensor and has the same shape as input_x. Otherwise, the output is a scalar.
- Supported Platforms:
AscendGPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.binary_cross_entropy = ops.BinaryCrossEntropy() ... def construct(self, x, y, weight): ... result = self.binary_cross_entropy(x, y, weight) ... return result ... >>> net = Net() >>> input_x = Tensor(np.array([0.2, 0.7, 0.1]), mindspore.float32) >>> input_y = Tensor(np.array([0., 1., 0.]), mindspore.float32) >>> weight = Tensor(np.array([1, 2, 2]), mindspore.float32) >>> output = net(input_x, input_y, weight) >>> print(output) 0.38240486
-
class
tinyms.primitives.BitwiseAnd(*args, **kwargs)[source]¶ Returns bitwise and of two tensors element-wise.
Inputs of input_x1 and input_x2 comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
input_x1 (Tensor) - The input tensor with int16, int32 or uint16 data type.
input_x2 (Tensor) - The input tensor with same type as the input_x1.
- Outputs:
Tensor, has the same type as the input_x1.
- Supported Platforms:
Ascend
Examples
>>> input_x1 = Tensor(np.array([0, 0, 1, -1, 1, 1, 1]), mindspore.int16) >>> input_x2 = Tensor(np.array([0, 1, 1, -1, -1, 2, 3]), mindspore.int16) >>> bitwise_and = ops.BitwiseAnd() >>> output = bitwise_and(input_x1, input_x2) >>> print(output) [ 0 0 1 -1 1 0 1]
-
class
tinyms.primitives.BitwiseOr(*args, **kwargs)[source]¶ Returns bitwise or of two tensors element-wise.
Inputs of input_x1 and input_x2 comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
input_x1 (Tensor) - The input tensor with int16, int32 or uint16 data type.
input_x2 (Tensor) - The input tensor with same type as the input_x1.
- Outputs:
Tensor, has the same type as the input_x1.
- Supported Platforms:
Ascend
Examples
>>> input_x1 = Tensor(np.array([0, 0, 1, -1, 1, 1, 1]), mindspore.int16) >>> input_x2 = Tensor(np.array([0, 1, 1, -1, -1, 2, 3]), mindspore.int16) >>> bitwise_or = ops.BitwiseOr() >>> output = bitwise_or(input_x1, input_x2) >>> print(output) [ 0 1 1 -1 -1 3 3]
-
class
tinyms.primitives.BitwiseXor(*args, **kwargs)[source]¶ Returns bitwise xor of two tensors element-wise.
Inputs of input_x1 and input_x2 comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
input_x1 (Tensor) - The input tensor with int16, int32 or uint16 data type.
input_x2 (Tensor) - The input tensor with same type as the input_x1.
- Outputs:
Tensor, has the same type as the input_x1.
- Supported Platforms:
Ascend
Examples
>>> input_x1 = Tensor(np.array([0, 0, 1, -1, 1, 1, 1]), mindspore.int16) >>> input_x2 = Tensor(np.array([0, 1, 1, -1, -1, 2, 3]), mindspore.int16) >>> bitwise_xor = ops.BitwiseXor() >>> output = bitwise_xor(input_x1, input_x2) >>> print(output) [ 0 1 0 0 -2 3 2]
-
class
tinyms.primitives.BoundingBoxDecode(*args, **kwargs)[source]¶ Decodes bounding boxes locations.
- Parameters
means (tuple) – The means of deltas calculation. Default: (0.0, 0.0, 0.0, 0.0).
stds (tuple) – The standard deviations of deltas calculation. Default: (1.0, 1.0, 1.0, 1.0).
max_shape (tuple) – The max size limit for decoding box calculation.
wh_ratio_clip (float) – The limit of width and height ratio for decoding box calculation. Default: 0.016.
- Inputs:
anchor_box (Tensor) - Anchor boxes. The shape of anchor_box must be (n, 4).
deltas (Tensor) - Delta of boxes. Which has the same shape with anchor_box.
- Outputs:
Tensor, decoded boxes.
- Supported Platforms:
AscendGPU
Examples
>>> anchor_box = Tensor([[4, 1, 2, 1], [2, 2, 2, 3]], mindspore.float32) >>> deltas = Tensor([[3, 1, 2, 2], [1, 2, 1, 4]], mindspore.float32) >>> boundingbox_decode = ops.BoundingBoxDecode(means=(0.0, 0.0, 0.0, 0.0), stds=(1.0, 1.0, 1.0, 1.0), ... max_shape=(768, 1280), wh_ratio_clip=0.016) >>> output = boundingbox_decode(anchor_box, deltas) >>> print(output) [[ 4.1953125 0. 0. 5.1953125] [ 2.140625 0. 3.859375 60.59375 ]]
-
class
tinyms.primitives.BoundingBoxEncode(*args, **kwargs)[source]¶ Encodes bounding boxes locations.
- Parameters
- Inputs:
anchor_box (Tensor) - Anchor boxes. The shape of anchor_box must be (n, 4).
groundtruth_box (Tensor) - Ground truth boxes. Which has the same shape with anchor_box.
- Outputs:
Tensor, encoded bounding boxes.
- Supported Platforms:
AscendGPU
Examples
>>> anchor_box = Tensor([[2, 2, 2, 3], [2, 2, 2, 3]], mindspore.float32) >>> groundtruth_box = Tensor([[1, 2, 1, 4], [1, 2, 1, 4]], mindspore.float32) >>> boundingbox_encode = ops.BoundingBoxEncode(means=(0.0, 0.0, 0.0, 0.0), stds=(1.0, 1.0, 1.0, 1.0)) >>> output = boundingbox_encode(anchor_box, groundtruth_box) >>> print(output) [[ －1. 0.25 0. 0.40551758] [ －1. 0.25 0. 0.40551758]]
-
class
tinyms.primitives.Broadcast(*args, **kwargs)[source]¶ Broadcasts the tensor to the whole group.
Note
The tensors must have the same shape and format in all processes of the collection.
- Parameters
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape of the input, i.e., \((x_1, x_2, ..., x_R)\). The contents depend on the data of the root_rank device.
- Raises
TypeError – If root_rank is not a integer or group is not a string.
- Supported Platforms:
Ascend,GPU
Examples
>>> # This example should be run with multiple processes. >>> # Please refer to the tutorial > Distributed Training on mindspore.cn. >>> from mindspore import Tensor >>> from mindspore import context >>> from mindspore.communication import init >>> import mindspore.nn as nn >>> import mindspore.ops.operations as ops >>> import numpy as np >>> >>> context.set_context(mode=context.GRAPH_MODE) >>> init() >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.broadcast = ops.Broadcast(1) ... ... def construct(self, x): ... return self.broadcast((x,)) ... >>> input_ = Tensor(np.ones([2, 4]).astype(np.int32)) >>> net = Net() >>> output = net(input_) >>> print(output) (Tensor(shape[2,4], dtype=Int32, value= [[1, 1, 1, 1], [1, 1, 1, 1]]),)
-
class
tinyms.primitives.BroadcastTo(*args, **kwargs)[source]¶ Broadcasts input tensor to a given shape.
Input shape can be broadcast to target shape if for each dimension pair they are either equal or input is one or the target dimension is -1. In case of -1 in target shape, it will be replaced by the input shape’s value in that dimension.
When input shape is broadcast to target shape, it starts with the trailing dimensions.
- Parameters
shape (tuple) – The target shape to broadcast. Can be fully specified, or have -1 in one position where it will be substituted by the input tensor’s shape in that position, see example.
- Inputs:
input_x (Tensor) - The input tensor. The data type should be one of the following types: float16, float32, int32, int8, uint8.
- Outputs:
Tensor, with the given shape and the same data type as input_x.
- Raises
ValueError – Given a shape tuple, if it has several -1; or if the -1 is in an invalid position such as one that does not have a opposing dimension in an input tensor; or if the target and input shapes are incompatible.
- Supported Platforms:
AscendGPU
Examples
>>> shape = (2, 3) >>> input_x = Tensor(np.array([1, 2, 3]).astype(np.float32)) >>> broadcast_to = ops.BroadcastTo(shape) >>> output = broadcast_to(input_x) >>> print(output) [[1. 2. 3.] [1. 2. 3.]]
>>> shape = (2, -1) >>> input_x = Tensor(np.array([1, 2, 3]).astype(np.float32)) >>> broadcast_to = ops.BroadcastTo(shape) >>> output = broadcast_to(input_x) >>> print(output) [[1. 2. 3.] [1. 2. 3.]]
-
class
tinyms.primitives.CTCGreedyDecoder(*args, **kwargs)[source]¶ Performs greedy decoding on the logits given in inputs.
- Parameters
merge_repeated (bool) – If true, merge repeated classes in output. Default: True.
- Inputs:
inputs (Tensor) - The input Tensor must be a 3-D tensor whose shape is (max_time, batch_size, num_classes). num_classes must be num_labels + 1 classes, num_labels indicates the number of actual labels. Blank labels are reserved. Default blank label is num_classes - 1. Data type must be float32 or float64.
sequence_length (Tensor) - A tensor containing sequence lengths with the shape of (batch_size). The type must be int32. Each value in the tensor must be equal to or less than max_time.
- Outputs:
decoded_indices (Tensor) - A tensor with shape of (total_decoded_outputs, 2). Data type is int64.
decoded_values (Tensor) - A tensor with shape of (total_decoded_outputs), it stores the decoded classes. Data type is int64.
decoded_shape (Tensor) - The value of tensor is [batch_size, max_decoded_legth]. Data type is int64.
log_probability (Tensor) - A tensor with shape of (batch_size, 1), containing sequence log-probability, has the same type as inputs.
Examples
>>> class CTCGreedyDecoderNet(nn.Cell): ... def __init__(self): ... super(CTCGreedyDecoderNet, self).__init__() ... self.ctc_greedy_decoder = P.CTCGreedyDecoder() ... self.assert_op = ops.Assert(300) ... ... def construct(self, inputs, sequence_length): ... out = self.ctc_greedy_decoder(inputs,sequence_length) ... self.assert_op(True, (out[0], out[1], out[2], out[3])) ... return out[2] ... >>> inputs = Tensor(np.random.random((2, 2, 3)), mindspore.float32) >>> sequence_length = Tensor(np.array([2, 2]), mindspore.int32) >>> net = CTCGreedyDecoderNet() >>> output = net(inputs, sequence_length) >>> print(output)
-
class
tinyms.primitives.CTCLoss(*args, **kwargs)[source]¶ Calculates the CTC (Connectionist Temporal Classification) loss and the gradient.
- Parameters
preprocess_collapse_repeated (bool) – If true, repeated labels will be collapsed prior to the CTC calculation. Default: False.
ctc_merge_repeated (bool) – If false, during CTC calculation, repeated non-blank labels will not be merged and these labels will be interpreted as individual ones. This is a simplfied version of CTC. Default: True.
ignore_longer_outputs_than_inputs (bool) – If true, sequences with longer outputs than inputs will be ignored. Default: False.
- Inputs:
inputs (Tensor) - The input Tensor must be a 3-D tensor whose shape is (max_time, batch_size, num_classes). num_classes must be num_labels + 1 classes, num_labels indicates the number of actual labels. Blank labels are reserved. Default blank label is num_classes - 1. Data type must be float16, float32 or float64.
labels_indices (Tensor) - The indices of labels. labels_indices[i, :] == [b, t] means labels_values[i] stores the id for (batch b, time t). The type must be int64 and rank must be 2.
labels_values (Tensor) - A 1-D input tensor. The values are associated with the given batch and time. The type must be int32. labels_values[i] must in the range of [0, num_classes).
sequence_length (Tensor) - A tensor containing sequence lengths with the shape of (batch_size). The type must be int32. Each value in the tensor must not be greater than max_time.
- Outputs:
loss (Tensor) - A tensor containing log-probabilities, the shape is (batch_size). The tensor has the same type with inputs.
gradient (Tensor) - The gradient of loss, has the same type and shape with inputs.
- Supported Platforms:
AscendGPU
Examples
>>> np.random.seed(0) >>> inputs = Tensor(np.random.random((2, 2, 3)), mindspore.float32) >>> labels_indices = Tensor(np.array([[0, 0], [1, 0]]), mindspore.int64) >>> labels_values = Tensor(np.array([2, 2]), mindspore.int32) >>> sequence_length = Tensor(np.array([2, 2]), mindspore.int32) >>> ctc_loss = ops.CTCLoss() >>> loss, gradient = ctc_loss(inputs, labels_indices, labels_values, sequence_length) >>> print(loss) [ 0.7864997 0.720426 ] >>> print(gradient) [[[ 0.30898064 0.36491138 -0.673892 ] [ 0.33421117 0.2960548 -0.63026595 ]] [[ 0.23434742 0.36907154 0.11261538 ] [ 0.27316454 0.41090325 0.07584976 ]]]
-
class
tinyms.primitives.Cast(*args, **kwargs)[source]¶ Returns a tensor with the new specified data type.
- Inputs:
input_x (Union[Tensor, Number]) - The shape of tensor is \((x_1, x_2, ..., x_R)\). The tensor to be cast.
type (dtype.Number) - The valid data type of the output tensor. Only constant value is allowed.
- Outputs:
Tensor, the shape of tensor is the same as input_x, \((x_1, x_2, ..., x_R)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_np = np.random.randn(2, 3, 4, 5).astype(np.float32) >>> input_x = Tensor(input_np) >>> type_dst = mindspore.int32 >>> cast = ops.Cast() >>> output = cast(input_x, type_dst) >>> print(output.dtype) Int32 >>> print(output.shape) (2, 3, 4, 5)
-
class
tinyms.primitives.Ceil(*args, **kwargs)[source]¶ Rounds a tensor up to the closest integer element-wise.
\[out_i = \lceil x_i \rceil = \lfloor x_i \rfloor + 1\]- Inputs:
input_x (Tensor) - The input tensor. It’s element data type must be float16 or float32.
- Outputs:
Tensor, has the same shape as input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([1.1, 2.5, -1.5]), mindspore.float32) >>> ceil_op = ops.Ceil() >>> output = ceil_op(input_x) >>> print(output) [ 2. 3. -1.]
-
class
tinyms.primitives.CheckBprop(*args, **kwargs)[source]¶ Checks whether the data type and the shape of corresponding elements from tuples x and y are the same.
- Raises
TypeError – If tuples x and y are not the same.
- Inputs:
input_x (tuple[Tensor]) - The input_x contains the outputs of bprop to be checked.
input_y (tuple[Tensor]) - The input_y contains the inputs of bprop to check against.
- Outputs:
(tuple[Tensor]), the input_x, if data type and shape of corresponding elements from input_x and input_y are the same.
Examples
>>> input_x = (Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32),) >>> input_y = (Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32),) >>> out = ops.CheckBprop()(input_x, input_y)
-
class
tinyms.primitives.CheckValid(*args, **kwargs)[source]¶ Checks bounding box.
Checks whether the bounding box cross data and data border are valid.
- Inputs:
bboxes (Tensor) - Bounding boxes tensor with shape (N, 4). Data type must be float16 or float32.
img_metas (Tensor) - Raw image size information with the format of (height, width, ratio). Data type must be float16 or float32.
- Outputs:
Tensor, with shape of (N,) and dtype of bool.
- Supported Platforms:
AscendGPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.check_valid = ops.CheckValid() ... def construct(self, x, y): ... valid_result = self.check_valid(x, y) ... return valid_result ... >>> bboxes = Tensor(np.linspace(0, 6, 12).reshape(3, 4), mindspore.float32) >>> img_metas = Tensor(np.array([2, 1, 3]), mindspore.float32) >>> net = Net() >>> output = net(bboxes, img_metas) >>> print(output) [ True False False]
-
class
tinyms.primitives.ComputeAccidentalHits(*args, **kwargs)[source]¶ Compute accidental hits of sampled classes which happen to match target classes.
When a target class matches the sample class, we call it “accidental hit”. The result of calculating accidental hit contains three parts (index, id, weight), where index represents the row number in true_classes, and id represents the position in sampled_candidates, the weight is -FLOAT_MAX.
- Parameters
num_true (int) – The number of target classes per training example.
- Inputs:
true_classes (Tensor) - The target classes. With data type of int32 or int64 and shape [batch_size, num_true].
sampled_candidates (Tensor) - The sampled_candidates output of CandidateSampler, with data type of int32 or int64 and shape [num_sampled].
- Outputs:
Tuple of 3 Tensors.
indices (Tensor) - A Tensor with shape (num_accidental_hits,), with the same type as true_classes.
ids (Tensor) - A Tensor with shape (num_accidental_hits,), with the same type as true_classes.
weights (Tensor) - A Tensor with shape (num_accidental_hits,), with the type float32.
- Supported Platforms:
Ascend
Examples
>>> x = np.array([[1, 2], [0, 4], [3, 3]]) >>> y = np.array([0, 1, 2, 3, 4]) >>> sampler = ops.ComputeAccidentalHits(2) >>> output1, output2, output3 = sampler(Tensor(x), Tensor(y)) >>> print(output1, output2, output3) [0 0 1 1 2 2] [1 2 0 4 3 3] [-3.4028235e+38 -3.4028235e+38 -3.4028235e+38 -3.4028235e+38 -3.4028235e+38 -3.4028235e+38]
-
class
tinyms.primitives.Concat(*args, **kwargs)[source]¶ Connect tensor in the specified axis.
Connect input tensors along with the given axis.
The input data is a tuple of tensors. These tensors have the same rank R. Set the given axis as m, and \(0 \le m < R\). Set the number of input tensors as N. For the \(i\)-th tensor \(t_i\), it has the shape of \((x_1, x_2, ..., x_{mi}, ..., x_R)\). \(x_{mi}\) is the \(m\)-th dimension of the \(i\)-th tensor. Then, the shape of the output tensor is
\[(x_1, x_2, ..., \sum_{i=1}^Nx_{mi}, ..., x_R)\]- Parameters
axis (int) – The specified axis. Default: 0.
- Inputs:
input_x (tuple, list) - A tuple or a list of input tensors.
- Outputs:
Tensor, the shape is \((x_1, x_2, ..., \sum_{i=1}^Nx_{mi}, ..., x_R)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> data1 = Tensor(np.array([[0, 1], [2, 1]]).astype(np.float32)) >>> data2 = Tensor(np.array([[0, 1], [2, 1]]).astype(np.float32)) >>> op = ops.Concat() >>> output = op((data1, data2)) >>> print(output) [[0. 1.] [2. 1.] [0. 1.] [2. 1.]]
-
class
tinyms.primitives.ConfusionMatrix(*args, **kwargs)[source]¶ Calculates the confusion matrix from labels and predictions.
- Parameters
- Inputs:
labels (Tensor) - real labels, tensor of 1-D. the dtype must be non-negative Integer.
predictions (Tensor) - the labels from prediction, tensor of 1-D. the shape same as labels and the dtype must be non-negative Integer.
weights (Tensor) - tensor of 1-D. the shape same as predictions.
- Outputs:
Tensor, the confusion matrix, with shape (num_classes, num_classes).
Examples
>>> confusion_matrix = ops.ConfusionMatrix(4) >>> labels = Tensor([0, 1, 1, 3], mindspore.int32) >>> predictions = Tensor([1, 2, 1, 3], mindspore.int32) >>> output = confusion_matrix(labels, predictions) >>> print(output) [[0 1 0 0] [0 1 1 0] [0 0 0 0] [0 0 0 1]]
-
class
tinyms.primitives.ControlDepend(**kwargs)[source]¶ Adds control dependency relation between source and destination operations.
In many cases, we need to control the execution order of operations. ControlDepend is designed for this. ControlDepend will instruct the execution engine to run the operations in a specific order. ControlDepend tells the engine that the destination operations must depend on the source operation which means the source operations must be executed before the destination.
Note
This operation does not work in PYNATIVE_MODE. ControlDepend is deprecated from version 1.1 and will be removed in a future version, use Depend instead.
- Parameters
depend_mode (int) – Use 0 for a normal dependency relation and 1 for a user-defined dependency relation. Default: 0.
- Inputs:
src (Any) - The source input. It can be a tuple of operations output or a single operation output. We do not concern about the input data, but concern about the operation that generates the input data. If depend_mode is 1 and the source input is Parameter, we will try to find the operations that used the parameter as input.
dst (Any) - The destination input. It can be a tuple of operations output or a single operation output. We do not concern about the input data, but concern about the operation that generates the input data. If depend_mode is 1 and the source input is Parameter, we will try to find the operations that used the parameter as input.
- Outputs:
This operation has no actual data output, it will be used to setup the order of relative operations.
- Supported Platforms:
AscendGPUCPU
Examples
>>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.control_depend = P.ControlDepend() ... self.softmax = ops.Softmax() ... ... def construct(self, x, y): ... mul = x * y ... softmax = self.softmax(x) ... ret = self.control_depend(mul, softmax) ... return ret ... >>> x = Tensor(np.ones([4, 5]), dtype=mindspore.float32) >>> y = Tensor(np.ones([4, 5]), dtype=mindspore.float32) >>> net = Net() >>> output = net(x, y) >>> print(output) [[1. 1. 1. 1. 1.] [1. 1. 1. 1. 1.] [1. 1. 1. 1. 1.] [1. 1. 1. 1. 1.]]
-
class
tinyms.primitives.Conv2D(*args, **kwargs)[source]¶ 2D convolution layer.
Applies a 2D convolution over an input tensor which is typically of shape \((N, C_{in}, H_{in}, W_{in})\), where \(N\) is batch size and \(C_{in}\) is channel number. For each batch of shape \((C_{in}, H_{in}, W_{in})\), the formula is defined as:
\[out_j = \sum_{i=0}^{C_{in} - 1} ccor(W_{ij}, X_i) + b_j,\]where \(ccor\) is the cross correlation operator, \(C_{in}\) is the input channel number, \(j\) ranges from \(0\) to \(C_{out} - 1\), \(W_{ij}\) corresponds to the \(i\)-th channel of the \(j\)-th filter and \(out_{j}\) corresponds to the \(j\)-th channel of the output. \(W_{ij}\) is a slice of kernel and it has shape \((\text{ks_h}, \text{ks_w})\), where \(\text{ks_h}\) and \(\text{ks_w}\) are the height and width of the convolution kernel. The full kernel has shape \((C_{out}, C_{in} // \text{group}, \text{ks_h}, \text{ks_w})\), where group is the group number to split the input in the channel dimension.
If the ‘pad_mode’ is set to be “valid”, the output height and width will be \(\left \lfloor{1 + \frac{H_{in} + 2 \times \text{padding} - \text{ks_h} - (\text{ks_h} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) and \(\left \lfloor{1 + \frac{W_{in} + 2 \times \text{padding} - \text{ks_w} - (\text{ks_w} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) respectively.
The first introduction can be found in paper Gradient Based Learning Applied to Document Recognition. More detailed introduction can be found here: http://cs231n.github.io/convolutional-networks/.
- Parameters
out_channel (int) – The dimension of the output.
kernel_size (Union[int, tuple[int]]) – The kernel size of the 2D convolution.
mode (int) – Modes for different convolutions. 0 Math convolutiuon, 1 cross-correlation convolution , 2 deconvolution, 3 depthwise convolution. Default: 1.
pad_mode (str) – Modes to fill padding. It could be “valid”, “same”, or “pad”. Default: “valid”.
pad (Union(int, tuple[int])) – The pad value to be filled. Default: 0. If pad is an integer, the paddings of top, bottom, left and right are the same, equal to pad. If pad is a tuple of four integers, the padding of top, bottom, left and right equal to pad[0], pad[1], pad[2], and pad[3] correspondingly.
stride (Union(int, tuple[int])) – The stride to be applied to the convolution filter. Default: 1.
dilation (Union(int, tuple[int])) – Specifies the space to use between kernel elements. Default: 1.
group (int) – Splits input into groups. Default: 1.
data_format (str) – The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: “NCHW”.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
weight (Tensor) - Set size of kernel is \((K_1, K_2)\), then the shape is \((C_{out}, C_{in}, K_1, K_2)\).
- Outputs:
Tensor, the value that applied 2D convolution. The shape is \((N, C_{out}, H_{out}, W_{out})\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input = Tensor(np.ones([10, 32, 32, 32]), mindspore.float32) >>> weight = Tensor(np.ones([32, 32, 3, 3]), mindspore.float32) >>> conv2d = ops.Conv2D(out_channel=32, kernel_size=3) >>> output = conv2d(input, weight) >>> print(output.shape) (10, 32, 30, 30)
-
class
tinyms.primitives.Conv2DBackpropInput(*args, **kwargs)[source]¶ Computes the gradients of convolution with respect to the input.
- Parameters
out_channel (int) – The dimensionality of the output space.
kernel_size (Union[int, tuple[int]]) – The size of the convolution window.
pad_mode (str) – Modes to fill padding. It could be “valid”, “same”, or “pad”. Default: “valid”.
pad (Union[int, tuple[int]]) – The pad value to be filled. Default: 0. If pad is an integer, the paddings of top, bottom, left and right are the same, equal to pad. If pad is a tuple of four integers, the padding of top, bottom, left and right equal to pad[0], pad[1], pad[2], and pad[3] correspondingly.
mode (int) – Modes for different convolutions. 0 Math convolutiuon, 1 cross-correlation convolution , 2 deconvolution, 3 depthwise convolution. Default: 1.
stride (Union[int. tuple[int]]) – The stride to be applied to the convolution filter. Default: 1.
dilation (Union[int. tuple[int]]) – Specifies the dilation rate to be used for the dilated convolution. Default: 1.
group (int) – Splits input into groups. Default: 1.
data_format (str) –
- Inputs:
dout (Tensor) - the gradients w.r.t the output of the convolution. The shape conforms to the default data_format \((N, C_{out}, H_{out}, W_{out})\).
weight (Tensor) - Set size of kernel is \((K_1, K_2)\), then the shape is \((C_{out}, C_{in}, K_1, K_2)\).
input_size (Tensor) - A tuple describes the shape of the input which conforms to the format \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor, the gradients w.r.t the input of convolution. It has the same shape as the input.
- Supported Platforms:
AscendGPU
Examples
>>> dout = Tensor(np.ones([10, 32, 30, 30]), mindspore.float32) >>> weight = Tensor(np.ones([32, 32, 3, 3]), mindspore.float32) >>> x = Tensor(np.ones([10, 32, 32, 32])) >>> conv2d_backprop_input = ops.Conv2DBackpropInput(out_channel=32, kernel_size=3) >>> output = conv2d_backprop_input(dout, weight, F.shape(x)) >>> print(output.shape) (10, 32, 32, 32)
-
class
tinyms.primitives.Cos(*args, **kwargs)[source]¶ Computes cosine of input element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Supported Platforms:
AscendGPU
Examples
>>> cos = ops.Cos() >>> input_x = Tensor(np.array([0.24, 0.83, 0.31, 0.09]), mindspore.float32) >>> output = cos(input_x) >>> print(output) [0.971338 0.67487574 0.95233357 0.9959527 ]
-
class
tinyms.primitives.Cosh(*args, **kwargs)[source]¶ Computes hyperbolic cosine of input element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Supported Platforms:
Ascend
Examples
>>> cosh = ops.Cosh() >>> input_x = Tensor(np.array([0.24, 0.83, 0.31, 0.09]), mindspore.float32) >>> output = cosh(input_x) >>> print(output) [1.0289385 1.364684 1.048436 1.0040528]
-
class
tinyms.primitives.CropAndResize(*args, **kwargs)[source]¶ Extracts crops from the input image tensor and resizes them.
Note
In case that the output shape depends on crop_size, the crop_size must be constant.
- Parameters
method (str) – An optional string that specifies the sampling method for resizing. It can be “bilinear”, “nearest” or “bilinear_v2”. The option “bilinear” stands for standard bilinear interpolation algorithm, while “bilinear_v2” may result in better result in some cases. Default: “bilinear”
extrapolation_value (float) – An optional float value used extrapolation, if applicable. Default: 0.
- Inputs:
x (Tensor) - The input image must be a 4-D tensor of shape [batch, image_height, image_width, depth]. Types allowed: int8, int16, int32, int64, float16, float32, float64, uint8, uint16.
boxes (Tensor) - A 2-D tensor of shape [num_boxes, 4]. The i-th row of the tensor specifies the coordinates of a box in the box_ind[i] image and is specified in normalized coordinates [y1, x1, y2, x2]. A normalized coordinate value of y is mapped to the image coordinate at y * (image_height - 1), so as the [0, 1] interval of normalized image height is mapped to [0, image_height - 1] in image height coordinates. We do allow y1 > y2, in which case the sampled crop is an up-down flipped version of the original image. The width dimension is treated similarly. Normalized coordinates outside the [0, 1] range are allowed, in which case we use extrapolation_value to extrapolate the input image values. Types allowed: float32.
box_index (Tensor) - A 1-D tensor of shape [num_boxes] with int32 values in [0, batch). The value of box_ind[i] specifies the image that the i-th box refers to. Types allowed: int32.
crop_size (Tuple[int]) - A tuple of two int32 elements: (crop_height, crop_width). Only constant value is allowed. All cropped image patches are resized to this size. The aspect ratio of the image content is not preserved. Both crop_height and crop_width need to be positive.
- Outputs:
A 4-D tensor of shape [num_boxes, crop_height, crop_width, depth] with type: float32.
- Supported Platforms:
Ascend
Examples
>>> class CropAndResizeNet(nn.Cell): ... def __init__(self, crop_size): ... super(CropAndResizeNet, self).__init__() ... self.crop_and_resize = ops.CropAndResize() ... self.crop_size = crop_size ... ... def construct(self, x, boxes, box_index): ... return self.crop_and_resize(x, boxes, box_index, self.crop_size) ... >>> BATCH_SIZE = 1 >>> NUM_BOXES = 5 >>> IMAGE_HEIGHT = 256 >>> IMAGE_WIDTH = 256 >>> CHANNELS = 3 >>> image = np.random.normal(size=[BATCH_SIZE, IMAGE_HEIGHT, IMAGE_WIDTH, CHANNELS]).astype(np.float32) >>> boxes = np.random.uniform(size=[NUM_BOXES, 4]).astype(np.float32) >>> box_index = np.random.uniform(size=[NUM_BOXES], low=0, high=BATCH_SIZE).astype(np.int32) >>> crop_size = (24, 24) >>> crop_and_resize = CropAndResizeNet(crop_size=crop_size) >>> output = crop_and_resize(Tensor(image), Tensor(boxes), Tensor(box_index)) >>> print(output.shape) (5, 24, 24, 3)
-
class
tinyms.primitives.CumProd(*args, **kwargs)[source]¶ Computes the cumulative product of the tensor x along axis.
- Parameters
- Inputs:
input_x (Tensor[Number]) - The input tensor.
axis (int) - The dimensions to compute the cumulative product. Only constant value is allowed.
- Outputs:
Tensor, has the same shape and dtype as the input_x.
- Supported Platforms:
Ascend
Examples
>>> a, b, c, = 1, 2, 3 >>> input_x = Tensor(np.array([a, b, c]).astype(np.float32)) >>> op0 = ops.CumProd() >>> output0 = op0(input_x, 0) # output=[a, a * b, a * b * c] >>> op1 = ops.CumProd(exclusive=True) >>> output1 = op1(input_x, 0) # output=[1, a, a * b] >>> op2 = ops.CumProd(reverse=True) >>> output2 = op2(input_x, 0) # output=[a * b * c, b * c, c] >>> op3 = ops.CumProd(exclusive=True, reverse=True) >>> output3 = op3(input_x, 0) # output=[b * c, c, 1] >>> print(output0) [1. 2. 6.] >>> print(output1) [1. 1. 2.] >>> print(output2) [6. 6. 3.] >>> print(output3) [6. 3. 1.]
-
class
tinyms.primitives.CumSum(*args, **kwargs)[source]¶ Computes the cumulative sum of input tensor along axis.
\[y_i = x_1 + x_2 + x_3 + ... + x_i\]- Parameters
- Inputs:
input (Tensor) - The input tensor to accumulate.
axis (int) - The axis to accumulate the tensor’s value. Only constant value is allowed. Must be in the range [-rank(input), rank(input)).
- Outputs:
Tensor, the shape of the output tensor is consistent with the input tensor’s.
- Supported Platforms:
AscendGPU
Examples
>>> input = Tensor(np.array([[3, 4, 6, 10], [1, 6, 7, 9], [4, 3, 8, 7], [1, 3, 7, 9]]).astype(np.float32)) >>> cumsum = ops.CumSum() >>> output = cumsum(input, 1) >>> print(output) [[ 3. 7. 13. 23.] [ 1. 7. 14. 23.] [ 4. 7. 15. 22.] [ 1. 4. 11. 20.]]
-
class
tinyms.primitives.DType(*args, **kwargs)[source]¶ Returns the data type of the input tensor as mindspore.dtype.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
mindspore.dtype, the data type of a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) >>> output = ops.DType()(input_tensor) >>> print(output) Float32
-
class
tinyms.primitives.DataFormatDimMap(*args, **kwargs)[source]¶ Returns the dimension index in the destination data format given in the source data format.
- Parameters
src_format (string) – An optional value for source data format. Default: ‘NHWC’.
dst_format (string) – An optional value for destination data format. Default: ‘NCHW’.
- Inputs:
input_x (Tensor) - A Tensor with each element as a dimension index in source data format. The suggested values is in the range [-4, 4). It’s type is int32.
- Outputs:
Tensor, has the same type as the input_x.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor([0, 1, 2, 3], mindspore.int32) >>> dfdm = ops.DataFormatDimMap() >>> output = dfdm(x) >>> print(output) [0 3 1 2]
-
class
tinyms.primitives.Depend(*args, **kwargs)[source]¶ Depend is used for processing dependency operations.
In some side-effect scenarios, we need to ensure the execution order of operators. In order to ensure that operator A is executed before operator B, it is recommended to insert the Depend operator between operators A and B.
Previously, the ControlDepend operator was used to control the execution order. Since the ControlDepend operator is deprecated from version 1.1, it is recommended to use the Depend operator instead. The replacement method is as follows:
a = A(x) ---> a = A(x) b = B(y) ---> y = Depend(y, a) ControlDepend(a, b) ---> b = B(y)
- Inputs:
value (Tensor) - the real value to return for depend operator.
expr (Expression) - the expression to execute with no outputs.
- Outputs:
Tensor, the value passed by last operator.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import numpy as np >>> import mindspore >>> import mindspore.nn as nn >>> import mindspore.ops.operations as P >>> from mindspore import Tensor >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.softmax = P.Softmax() ... self.depend = P.Depend() ... ... def construct(self, x, y): ... mul = x * y ... y = self.depend(y, mul) ... ret = self.softmax(y) ... return ret ... >>> x = Tensor(np.ones([4, 5]), dtype=mindspore.float32) >>> y = Tensor(np.ones([4, 5]), dtype=mindspore.float32) >>> net = Net() >>> output = net(x, y) >>> print(output) [[0.2 0.2 0.2 0.2 0.2] [0.2 0.2 0.2 0.2 0.2] [0.2 0.2 0.2 0.2 0.2] [0.2 0.2 0.2 0.2 0.2]]
-
class
tinyms.primitives.DepthToSpace(*args, **kwargs)[source]¶ Rearranges blocks of depth data into spatial dimensions.
This is the reverse operation of SpaceToDepth.
The depth of output tensor is \(input\_depth / (block\_size * block\_size)\).
The output tensor’s height dimension is \(height * block\_size\).
The output tensor’s weight dimension is \(weight * block\_size\).
The input tensor’s depth must be divisible by block_size * block_size. The data format is “NCHW”.
- Parameters
block_size (int) – The block size used to divide depth data. It must be >= 2.
- Inputs:
x (Tensor) - The target tensor. It must be a 4-D tensor with shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{in} / \text{block_size}, H_{in} * \text{block_size}, W_{in} * \text{block_size})\).
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.random.rand(1, 12, 1, 1), mindspore.float32) >>> block_size = 2 >>> depth_to_space = ops.DepthToSpace(block_size) >>> output = depth_to_space(x) >>> print(output.shape) (1, 3, 2, 2)
-
class
tinyms.primitives.DepthwiseConv2dNative(*args, **kwargs)[source]¶ Returns the depth-wise convolution value for the input.
Applies depthwise conv2d for the input, which will generate more channels with channel_multiplier. Given an input tensor of shape \((N, C_{in}, H_{in}, W_{in})\) where \(N\) is the batch size and a filter tensor with kernel size \((ks_{h}, ks_{w})\), containing \(C_{in} * \text{channel_multiplier}\) convolutional filters of depth 1; it applies different filters to each input channel (channel_multiplier channels for each input channel has the default value 1), then concatenates the results together. The output has \(\text{in_channels} * \text{channel_multiplier}\) channels.
- Parameters
channel_multiplier (int) – The multipiler for the original output convolution. Its value must be greater than 0.
kernel_size (Union[int, tuple[int]]) – The size of the convolution kernel.
mode (int) – Modes for different convolutions. 0 Math convolution, 1 cross-correlation convolution , 2 deconvolution, 3 depthwise convolution. Default: 3.
pad_mode (str) – Modes to fill padding. It could be “valid”, “same”, or “pad”. Default: “valid”.
pad (Union[int, tuple[int]]) – The pad value to be filled. If pad is an integer, the paddings of top, bottom, left and right are the same, equal to pad. If pad is a tuple of four integers, the padding of top, bottom, left and right equal to pad[0], pad[1], pad[2], and pad[3] correspondingly. Default: 0.
stride (Union[int, tuple[int]]) – The stride to be applied to the convolution filter. Default: 1.
dilation (Union[int, tuple[int]]) – Specifies the dilation rate to be used for the dilated convolution. Default: 1.
group (int) – Splits input into groups. Default: 1.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
weight (Tensor) - Set the size of kernel as \((K_1, K_2)\), then the shape is \((K, C_{in}, K_1, K_2)\), K must be 1.
- Outputs:
Tensor of shape \((N, C_{in} * \text{channel_multiplier}, H_{out}, W_{out})\).
- Supported Platforms:
Ascend
Examples
>>> input = Tensor(np.ones([10, 32, 32, 32]), mindspore.float32) >>> weight = Tensor(np.ones([1, 32, 3, 3]), mindspore.float32) >>> depthwise_conv2d = ops.DepthwiseConv2dNative(channel_multiplier=3, kernel_size=(3, 3)) >>> output = depthwise_conv2d(input, weight) >>> print(output.shape) (10, 96, 30, 30)
-
class
tinyms.primitives.Diag(*args, **kwargs)[source]¶ Constructs a diagonal tensor with a given diagonal values.
Assume input_x has dimensions \([D_1,... D_k]\), the output is a tensor of rank 2k with dimensions \([D_1,..., D_k, D_1,..., D_k]\) where: \(output[i_1,..., i_k, i_1,..., i_k] = input_x[i_1,..., i_k]\) and 0 everywhere else.
- Inputs:
input_x (Tensor) - The input tensor. The input shape must be less than 5d.
- Outputs:
Tensor, has the same dtype as the input_x.
Examples
>>> input_x = Tensor([1, 2, 3, 4]) >>> diag = ops.Diag() >>> output = diag(input_x) >>> print(output) [[1, 0, 0, 0], [0, 2, 0, 0], [0, 0, 3, 0], [0, 0, 0, 4]]
-
class
tinyms.primitives.DiagPart(*args, **kwargs)[source]¶ Extracts the diagonal part from given tensor.
Assume input has dimensions \([D_1,..., D_k, D_1,..., D_k]\), the output is a tensor of rank k with dimensions \([D_1,..., D_k]\) where: \(output[i_1,..., i_k] = input[i_1,..., i_k, i_1,..., i_k]\).
- Inputs:
input_x (Tensor) - tensor of rank k where k is even and not zero.
- Outputs:
Tensor, the extracted diagonal has the same dtype as the input_x.
- Examples
>>> input_x = Tensor([[1, 0, 0, 0], ... [0, 2, 0, 0], ... [0, 0, 3, 0], ... [0, 0, 0, 4]]) >>> diag_part = ops.DiagPart() >>> output = diag_part(input_x) >>> print(output) [1 2 3 4]
-
class
tinyms.primitives.Div(*args, **kwargs)[source]¶ Computes the quotient of dividing the first input tensor by the second input tensor element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
\[out_{i} = \frac{x_i}{y_i}\]- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - When the first input is a tensor, The second input could be a number, a bool, or a tensor whose data type is number or bool. When the first input is a number or a bool, the second input must be a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([-4.0, 5.0, 6.0]), mindspore.float32) >>> input_y = Tensor(np.array([3.0, 2.0, 3.0]), mindspore.float32) >>> div = ops.Div() >>> output = div(input_x, input_y) >>> print(output) [-1.3333334 2.5 2. ]
-
class
tinyms.primitives.DivNoNan(*args, **kwargs)[source]¶ Computes a safe divide and returns 0 if the y is zero.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([-1.0, 0., 1.0, 5.0, 6.0]), mindspore.float32) >>> input_y = Tensor(np.array([0., 0., 0., 2.0, 3.0]), mindspore.float32) >>> div_no_nan = ops.DivNoNan() >>> output = div_no_nan(input_x, input_y) >>> print(output) [0. 0. 0. 2.5 2. ]
-
class
tinyms.primitives.Dropout(*args, **kwargs)[source]¶ During training, randomly zeroes some of the elements of the input tensor with probability.
- Parameters
keep_prob (float) – The keep rate, between 0 and 1, e.g. keep_prob = 0.9, means dropping out 10% of input units.
- Inputs:
input (Tensor) - The input tensor.
- Outputs:
output (Tensor) - with the same shape as the input tensor.
mask (Tensor) - with the same shape as the input tensor.
Examples
>>> dropout = ops.Dropout(keep_prob=0.5) >>> x = Tensor((20, 16, 50, 50), mindspore.float32) >>> output, mask = dropout(x) >>> print(output) [0. 32. 0. 0.] >>> print(mask) [0. 1. 0. 0.]
-
class
tinyms.primitives.DropoutDoMask(*args, **kwargs)[source]¶ Applies dropout mask on the input tensor.
Take the mask output of DropoutGenMask as input, and apply dropout on the input.
- Inputs:
input_x (Tensor) - The input tensor.
mask (Tensor) - The mask to be applied on input_x, which is the output of DropoutGenMask. And the shape of input_x must be the same as the value of DropoutGenMask’s input shape. If input wrong mask, the output of DropoutDoMask are unpredictable.
keep_prob (Union[Tensor, float]) - The keep rate, greater than 0 and less equal than 1, e.g. keep_prob = 0.9, means dropping out 10% of input units. The value of keep_prob is the same as the input keep_prob of DropoutGenMask.
- Outputs:
Tensor, the value that applied dropout on.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.ones([2, 2, 3]), mindspore.float32) >>> shape = (2, 2, 3) >>> keep_prob = Tensor(0.5, mindspore.float32) >>> dropout_gen_mask = ops.DropoutGenMask() >>> dropout_do_mask = ops.DropoutDoMask() >>> mask = dropout_gen_mask(shape, keep_prob) >>> output = dropout_do_mask(x, mask, keep_prob) >>> print(output.shape) (2, 2, 3)
-
class
tinyms.primitives.DropoutGenMask(*args, **kwargs)[source]¶ Generates the mask value for the input shape.
- Parameters
- Inputs:
shape (tuple[int]) - The shape of target mask.
keep_prob (Tensor) - The keep rate, greater than 0 and less equal than 1, e.g. keep_prob = 0.9, means dropping out 10% of input units.
- Outputs:
Tensor, the value of generated mask for input shape.
- Supported Platforms:
Ascend
Examples
>>> dropout_gen_mask = ops.DropoutGenMask() >>> shape = (2, 4, 5) >>> keep_prob = Tensor(0.5, mindspore.float32) >>> output = dropout_gen_mask(shape, keep_prob) >>> print(output.shape) (16,)
-
class
tinyms.primitives.DynamicGRUV2(*args, **kwargs)[source]¶ Applies a single-layer gated recurrent unit (GRU) to an input sequence.
\[\begin{split}\begin{array}{ll} r_t = \sigma(W_{ir} x_t + b_{ir} + W_{hr} h_{(t-1)} + b_{hr}) \\ z_t = \sigma(W_{iz} x_t + b_{iz} + W_{hz} h_{(t-1)} + b_{hz}) \\ n_t = \tanh(W_{in} x_t + b_{in} + r_t * (W_{hn} h_{(t-1)}+ b_{hn})) \\ h_t = (1 - z_t) * n_t + z_t * h_{(t-1)} \end{array}\end{split}\]where \(h_t\) is the hidden state at time t, \(x_t\) is the input at time t, \(h_{(t-1)}\) is the hidden state of the layer at time t-1 or the initial hidden state at time 0, and \(r_t\), \(z_t\), \(n_t\) are the reset, update, and new gates, respectively. \(\sigma\) is the sigmoid function, and \(*\) is the Hadamard product.
- Parameters
direction (str) – A string identifying the direction in the op. Default: ‘UNIDIRECTIONAL’. Only ‘UNIDIRECTIONAL’ is currently supported.
cell_depth (int) – An integer identifying the cell depth in the op. Default: 1.
keep_prob (float) – A float identifying the keep prob in the op. Default: 1.0.
cell_clip (float) – A float identifying the cell clip in the op. Default: -1.0.
num_proj (int) – An integer identifying the num proj in the op. Default: 0.
time_major (bool) – A bool identifying the time major in the op. Default: True.
activation (str) – A string identifying the type of activation function in the op. Default: ‘tanh’. Only ‘tanh’ is currently supported.
gate_order (str) – A string identifying the gate order in weight and bias. Default: ‘rzh. ‘zrh’ is another option.
reset_after (bool) – A bool identifying whether to apply reset gate after matrix multiplication. Default: True.
is_training (bool) – A bool identifying is training in the op. Default: True.
- Inputs:
x (Tensor) - Current words. Tensor of shape \((\text{num_step}, \text{batch_size}, \text{input_size})\). The data type must be float16.
weight_input (Tensor) - Input-hidden weight. Tensor of shape \((\text{input_size}, 3 \times \text{hidden_size})\). The data type must be float16.
weight_hidden (Tensor) - Hidden-hidden weight. Tensor of shape \((\text{hidden_size}, 3 \times \text{hidden_size})\). The data type must be float16.
bias_input (Tensor) - Input-hidden bias. Tensor of shape \((3 \times \text{hidden_size})\), or None. Has the same data type with input init_h.
bias_hidden (Tensor) - Hidden-hidden bias. Tensor of shape \((3 \times \text{hidden_size})\), or None. Has the same data type with input init_h.
seq_length (Tensor) - The length of each batch. Tensor of shape \((\text{batch_size})\). Only None is currently supported.
init_h (Tensor) - Hidden state of initial time. Tensor of shape \((\text{batch_size}, \text{hidden_size})\). The data type must be float16 or float32.
- Outputs:
y (Tensor) - A Tensor of shape :math: if num_proj > 0 (num_step, batch_size, min(hidden_size, num_proj), if num_proj == 0 (num_step, batch_size, hidden_size). Has the same data type with input bias_type.
output_h (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
update (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
reset (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
new (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
hidden_new (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
A note about the bias_type:
If bias_input and bias_hidden both are None, bias_type is date type of init_h.
If bias_input is not None, bias_type is the date type of bias_input.
If bias_input is None and bias_hidden is not None, `bias_type is the date type of bias_hidden.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.random.rand(2, 8, 64).astype(np.float16)) >>> weight_i = Tensor(np.random.rand(64, 48).astype(np.float16)) >>> weight_h = Tensor(np.random.rand(16, 48).astype(np.float16)) >>> bias_i = Tensor(np.random.rand(48).astype(np.float16)) >>> bias_h = Tensor(np.random.rand(48).astype(np.float16)) >>> init_h = Tensor(np.random.rand(8, 16).astype(np.float16)) >>> dynamic_gru_v2 = ops.DynamicGRUV2() >>> output = dynamic_gru_v2(x, weight_i, weight_h, bias_i, bias_h, None, init_h) >>> print(output[0].shape) (2, 8, 16)
-
class
tinyms.primitives.DynamicRNN(*args, **kwargs)[source]¶ DynamicRNN Operator.
- Parameters
cell_type (str) – A string identifying the cell type in the op. Default: ‘LSTM’. Only ‘LSTM’ is currently supported.
direction (str) – A string identifying the direction in the op. Default: ‘UNIDIRECTIONAL’. Only ‘UNIDIRECTIONAL’ is currently supported.
cell_depth (int) – An integer identifying the cell depth in the op. Default: 1.
use_peephole (bool) – A bool identifying if use peephole in the op. Default: False.
keep_prob (float) – A float identifying the keep prob in the op. Default: 1.0.
cell_clip (float) – A float identifying the cell clip in the op. Default: -1.0.
num_proj (int) – An integer identifying the num proj in the op. Default: 0.
time_major (bool) – A bool identifying the time major in the op. Default: True. Only True is currently supported.
activation (str) – A string identifying the type of activation function in the op. Default: ‘tanh’. Only ‘tanh’ is currently supported.
forget_bias (float) – A float identifying the forget bias in the op. Default: 0.0.
is_training (bool) – A bool identifying is training in the op. Default: True.
- Inputs:
x (Tensor) - Current words. Tensor of shape (num_step, batch_size, input_size). The data type must be float16.
w (Tensor) - Weight. Tensor of shape (input_size + hidden_size, 4 x hidden_size). The data type must be float16.
b (Tensor) - Bias. Tensor of shape (4 x hidden_size). The data type must be float16 or float32.
seq_length (Tensor) - The length of each batch. Tensor of shape (batch_size). Only None is currently supported.
init_h (Tensor) - Hidden state of initial time. Tensor of shape (1, batch_size, hidden_size). The data type must be float16.
init_c (Tensor) - Cell state of initial time. Tensor of shape (1, batch_size, hidden_size). The data type must be float16.
- Outputs:
y (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
output_h (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). With data type of float16.
output_c (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
i (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
j (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
f (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
o (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
tanhct (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.random.rand(2, 16, 64).astype(np.float16)) >>> w = Tensor(np.random.rand(96, 128).astype(np.float16)) >>> b = Tensor(np.random.rand(128).astype(np.float16)) >>> init_h = Tensor(np.random.rand(1, 16, 32).astype(np.float16)) >>> init_c = Tensor(np.random.rand(1, 16, 32).astype(np.float16)) >>> dynamic_rnn = ops.DynamicRNN() >>> output = dynamic_rnn(x, w, b, None, init_h, init_c) >>> print(output[0].shape) (2, 16, 32)
-
class
tinyms.primitives.DynamicShape(*args, **kwargs)[source]¶ Returns the shape of the input tensor.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor[int], 1-dim Tensor of type int32
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.ones(shape=[3, 2, 1]), mindspore.float32) >>> shape = ops.DynamicShape() >>> output = shape(input_tensor) >>> print(output) [3 2 1]
-
class
tinyms.primitives.EditDistance(*args, **kwargs)[source]¶ Computes the Levenshtein Edit Distance. It is used to measure the similarity of two sequences. The inputs are variable-length sequences provided by SparseTensors (hypothesis_indices, hypothesis_values, hypothesis_shape) and (truth_indices, truth_values, truth_shape).
- Parameters
normalize (bool) – If true, edit distances are normalized by length of truth. Default: True.
- Inputs:
hypothesis_indices (Tensor) - The indices of the hypothesis list SparseTensor. With int64 data type. The shape of tensor is \((N, R)\).
hypothesis_values (Tensor) - The values of the hypothesis list SparseTensor. Must be 1-D vector with length of N.
hypothesis_shape (Tensor) - The shape of the hypothesis list SparseTensor. Must be R-length vector with int64 data type. Only constant value is allowed.
truth_indices (Tensor) - The indices of the truth list SparseTensor. With int64 data type. The shape of tensor is \((M, R)\).
truth_values (Tensor) - The values of the truth list SparseTensor. Must be 1-D vector with length of M.
truth_shape (Tensor) - The shape of the truth list SparseTensor. Must be R-length vector with int64 data type. Only constant value is allowed.
- Outputs:
Tensor, a dense tensor with rank R-1 and float32 data type.
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> from mindspore import context >>> from mindspore import Tensor >>> import mindspore.nn as nn >>> import mindspore.ops.operations as ops >>> context.set_context(mode=context.GRAPH_MODE) >>> class EditDistance(nn.Cell): ... def __init__(self, hypothesis_shape, truth_shape, normalize=True): ... super(EditDistance, self).__init__() ... self.edit_distance = P.EditDistance(normalize) ... self.hypothesis_shape = hypothesis_shape ... self.truth_shape = truth_shape ... ... def construct(self, hypothesis_indices, hypothesis_values, truth_indices, truth_values): ... return self.edit_distance(hypothesis_indices, hypothesis_values, self.hypothesis_shape, ... truth_indices, truth_values, self.truth_shape) ... >>> hypothesis_indices = Tensor(np.array([[0, 0, 0], [1, 0, 1], [1, 1, 1]]).astype(np.int64)) >>> hypothesis_values = Tensor(np.array([1, 2, 3]).astype(np.float32)) >>> hypothesis_shape = Tensor(np.array([1, 1, 2]).astype(np.int64)) >>> truth_indices = Tensor(np.array([[0, 1, 0], [0, 0, 1], [1, 1, 0], [1, 0, 1]]).astype(np.int64)) >>> truth_values = Tensor(np.array([1, 3, 2, 1]).astype(np.float32)) >>> truth_shape = Tensor(np.array([2, 2, 2]).astype(np.int64)) >>> edit_distance = EditDistance(hypothesis_shape, truth_shape) >>> output = edit_distance(hypothesis_indices, hypothesis_values, truth_indices, truth_values) >>> print(output) [[1. 1.] [1. 1.]]
-
class
tinyms.primitives.Elu(*args, **kwargs)[source]¶ Computes exponential linear:
\[\begin{split}\text{ELU}(x)= \left\{ \begin{array}{align} \alpha(e^{x} - 1) & \text{if } x \le 0\\ x & \text{if } x \gt 0\\ \end{array}\right.\end{split}\]The data type of input tensor must be float.
- Parameters
alpha (float) – The coefficient of negative factor whose type is float, only support ‘1.0’ currently. Default: 1.0.
- Inputs:
input_x (Tensor) - The input tensor whose data type must be float.
- Outputs:
Tensor, has the same shape and data type as input_x.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> elu = ops.Elu() >>> output = elu(input_x) >>> print(output) [[-0.63212055 4. -0.99966455] [ 2. -0.99326205 9. ]]
-
class
tinyms.primitives.EmbeddingLookup(*args, **kwargs)[source]¶ Returns a slice of input tensor based on the specified indices.
This Primitive has the similar functionality as GatherV2 operating on axis = 0, but has one more inputs: offset.
- Inputs:
input_params (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). This represents a Tensor slice, instead of the entire Tensor. Currently, the dimension is restricted to be 2.
input_indices (Tensor) - The shape of tensor is \((y_1, y_2, ..., y_S)\). Specifies the indices of elements of the original Tensor. Values can be out of range of input_params, and the exceeding part will be filled with 0 in the output. Values does not support negative and the result is undefined if values are negative.
offset (int) - Specifies the offset value of this input_params slice. Thus the real indices are equal to input_indices minus offset.
- Outputs:
Tensor, the shape of tensor is \((z_1, z_2, ..., z_N)\).
- Supported Platforms:
AscendCPU
Examples
>>> input_params = Tensor(np.array([[8, 9], [10, 11], [12, 13], [14, 15]]), mindspore.float32) >>> input_indices = Tensor(np.array([[5, 2], [8, 5]]), mindspore.int32) >>> offset = 4 >>> output = ops.EmbeddingLookup()(input_params, input_indices, offset) >>> print(output) [[[10. 11.] [ 0. 0.]] [[ 0. 0.] [10. 11.]]]
-
class
tinyms.primitives.Eps(*args, **kwargs)[source]¶ Creates a tensor filled with input_x dtype minimum value.
- Inputs:
input_x (Tensor) - Input tensor. The data type must be float16 or float32.
- Outputs:
Tensor, has the same type and shape as input_x, but filled with input_x dtype minimum val.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor([4, 1, 2, 3], mindspore.float32) >>> output = ops.Eps()(input_x) >>> print(output) [1.5258789e-05 1.5258789e-05 1.5258789e-05 1.5258789e-05]
-
class
tinyms.primitives.Equal(*args, **kwargs)[source]¶ Computes the equivalence between two tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number]) - The first input is a number or a tensor whose data type is number.
input_y (Union[Tensor, Number]) - The second input is a number when the first input is a tensor or a tensor whose data type is number. The data type is the same as the first input.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is bool.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> equal = ops.Equal() >>> output = equal(input_x, 2.0) >>> print(output) [False True False] >>> >>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 2, 4]), mindspore.int32) >>> equal = ops.Equal() >>> output = equal(input_x, input_y) >>> print(output) [ True True False]
-
class
tinyms.primitives.EqualCount(*args, **kwargs)[source]¶ Computes the number of the same elements of two tensors.
The two input tensors must have the same data type and shape.
- Inputs:
input_x (Tensor) - The first input tensor.
input_y (Tensor) - The second input tensor.
- Outputs:
Tensor, with the type same as input tensor and size as (1,).
- Supported Platforms:
GPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 2, 4]), mindspore.int32) >>> equal_count = ops.EqualCount() >>> output = equal_count(input_x, input_y) >>> print(output) [2]
-
class
tinyms.primitives.Erf(*args, **kwargs)[source]¶ Computes the Gauss error function of input_x element-wise.
\[erf(x)=\frac{2} {\sqrt{\pi}} \int\limits_0^{x} e^{-t^{2}} dt\]- Inputs:
input_x (Tensor) - The input tensor. The data type must be float16 or float32.
- Outputs:
Tensor, has the same shape and dtype as the input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([-1, 0, 1, 2, 3]), mindspore.float32) >>> erf = ops.Erf() >>> output = erf(input_x) >>> print(output) [-0.8427168 0. 0.8427168 0.99530876 0.99997765]
-
class
tinyms.primitives.Erfc(*args, **kwargs)[source]¶ Computes the complementary error function of input_x element-wise.
\[erfc(x) = 1 - \frac{2} {\sqrt{\pi}} \int\limits_0^{x} e^{-t^{2}} dt\]- Inputs:
input_x (Tensor) - The input tensor. The data type must be float16 or float32.
- Outputs:
Tensor, has the same shape and dtype as the input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([-1, 0, 1, 2, 3]), mindspore.float32) >>> erfc = ops.Erfc() >>> output = erfc(input_x) >>> print(output) [1.8427168e+00 1.0000000e+00 1.5728319e-01 4.6912432e-03 2.2351742e-05]
-
class
tinyms.primitives.Exp(*args, **kwargs)[source]¶ Returns exponential of a tensor element-wise.
\[out_i = e^{x_i}\]- Inputs:
input_x (Tensor) - The input tensor. The data type mast be float16 or float32.
- Outputs:
Tensor, has the same shape and dtype as the input_x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) >>> exp = ops.Exp() >>> output = exp(input_x) >>> print(output) [ 2.718282 7.389056 54.598152]
-
class
tinyms.primitives.ExpandDims(*args, **kwargs)[source]¶ Adds an additional dimension at the given axis.
Note
If the specified axis is a negative number, the index is counted backward from the end and starts at 1.
- Raises
ValueError – If axis is not an integer or not in the valid range.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
axis (int) - Specifies the dimension index at which to expand the shape of input_x. The value of axis must be in the range [-input_x.ndim-1, input_x.ndim]. Only constant value is allowed.
- Outputs:
Tensor, the shape of tensor is \((1, x_1, x_2, ..., x_R)\) if the value of axis is 0. It has the same type as input_x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) >>> expand_dims = ops.ExpandDims() >>> output = expand_dims(input_tensor, 0) >>> print(output) [[[2. 2.] [2. 2.]]]
-
class
tinyms.primitives.Expm1(*args, **kwargs)[source]¶ Returns exponential then minus 1 of a tensor element-wise.
\[out_i = e^{x_i} - 1\]- Inputs:
input_x (Tensor) - The input tensor. With float16 or float32 data type.
- Outputs:
Tensor, has the same shape as the input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([0.0, 1.0, 2.0, 4.0]), mindspore.float32) >>> expm1 = ops.Expm1() >>> output = expm1(input_x) >>> print(output) [ 0. 1.718282 6.389056 53.598152]
-
class
tinyms.primitives.Eye(*args, **kwargs)[source]¶ Creates a tensor with ones on the diagonal and zeros the rest.
- Inputs:
n (int) - The number of rows of returned tensor
m (int) - The number of columns of returned tensor
t (mindspore.dtype) - MindSpore’s dtype, The data type of the returned tensor.
- Outputs:
Tensor, a tensor with ones on the diagonal and the rest of elements are zero.
- Supported Platforms:
AscendGPUCPU
Examples
>>> eye = ops.Eye() >>> output = eye(2, 2, mindspore.int32) >>> print(output) [[1 0] [0 1]]
-
class
tinyms.primitives.FastGeLU(*args, **kwargs)[source]¶ Fast Gaussian Error Linear Units activation function.
FastGeLU is defined as follows:
\[\text{output} = \frac {x} {1 + \exp(-1.702 * \left| x \right|)} * \exp(0.851 * (x - \left| x \right|)),\]where \(x\) is the element of the input.
- Inputs:
input_x (Tensor) - Input to compute the FastGeLU with data type of float16 or float32.
- Outputs:
Tensor, with the same type and shape as input.
- Supported Platforms:
Ascend
Examples
>>> tensor = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> fast_gelu = P.FastGeLU() >>> output = fast_gelu(tensor) >>> print(output) [[-1.5420423e-01 3.9955849e+00 -9.7664278e-06] [ 1.9356585e+00 -1.0070159e-03 8.9999981e+00]]
-
tinyms.primitives.FastGelu()[source]¶ Fast Gaussian Error Linear Units activation function.
The usage of FastGelu is deprecated. Please use FastGeLU.
-
class
tinyms.primitives.Fill(*args, **kwargs)[source]¶ Creates a tensor filled with a scalar value.
Creates a tensor with shape described by the first argument and fills it with values in the second argument.
- Inputs:
type (mindspore.dtype) - The specified type of output tensor. Only constant value is allowed.
shape (tuple) - The specified shape of output tensor. Only constant value is allowed.
value (scalar) - Value to fill the returned tensor. Only constant value is allowed.
- Outputs:
Tensor, has the same type and shape as input value.
- Supported Platforms:
AscendGPUCPU
Examples
>>> fill = ops.Fill() >>> output = fill(mindspore.float32, (2, 2), 1) >>> print(output) [[1. 1.] [1. 1.]]
-
class
tinyms.primitives.Flatten(*args, **kwargs)[source]¶ Flattens a tensor without changing its batch size on the 0-th axis.
- Inputs:
input_x (Tensor) - Tensor of shape \((N, \ldots)\) to be flattened.
- Outputs:
Tensor, the shape of the output tensor is \((N, X)\), where \(X\) is the product of the remaining dimension.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.ones(shape=[1, 2, 3, 4]), mindspore.float32) >>> flatten = ops.Flatten() >>> output = flatten(input_tensor) >>> print(output.shape) (1, 24)
-
class
tinyms.primitives.FloatStatus(*args, **kwargs)[source]¶ Determines if the elements contain Not a Number(NaN), infinite or negative infinite. 0 for normal, 1 for overflow.
- Inputs:
input_x (Tensor) - The input tensor. The data type must be float16 or float32.
- Outputs:
Tensor, has the shape of (1,), and has the same dtype of input mindspore.dtype.float32 or mindspore.dtype.float16.
- Supported Platforms:
GPU
Examples
>>> float_status = ops.FloatStatus() >>> input_x = Tensor(np.array([np.log(-1), 1, np.log(0)]), mindspore.float32) >>> result = float_status(input_x) >>> print(result) [1.]
-
class
tinyms.primitives.Floor(*args, **kwargs)[source]¶ Rounds a tensor down to the closest integer element-wise.
\[out_i = \lfloor x_i \rfloor\]- Inputs:
input_x (Tensor) - The input tensor. Its element data type must be float.
- Outputs:
Tensor, has the same shape as input_x.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([1.1, 2.5, -1.5]), mindspore.float32) >>> floor = ops.Floor() >>> output = floor(input_x) >>> print(output) [ 1. 2. -2.]
-
class
tinyms.primitives.FloorDiv(*args, **kwargs)[source]¶ Divides the first input tensor by the second input tensor element-wise and round down to the closest integer.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([2, 4, -1]), mindspore.int32) >>> input_y = Tensor(np.array([3, 3, 3]), mindspore.int32) >>> floor_div = ops.FloorDiv() >>> output = floor_div(input_x, input_y) >>> print(output) [ 0 1 -1]
-
class
tinyms.primitives.FloorMod(*args, **kwargs)[source]¶ Computes the remainder of division element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool , and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([2, 4, -1]), mindspore.int32) >>> input_y = Tensor(np.array([3, 3, 3]), mindspore.int32) >>> floor_mod = ops.FloorMod() >>> output = floor_mod(input_x, input_y) >>> print(output) [2 1 2]
-
class
tinyms.primitives.FusedBatchNorm(*args, **kwargs)[source]¶ FusedBatchNorm is a BatchNorm. Moving mean and moving variance will be computed instead of being loaded.
Batch Normalization is widely used in convolutional networks. This operation applies Batch Normalization over input to avoid internal covariate shift as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. It rescales and recenters the feature using a mini-batch of data and the learned parameters which can be described in the following formula.
\[y = \frac{x - mean}{\sqrt{variance + \epsilon}} * \gamma + \beta\]where \(\gamma\) is scale, \(\beta\) is bias, \(\epsilon\) is epsilon.
- Parameters
mode (int) – Mode of batch normalization, value is 0 or 1. Default: 0.
epsilon (float) – A small value added for numerical stability. Default: 1e-5.
momentum (float) – The hyper parameter to compute moving average for running_mean and running_var (e.g. \(new\_running\_mean = momentum * running\_mean + (1 - momentum) * current\_mean\)). Momentum value must be [0, 1]. Default: 0.9.
- Inputs:
input_x (Tensor) - Tensor of shape \((N, C)\).
scale (Parameter) - Tensor of shape \((C,)\).
bias (Parameter) - Tensor of shape \((C,)\).
mean (Parameter) - Tensor of shape \((C,)\).
variance (Parameter) - Tensor of shape \((C,)\).
- Outputs:
Tuple of 5 Tensor, the normalized input and the updated parameters.
output_x (Tensor) - The same type and shape as the input_x.
updated_scale (Tensor) - Tensor of shape \((C,)\).
updated_bias (Tensor) - Tensor of shape \((C,)\).
updated_moving_mean (Tensor) - Tensor of shape \((C,)\).
updated_moving_variance (Tensor) - Tensor of shape \((C,)\).
- Supported Platforms:
CPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Parameter >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class FusedBatchNormNet(nn.Cell): >>> def __init__(self): >>> super(FusedBatchNormNet, self).__init__() >>> self.fused_batch_norm = ops.FusedBatchNorm() >>> self.scale = Parameter(Tensor(np.ones([64]), mindspore.float32), name="scale") >>> self.bias = Parameter(Tensor(np.ones([64]), mindspore.float32), name="bias") >>> self.mean = Parameter(Tensor(np.ones([64]), mindspore.float32), name="mean") >>> self.variance = Parameter(Tensor(np.ones([64]), mindspore.float32), name="variance") >>> >>> def construct(self, input_x): >>> out = self.fused_batch_norm(input_x, self.scale, self.bias, self.mean, self.variance) >>> return out >>> >>> input_x = Tensor(np.ones([128, 64, 32, 64]), mindspore.float32) >>> net = FusedBatchNormNet() >>> output = net(input_x) >>> result = output[0].shape >>> print(result) (128, 64, 32, 64)
-
class
tinyms.primitives.FusedBatchNormEx(*args, **kwargs)[source]¶ FusedBatchNormEx is an extension of FusedBatchNorm, FusedBatchNormEx has one more output(output reserve) than FusedBatchNorm, reserve will be used in backpropagation phase. FusedBatchNorm is a BatchNorm that moving mean and moving variance will be computed instead of being loaded.
Batch Normalization is widely used in convolutional networks. This operation applies Batch Normalization over input to avoid internal covariate shift as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. It rescales and recenters the feature using a mini-batch of data and the learned parameters which can be described in the following formula.
\[y = \frac{x - mean}{\sqrt{variance + \epsilon}} * \gamma + \beta\]where \(\gamma\) is scale, \(\beta\) is bias, \(\epsilon\) is epsilon.
- Parameters
mode (int) – Mode of batch normalization, value is 0 or 1. Default: 0.
epsilon (float) – A small value added for numerical stability. Default: 1e-5.
momentum (float) – The hyper parameter to compute moving average for running_mean and running_var (e.g. \(new\_running\_mean = momentum * running\_mean + (1 - momentum) * current\_mean\)). Momentum value must be [0, 1]. Default: 0.9.
data_format (str) – The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: “NCHW”.
- Inputs:
input_x (Tensor) - The input of FusedBatchNormEx, Tensor of shape \((N, C)\), data type: float16 or float32.
scale (Parameter) - Parameter scale, same with gamma above-mentioned, Tensor of shape \((C,)\), data type: float32.
bias (Parameter) - Parameter bias, same with beta above-mentioned, Tensor of shape \((C,)\), data type: float32.
mean (Parameter) - mean value, Tensor of shape \((C,)\), data type: float32.
variance (Parameter) - variance value, Tensor of shape \((C,)\), data type: float32.
- Outputs:
Tuple of 6 Tensors, the normalized input, the updated parameters and reserve.
output_x (Tensor) - The output of FusedBatchNormEx, same type and shape as the input_x.
updated_scale (Tensor) - Updated parameter scale, Tensor of shape \((C,)\), data type: float32.
updated_bias (Tensor) - Updated parameter bias, Tensor of shape \((C,)\), data type: float32.
updated_moving_mean (Tensor) - Updated mean value, Tensor of shape \((C,)\), data type: float32.
updated_moving_variance (Tensor) - Updated variance value, Tensor of shape \((C,)\), data type: float32.
reserve (Tensor) - reserve space, Tensor of shape \((C,)\), data type: float32.
- Supported Platforms:
GPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Parameter >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class FusedBatchNormExNet(nn.Cell): >>> def __init__(self): >>> super(FusedBatchNormExNet, self).__init__() >>> self.fused_batch_norm_ex = ops.FusedBatchNormEx() >>> self.scale = Parameter(Tensor(np.ones([64]), mindspore.float32), name="scale") >>> self.bias = Parameter(Tensor(np.ones([64]), mindspore.float32), name="bias") >>> self.mean = Parameter(Tensor(np.ones([64]), mindspore.float32), name="mean") >>> self.variance = Parameter(Tensor(np.ones([64]), mindspore.float32), name="variance") >>> >>> def construct(self, input_x): >>> out = self.fused_batch_norm_ex(input_x, self.scale, self.bias, self.mean, self.variance) >>> return out >>> >>> input_x = Tensor(np.ones([128, 64, 32, 64]), mindspore.float32) >>> net = FusedBatchNormExNet() >>> output = net(input_x) >>> result = output[0].shape >>> print(result) (128, 64, 32, 64)
-
class
tinyms.primitives.FusedSparseAdam(*args, **kwargs)[source]¶ Merges the duplicate value of the gradient and then updates parameters by the Adaptive Moment Estimation (Adam) algorithm. This operator is used when the gradient is sparse.
The Adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ w = w - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector, \(v\) represents the 2nd moment vector, \(g\) represents gradient, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents var, \(\epsilon\) represents epsilon.
All of inputs except indices comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
- Inputs:
var (Parameter) - Parameters to be updated with float32 data type.
m (Parameter) - The 1st moment vector in the updating formula, has the same type as var with float32 data type.
v (Parameter) - The 2nd moment vector in the updating formula. Mean square gradients, has the same type as var with float32 data type.
beta1_power (Tensor) - \(beta_1^t\) in the updating formula with float32 data type.
beta2_power (Tensor) - \(beta_2^t\) in the updating formula with float32 data type.
lr (Tensor) - \(l\) in the updating formula. With float32 data type.
beta1 (Tensor) - The exponential decay rate for the 1st moment estimations with float32 data type.
beta2 (Tensor) - The exponential decay rate for the 2nd moment estimations with float32 data type.
epsilon (Tensor) - Term added to the denominator to improve numerical stability with float32 data type.
gradient (Tensor) - Gradient value with float32 data type.
indices (Tensor) - Gradient indices with int32 data type.
- Outputs:
Tuple of 3 Tensors, this operator will update the input parameters directly, the outputs are useless.
var (Tensor) - A Tensor with shape (1,).
m (Tensor) - A Tensor with shape (1,).
v (Tensor) - A Tensor with shape (1,).
- Supported Platforms:
CPU
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.sparse_apply_adam = ops.FusedSparseAdam() ... self.var = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="var") ... self.m = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="m") ... self.v = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="v") ... def construct(self, beta1_power, beta2_power, lr, beta1, beta2, epsilon, grad, indices): ... out = self.sparse_apply_adam(self.var, self.m, self.v, beta1_power, beta2_power, lr, beta1, beta2, ... epsilon, grad, indices) ... return out ... >>> net = Net() >>> beta1_power = Tensor(0.9, mstype.float32) >>> beta2_power = Tensor(0.999, mstype.float32) >>> lr = Tensor(0.001, mstype.float32) >>> beta1 = Tensor(0.9, mstype.float32) >>> beta2 = Tensor(0.999, mstype.float32) >>> epsilon = Tensor(1e-8, mstype.float32) >>> gradient = Tensor(np.random.rand(2, 1, 2), mstype.float32) >>> indices = Tensor([0, 1], mstype.int32) >>> output = net(beta1_power, beta2_power, lr, beta1, beta2, epsilon, gradient, indices) >>> print(output)
-
class
tinyms.primitives.FusedSparseFtrl(*args, **kwargs)[source]¶ Merges the duplicate value of the gradient and then updates relevant entries according to the FTRL-proximal scheme.
All of inputs except indices comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
lr (float) – The learning rate value, must be positive.
l1 (float) – l1 regularization strength, must be greater than or equal to zero.
l2 (float) – l2 regularization strength, must be greater than or equal to zero.
lr_power (float) – Learning rate power controls how the learning rate decreases during training, must be less than or equal to zero. Use fixed learning rate if lr_power is zero.
use_locking (bool) – Use locks for updating operation if true . Default: False.
- Inputs:
var (Parameter) - The variable to be updated. The data type must be float32.
accum (Parameter) - The accumulation to be updated, must be same type and shape as var.
linear (Parameter) - the linear coefficient to be updated, must be same type and shape as var.
grad (Tensor) - A tensor of the same type as var, for the gradient.
indices (Tensor) - A vector of indices into the first dimension of var and accum. The shape of indices must be the same as grad in first dimension. The type must be int32.
- Outputs:
Tuple of 3 Tensor, this operator will update the input parameters directly, the outputs are useless.
var (Tensor) - A Tensor with shape (1,).
accum (Tensor) - A Tensor with shape (1,).
linear (Tensor) - A Tensor with shape (1,).
- Supported Platforms:
AscendCPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Parameter >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class SparseApplyFtrlNet(nn.Cell): ... def __init__(self): ... super(SparseApplyFtrlNet, self).__init__() ... self.sparse_apply_ftrl = ops.FusedSparseFtrl(lr=0.01, l1=0.0, l2=0.0, lr_power=-0.5) ... self.var = Parameter(Tensor(np.random.rand(3, 1, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(3, 1, 2).astype(np.float32)), name="accum") ... self.linear = Parameter(Tensor(np.random.rand(3, 1, 2).astype(np.float32)), name="linear") ... ... def construct(self, grad, indices): ... out = self.sparse_apply_ftrl(self.var, self.accum, self.linear, grad, indices) ... return out ... >>> net = SparseApplyFtrlNet() >>> grad = Tensor(np.random.rand(2, 1, 2).astype(np.float32)) >>> indices = Tensor(np.array([0, 1]).astype(np.int32)) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[1], dtype=Float32, value= [0.00000000e+00]), Tensor(shape=[1], dtype=Float32, value= [0.00000000e+00]), Tensor(shape=[1], dtype=Float32, value= [0.00000000e+00]))
-
class
tinyms.primitives.FusedSparseLazyAdam(*args, **kwargs)[source]¶ Merges the duplicate value of the gradient and then updates parameters by the Adaptive Moment Estimation (LazyAdam) algorithm. This operator is used when the gradient is sparse. The behavior is not equivalent to the original Adam algorithm, as only the current indices parameters will be updated.
The Adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ w = w - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector, \(v\) represents the 2nd moment vector, \(g\) represents gradient, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents var, \(\epsilon\) represents epsilon.
All of inputs except indices comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
- Inputs:
var (Parameter) - Parameters to be updated with float32 data type.
m (Parameter) - The 1st moment vector in the updating formula, has the same type as var with float32 data type.
v (Parameter) - The 2nd moment vector in the updating formula. Mean square gradients, has the same type as var with float32 data type.
beta1_power (Tensor) - \(beta_1^t\) in the updating formula with float32 data type.
beta2_power (Tensor) - \(beta_2^t\) in the updating formula with float32 data type.
lr (Tensor) - \(l\) in the updating formula with float32 data type.
beta1 (Tensor) - The exponential decay rate for the 1st moment estimations with float32 data type.
beta2 (Tensor) - The exponential decay rate for the 2nd moment estimations with float32 data type.
epsilon (Tensor) - Term added to the denominator to improve numerical stability with float32 data type.
gradient (Tensor) - Gradient value with float32 data type.
indices (Tensor) - Gradient indices with int32 data type.
- Outputs:
Tuple of 3 Tensors, this operator will update the input parameters directly, the outputs are useless.
var (Tensor) - A Tensor with shape (1,).
m (Tensor) - A Tensor with shape (1,).
v (Tensor) - A Tensor with shape (1,).
- Supported Platforms:
CPU
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.sparse_apply_lazyadam = ops.FusedSparseLazyAdam() ... self.var = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="var") ... self.m = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="m") ... self.v = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="v") ... def construct(self, beta1_power, beta2_power, lr, beta1, beta2, epsilon, grad, indices): ... out = self.sparse_apply_lazyadam(self.var, self.m, self.v, beta1_power, beta2_power, lr, beta1, ... beta2, epsilon, grad, indices) ... return out ... >>> net = Net() >>> beta1_power = Tensor(0.9, mstype.float32) >>> beta2_power = Tensor(0.999, mstype.float32) >>> lr = Tensor(0.001, mstype.float32) >>> beta1 = Tensor(0.9, mstype.float32) >>> beta2 = Tensor(0.999, mstype.float32) >>> epsilon = Tensor(1e-8, mstype.float32) >>> gradient = Tensor(np.random.rand(2, 1, 2), mstype.float32) >>> indices = Tensor([0, 1], mstype.int32) >>> output = net(beta1_power, beta2_power, lr, beta1, beta2, epsilon, gradient, indices) >>> print(output)
-
class
tinyms.primitives.FusedSparseProximalAdagrad(*args, **kwargs)[source]¶ Merges the duplicate value of the gradient and then updates relevant entries according to the proximal adagrad algorithm.
\[accum += grad * grad\]\[\text{prox_v} = var - lr * grad * \frac{1}{\sqrt{accum}}\]\[var = \frac{sign(\text{prox_v})}{1 + lr * l2} * \max(\left| \text{prox_v} \right| - lr * l1, 0)\]All of inputs except indices comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – If true, the variable and accumulation tensors will be protected from being updated. Default: False.
- Inputs:
var (Parameter) - Variable tensor to be updated. The data type must be float32.
accum (Parameter) - Variable tensor to be updated, has the same dtype as var.
lr (Tensor) - The learning rate value. The data type must be float32.
l1 (Tensor) - l1 regularization strength. The data type must be float32.
l2 (Tensor) - l2 regularization strength. The data type must be float32.
grad (Tensor) - A tensor of the same type as var, for the gradient. The data type must be float32.
indices (Tensor) - A vector of indices into the first dimension of var and accum. The data type must be int32.
- Outputs:
Tuple of 2 Tensors, this operator will update the input parameters directly, the outputs are useless.
var (Tensor) - A Tensor with shape (1,).
accum (Tensor) - A Tensor with shape (1,).
- Supported Platforms:
CPU
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> import mindspore.common.dtype as mstype >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.sparse_apply_proximal_adagrad = ops.FusedSparseProximalAdagrad() ... self.var = Parameter(Tensor(np.random.rand(3, 1, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(3, 1, 2).astype(np.float32)), name="accum") ... self.lr = Tensor(0.01, mstype.float32) ... self.l1 = Tensor(0.0, mstype.float32) ... self.l2 = Tensor(0.0, mstype.float32) ... def construct(self, grad, indices): ... out = self.sparse_apply_proximal_adagrad(self.var, self.accum, self.lr, self.l1, ... self.l2, grad, indices) ... return out ... >>> net = Net() >>> grad = Tensor(np.random.rand(2, 1, 2).astype(np.float32)) >>> indices = Tensor(np.array([0, 1]).astype(np.int32)) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[1], dtype=Float32, value= [0.00000000e+00]), Tensor(shape=[1], dtype=Float32, value= [0.00000000e+00]))
-
class
tinyms.primitives.Gamma(*args, **kwargs)[source]¶ Produces random positive floating-point values x, distributed according to probability density function:
\[\text{P}(x|α,β) = \frac{\exp(-x/β)}{{β^α}\cdot{\Gamma(α)}}\cdot{x^{α-1}},\]- Parameters
- Inputs:
shape (tuple) - The shape of random tensor to be generated. Only constant value is allowed.
alpha (Tensor) - The α distribution parameter. It must be greater than 0. It is also known as the shape parameter with float32 data type.
beta (Tensor) - The β distribution parameter. It must be greater than 0. It is also known as the scale parameter with float32 data type.
- Outputs:
Tensor. The shape must be the broadcasted shape of Input “shape” and shapes of alpha and beta. The dtype is float32.
- Supported Platforms:
Ascend
Examples
>>> shape = (3, 1, 2) >>> alpha = Tensor(np.array([[3, 4], [5, 6]]), mstype.float32) >>> beta = Tensor(np.array([1.0]), mstype.float32) >>> gamma = ops.Gamma(seed=3) >>> output = gamma(shape, alpha, beta) >>> result = output.shape >>> print(result) (3, 2, 2)
-
class
tinyms.primitives.Gather(*args, **kwargs)[source]¶ Returns a slice of the input tensor based on the specified indices and axis.
- Inputs:
input_params (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). The original Tensor.
input_indices (Tensor) - The shape of tensor is \((y_1, y_2, ..., y_S)\). Specifies the indices of elements of the original Tensor. Must be in the range [0, input_param.shape[axis]).
axis (int) - Specifies the dimension index to gather indices.
- Outputs:
Tensor, the shape of tensor is \((z_1, z_2, ..., z_N)\).
- Supported Platforms:
AscendGPU
Examples
>>> input_params = Tensor(np.array([[1, 2, 7, 42], [3, 4, 54, 22], [2, 2, 55, 3]]), mindspore.float32) >>> input_indices = Tensor(np.array([1, 2]), mindspore.int32) >>> axis = 1 >>> output = ops.Gather()(input_params, input_indices, axis) >>> print(output) [[ 2. 7.] [ 4. 54.] [ 2. 55.]]
-
class
tinyms.primitives.GatherD(*args, **kwargs)[source]¶ Gathers values along an axis specified by dim.
- Inputs:
x (Tensor) - The source tensor.
dim (int) - The axis along which to index. It must be int32. Only constant value is allowed.
index (Tensor) - The indices of elements to gather. It can be one of the following data types: int32, int64. The value range of each index element is [-x_rank[dim], x_rank[dim]).
- Outputs:
Tensor, the shape of tensor is \((z_1, z_2, ..., z_N)\).
- Supported Platforms:
AscendGPU
Examples
>>> x = Tensor(np.array([[1, 2], [3, 4]]), mindspore.int32) >>> index = Tensor(np.array([[0, 0], [1, 0]]), mindspore.int32) >>> dim = 1 >>> output = ops.GatherD()(x, dim, index) >>> print(output) [[1 1] [4 3]]
-
class
tinyms.primitives.GatherNd(*args, **kwargs)[source]¶ Gathers slices from a tensor by indices.
Using given indices to gather slices from a tensor with a specified shape.
- Inputs:
input_x (Tensor) - The target tensor to gather values.
indices (Tensor) - The index tensor, with int data type.
- Outputs:
Tensor, has the same type as input_x and the shape is indices_shape[:-1] + x_shape[indices_shape[-1]:].
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]), mindspore.float32) >>> indices = Tensor(np.array([[0, 0], [1, 1]]), mindspore.int32) >>> op = ops.GatherNd() >>> output = op(input_x, indices) >>> print(output) [-0.1 0.5]
-
tinyms.primitives.GatherV2()[source]¶ Returns a slice of the input tensor based on the specified indices and axis.
The usage of GatherV2 is deprecated. Please use Gather.
-
class
tinyms.primitives.GeLU(*args, **kwargs)[source]¶ Gaussian Error Linear Units activation function.
GeLU is described in the paper Gaussian Error Linear Units (GELUs). And also please refer to BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.
GeLU is defined as follows:
\[\text{output} = 0.5 * x * (1 + erf(x / \sqrt{2})),\]where \(erf\) is the “Gauss error function” .
- Inputs:
input_x (Tensor) - Input to compute the GeLU with data type of float16 or float32.
- Outputs:
Tensor, with the same type and shape as input.
- Supported Platforms:
AscendGPU
Examples
>>> tensor = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) >>> gelu = ops.GeLU() >>> result = gelu(tensor) >>> print(result) [0.841192 1.9545976 2.9963627]
-
class
tinyms.primitives.GeSwitch(*args, **kwargs)[source]¶ Adds control switch to data.
Switch data flows into false or true branch depending on the condition. If the condition is true, the true branch will be activated, or vise verse.
- Inputs:
data (Union[Tensor, Number]) - The data to be used for switch control.
pred (Tensor) - It must be a scalar whose type is bool and shape is (), It is used as condition for switch control.
- Outputs:
tuple. Output is tuple(false_output, true_output). The Elements in the tuple has the same shape of input data. The false_output connects with the false_branch and the true_output connects with the true_branch.
Examples
>>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.square = ops.Square() ... self.add = ops.Add() ... self.value = Tensor(np.full((1), 3), mindspore.float32) ... self.switch = ops.GeSwitch() ... self.merge = ops.Merge() ... self.less = ops.Less() ... ... def construct(self, x, y): ... cond = self.less(x, y) ... st1, sf1 = self.switch(x, cond) ... st2, sf2 = self.switch(y, cond) ... add_ret = self.add(st1, st2) ... st3, sf3 = self.switch(self.value, cond) ... sq_ret = self.square(sf3) ... ret = self.merge((add_ret, sq_ret)) ... return ret[0] ... >>> x = Tensor(10.0, dtype=mindspore.float32) >>> y = Tensor(5.0, dtype=mindspore.float32) >>> net = Net() >>> output = net(x, y) >>> print(output)
-
tinyms.primitives.Gelu()[source]¶ Gaussian Error Linear Units activation function.
The usage of Gelu is deprecated. Please use GeLU.
-
class
tinyms.primitives.GetNext(*args, **kwargs)[source]¶ Returns the next element in the dataset queue.
Note
The GetNext operation needs to be associated with network and it also depends on the init_dataset interface, it can’t be used directly as a single operation. For details, please refer to connect_network_with_dataset source code.
- Parameters
- Inputs:
No inputs.
- Outputs:
tuple[Tensor], the output of Dataset. The shape is described in shapes and the type is described in types.
- Supported Platforms:
AscendGPU
Examples
>>> train_dataset = create_custom_dataset() >>> dataset_helper = mindspore.DatasetHelper(train_dataset, dataset_sink_mode=True) >>> dataset = dataset_helper.iter.dataset >>> dataset_types, dataset_shapes = dataset_helper.types_shapes() >>> queue_name = dataset.__transfer_dataset__.queue_name >>> get_next = P.GetNext(dataset_types, dataset_shapes, len(dataset_types), queue_name) >>> data, label = get_next() >>> relu = P.ReLU() >>> result = relu(data).asnumpy() >>> print(result.shape) (32, 1, 32, 32)
-
class
tinyms.primitives.Greater(*args, **kwargs)[source]¶ Computes the boolean value of \(x > y\) element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is bool.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 1, 4]), mindspore.int32) >>> greater = ops.Greater() >>> output = greater(input_x, input_y) >>> print(output) [False True False]
-
class
tinyms.primitives.GreaterEqual(*args, **kwargs)[source]¶ Computes the boolean value of \(x >= y\) element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is bool.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 1, 4]), mindspore.int32) >>> greater_equal = ops.GreaterEqual() >>> output = greater_equal(input_x, input_y) >>> print(output) [ True True False]
-
class
tinyms.primitives.HSigmoid(*args, **kwargs)[source]¶ Hard sigmoid activation function.
Applies hard sigmoid activation element-wise. The input is a Tensor with any valid shape.
Hard sigmoid is defined as:
\[\text{hsigmoid}(x_{i}) = max(0, min(1, \frac{x_{i} + 3}{6})),\]where \(x_{i}\) is the \(i\)-th slice in the given dimension of the input Tensor.
- Inputs:
input_data (Tensor) - The input of HSigmoid, data type must be float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Supported Platforms:
GPU
Examples
>>> hsigmoid = ops.HSigmoid() >>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> result = hsigmoid(input_x) >>> print(result) [0.3333 0.1666 0.5 0.833 0.6665]
-
class
tinyms.primitives.HSwish(*args, **kwargs)[source]¶ Hard swish activation function.
Applies hswish-type activation element-wise. The input is a Tensor with any valid shape.
Hard swish is defined as:
\[\text{hswish}(x_{i}) = x_{i} * \frac{ReLU6(x_{i} + 3)}{6},\]where \(x_{i}\) is the \(i\)-th slice in the given dimension of the input Tensor.
- Inputs:
input_data (Tensor) - The input of HSwish, data type must be float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Supported Platforms:
GPU
Examples
>>> hswish = ops.HSwish() >>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> result = hswish(input_x) >>> print(result) [-0.3333 -0.3333 0 1.666 0.6665]
-
class
tinyms.primitives.HistogramFixedWidth(*args, **kwargs)[source]¶ Returns a rank 1 histogram counting the number of entries in values that fall into every bin. The bins are equal width and determined by the arguments range and nbins.
- Parameters
- Inputs:
x (Tensor) - Numeric Tensor. Must be one of the following types: int32, float32, float16.
range (Tensor) - Must has the same data type as x, and the shape is [2]. x <= range[0] will be mapped to hist[0], x >= range[1] will be mapped to hist[-1].
- Outputs:
Tensor, the type is int32.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor([-1.0, 0.0, 1.5, 2.0, 5.0, 15], mindspore.float16) >>> range = Tensor([0.0, 5.0], mindspore.float16) >>> hist = ops.HistogramFixedWidth(5) >>> output = hist(x, range) >>> print(output) [2 1 1 0 2]
-
class
tinyms.primitives.HistogramSummary(*args, **kwargs)[source]¶ Outputs the tensor to protocol buffer through histogram summary operator.
- Inputs:
name (str) - The name of the input variable.
value (Tensor) - The value of tensor, and the rank of tensor must be greater than 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> class SummaryDemo(nn.Cell): ... def __init__(self,): ... super(SummaryDemo, self).__init__() ... self.summary = ops.HistogramSummary() ... self.add = ops.Add() ... ... def construct(self, x, y): ... x = self.add(x, y) ... name = "x" ... self.summary(name, x) ... return x ...
-
class
tinyms.primitives.HookBackward(hook_fn, cell_id='')[source]¶ This operation is used as a tag to hook gradient in intermediate variables. Note that this function is only supported in Pynative Mode.
Note
The hook function must be defined like hook_fn(grad) -> Tensor or None, where grad is the gradient passed to the primitive and gradient may be modified and passed to next primitive. The difference between a hook function and callback of InsertGradientOf is that a hook function is executed in the python environment while callback will be parsed and added to the graph.
- Parameters
hook_fn (Function) – Python function. hook function.
- Inputs:
inputs (Tensor) - The variable to hook.
Examples
>>> def hook_fn(grad_out): ... print(grad_out) ... >>> grad_all = GradOperation(get_all=True) >>> hook = ops.HookBackward(hook_fn) >>> def hook_test(x, y): ... z = x * y ... z = hook(z) ... z = z * y ... return z ... >>> def backward(x, y): ... return grad_all(hook_test)(x, y) ... >>> output = backward(1, 2) >>> print(output)
-
class
tinyms.primitives.IOU(*args, **kwargs)[source]¶ Calculates intersection over union for boxes.
Computes the intersection over union (IOU) or the intersection over foreground (IOF) based on the ground-truth and predicted regions.
\[ \begin{align}\begin{aligned}\text{IOU} = \frac{\text{Area of Overlap}}{\text{Area of Union}}\\\text{IOF} = \frac{\text{Area of Overlap}}{\text{Area of Ground Truth}}\end{aligned}\end{align} \]- Parameters
mode (string) – The mode is used to specify the calculation method, now supporting ‘iou’ (intersection over union) or ‘iof’ (intersection over foreground) mode. Default: ‘iou’.
- Inputs:
anchor_boxes (Tensor) - Anchor boxes, tensor of shape (N, 4). “N” indicates the number of anchor boxes, and the value “4” refers to “x0”, “y0”, “x1”, and “y1”. Data type must be float16 or float32.
gt_boxes (Tensor) - Ground truth boxes, tensor of shape (M, 4). “M” indicates the number of ground truth boxes, and the value “4” refers to “x0”, “y0”, “x1”, and “y1”. Data type must be float16 or float32.
- Outputs:
Tensor, the ‘iou’ values, tensor of shape (M, N), with the same data type as anchor_boxes.
- Raises
KeyError – When mode is not ‘iou’ or ‘iof’.
- Supported Platforms:
AscendGPU
Examples
>>> iou = ops.IOU() >>> anchor_boxes = Tensor(np.random.randint(1.0, 5.0, [3, 4]), mindspore.float16) >>> gt_boxes = Tensor(np.random.randint(1.0, 5.0, [3, 4]), mindspore.float16) >>> output = iou(anchor_boxes, gt_boxes) >>> print(output.shape) (3, 3)
-
class
tinyms.primitives.Identity(*args, **kwargs)[source]¶ Returns a Tensor with the same shape and contents as input.
- Inputs:
x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, the shape of tensor is the same as input_x, \((x_1, x_2, ..., x_R)\).
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.array([1, 2, 3, 4]), mindspore.int64) >>> output = ops.Identity()(x) >>> print(output) [1 2 3 4]
-
class
tinyms.primitives.ImageSummary(*args, **kwargs)[source]¶ Outputs the image tensor to protocol buffer through image summary operator.
- Inputs:
name (str) - The name of the input variable, it must not be an empty string.
value (Tensor) - The value of image, the rank of tensor must be 4.
- Supported Platforms:
AscendGPUCPU
Examples
>>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.summary = ops.ImageSummary() ... ... def construct(self, x): ... name = "image" ... out = self.summary(name, x) ... return out ...
-
class
tinyms.primitives.InTopK(*args, **kwargs)[source]¶ Determines whether the targets are in the top k predictions.
- Parameters
k (int) – Specifies the number of top elements to be used for computing precision.
- Inputs:
x1 (Tensor) - A 2D Tensor defines the predictions of a batch of samples with float16 or float32 data type.
x2 (Tensor) - A 1D Tensor defines the labels of a batch of samples with int32 data type. The size of x2 must be equal to x1’s first dimension. The values of x2 can not be negative and must be equal to or less than index of x1’s second dimension.
- Outputs:
Tensor has 1 dimension of type bool and the same shape with x2. For labeling sample i in x2, if the label in the first k predictions for sample i is in x1, then the value is True, otherwise False.
- Supported Platforms:
Ascend
Examples
>>> x1 = Tensor(np.array([[1, 8, 5, 2, 7], [4, 9, 1, 3, 5]]), mindspore.float32) >>> x2 = Tensor(np.array([1, 3]), mindspore.int32) >>> in_top_k = ops.InTopK(3) >>> output = in_top_k(x1, x2) >>> print(output) [ True False]
-
class
tinyms.primitives.InplaceAdd(*args, **kwargs)[source]¶ Adds v into specified rows of x. Computes y = x; y[i,] += v.
- Parameters
indices (Union[int, tuple]) – Indices into the left-most dimension of x, and determines which rows of x to add with v. It is an integer or a tuple, whose value is in [0, the first dimension size of x).
- Inputs:
input_x (Tensor) - The first input is a tensor whose data type is float16, float32 or int32.
input_v (Tensor) - The second input is a tensor that has the same dimension sizes as x except the first dimension, which must be the same as indices’s size. It has the same data type with input_x.
- Outputs:
Tensor, has the same shape and dtype as input_x.
- Supported Platforms:
Ascend
Examples
>>> indices = (0, 1) >>> input_x = Tensor(np.array([[1, 2], [3, 4], [5, 6]]), mindspore.float32) >>> input_v = Tensor(np.array([[0.5, 1.0], [1.0, 1.5]]), mindspore.float32) >>> inplaceAdd = ops.InplaceAdd(indices) >>> output = inplaceAdd(input_x, input_v) >>> print(output) [[1.5 3. ] [4. 5.5] [5. 6. ]]
-
class
tinyms.primitives.InplaceSub(*args, **kwargs)[source]¶ Subtracts v into specified rows of x. Computes y = x; y[i, :] -= v.
- Parameters
indices (Union[int, tuple]) – Indices into the left-most dimension of x, and determines which rows of x to subtract with v. It is a int or tuple, whose value is in [0, the first dimension size of x).
- Inputs:
input_x (Tensor) - The first input is a tensor whose data type is float16, float32 or int32.
input_v (Tensor) - The second input is a tensor who has the same dimension sizes as x except the first dimension, which must be the same as indices’s size. It has the same data type with input_x.
- Outputs:
Tensor, has the same shape and dtype as input_x.
- Supported Platforms:
Ascend
Examples
>>> indices = (0, 1) >>> input_x = Tensor(np.array([[1, 2], [3, 4], [5, 6]]), mindspore.float32) >>> input_v = Tensor(np.array([[0.5, 1.0], [1.0, 1.5]]), mindspore.float32) >>> inplaceSub = ops.InplaceSub(indices) >>> output = inplaceSub(input_x, input_v) >>> print(output) [[0.5 1. ] [2. 2.5] [5. 6. ]]
-
class
tinyms.primitives.InplaceUpdate(*args, **kwargs)[source]¶ Updates specified rows with values in v.
- Parameters
indices (Union[int, tuple]) – Indices into the left-most dimension of x, and determines which rows of x to update with v. It is a int or tuple, whose value is in [0, the first dimension size of x).
- Inputs:
x (Tensor) - A tensor which to be inplace updated. It can be one of the following data types: float32, float16 and int32.
v (Tensor) - A tensor with the same type as x and the same dimension size as x except the first dimension, which must be the same as the size of indices.
- Outputs:
Tensor, with the same type and shape as the input x.
- Supported Platforms:
Ascend
Examples
>>> indices = (0, 1) >>> x = Tensor(np.array([[1, 2], [3, 4], [5, 6]]), mindspore.float32) >>> v = Tensor(np.array([[0.5, 1.0], [1.0, 1.5]]), mindspore.float32) >>> inplace_update = ops.InplaceUpdate(indices) >>> output = inplace_update(x, v) >>> print(output) [[0.5 1. ] [1. 1.5] [5. 6. ]]
-
class
tinyms.primitives.InsertGradientOf(*args, **kwargs)[source]¶ Attaches callback to the graph node that will be invoked on the node’s gradient.
- Parameters
f (Function) – MindSpore’s Function. Callback function.
- Inputs:
input_x (Any) - The graph node to attach to.
- Outputs:
Tensor, returns input_x directly. InsertGradientOf does not affect the forward result.
- Supported Platforms:
AscendGPUCPU
Examples
>>> def clip_gradient(dx): ... ret = dx ... if ret > 1.0: ... ret = 1.0 ... ... if ret < 0.2: ... ret = 0.2 ... ... return ret ... >>> clip = ops.InsertGradientOf(clip_gradient) >>> grad_all = ops.GradOperation(get_all=True) >>> def InsertGradientOfClipDemo(): ... def clip_test(x, y): ... x = clip(x) ... y = clip(y) ... c = x * y ... return c ... ... @ms_function ... def f(x, y): ... return clip_test(x, y) ... ... def fd(x, y): ... return grad_all(clip_test)(x, y) ... ... print("forward: ", f(1.1, 0.1)) ... print("clip_gradient:", fd(1.1, 0.1)) ...
-
class
tinyms.primitives.Inv(*args, **kwargs)[source]¶ Computes Inv(Reciprocal) of input tensor element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). Must be one of the following types: float16, float32, int32.
- Outputs:
Tensor, has the same shape and data type as input_x.
- Supported Platforms:
Ascend
Examples
>>> inv = ops.Inv() >>> input_x = Tensor(np.array([0.25, 0.4, 0.31, 0.52]), mindspore.float32) >>> output = inv(input_x) >>> print(output) [4. 2.5 3.2258065 1.923077 ]
-
class
tinyms.primitives.Invert(*args, **kwargs)[source]¶ Flips all bits of input tensor element-wise.
- Inputs:
input_x (Tensor[int16], Tensor[uint16]) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Supported Platforms:
Ascend
Examples
>>> invert = ops.Invert() >>> input_x = Tensor(np.array([25, 4, 13, 9]), mindspore.int16) >>> output = invert(input_x) >>> print(output) [-26 -5 -14 -10]
-
class
tinyms.primitives.InvertPermutation(*args, **kwargs)[source]¶ Computes the inverse of an index permutation.
Given a tuple input, this operation inserts a dimension of 1 at the dimension This operation calculates the inverse of the index replacement. It requires a 1-dimensional tuple x, which represents the array starting at zero, and swaps each value with its index position. In other words, for the output tuple y and the input tuple x, this operation calculates the following: \(y[x[i]] = i, \quad i \in [0, 1, \ldots, \text{len}(x)-1]\).
Note
These values must include 0. There must be no duplicate values and the values can not be negative.
- Inputs:
input_x (Union(tuple[int], list[int]) - The input is constructed by multiple integers, i.e., \((y_1, y_2, ..., y_S)\) representing the indices. The values must include 0. There can be no duplicate values or negative values. Only constant value is allowed. The maximum value must be equal to length of input_x.
- Outputs:
tuple[int]. It has the same length as the input.
- Supported Platforms:
AscendGPUCPU
Examples
>>> invert = ops.InvertPermutation() >>> input_data = (3, 4, 0, 2, 1) >>> output = invert(input_data) >>> print(output) (2, 4, 3, 0, 1)
-
class
tinyms.primitives.IsFinite(*args, **kwargs)[source]¶ Determines which elements are finite for each position.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, has the same shape of input, and the dtype is bool.
- Supported Platforms:
AscendGPUCPU
Examples
>>> is_finite = ops.IsFinite() >>> input_x = Tensor(np.array([np.log(-1), 1, np.log(0)]), mindspore.float32) >>> output = is_finite(input_x) >>> print(output) [False True False]
-
class
tinyms.primitives.IsInf(*args, **kwargs)[source]¶ Determines which elements are inf or -inf for each position
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, has the same shape of input, and the dtype is bool.
- Supported Platforms:
GPU
Examples
>>> is_inf = ops.IsInf() >>> input_x = Tensor(np.array([np.log(-1), 1, np.log(0)]), mindspore.float32) >>> output = is_inf(input_x) >>> print(output) [False False True]
-
class
tinyms.primitives.IsInstance(*args, **kwargs)[source]¶ Checks whether an object is an instance of a target type.
- Inputs:
inst (Any Object) - The instance to be checked. Only constant value is allowed.
type_ (mindspore.dtype) - The target type. Only constant value is allowed.
- Outputs:
bool, the check result.
- Supported Platforms:
AscendGPUCPU
Examples
>>> a = 1 >>> output = ops.IsInstance()(a, mindspore.int32) >>> print(output) False
-
class
tinyms.primitives.IsNan(*args, **kwargs)[source]¶ Determines which elements are NaN for each position.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, has the same shape of input, and the dtype is bool.
- Supported Platforms:
GPU
Examples
>>> is_nan = ops.IsNan() >>> input_x = Tensor(np.array([np.log(-1), 1, np.log(0)]), mindspore.float32) >>> output = is_nan(input_x) >>> print(output) [True False False]
-
class
tinyms.primitives.IsSubClass(*args, **kwargs)[source]¶ Checks whether this type is a sub-class of another type.
- Inputs:
sub_type (mindspore.dtype) - The type to be checked. Only constant value is allowed.
type_ (mindspore.dtype) - The target type. Only constant value is allowed.
- Outputs:
bool, the check result.
- Supported Platforms:
AscendGPUCPU
Examples
>>> output = ops.IsSubClass()(mindspore.int32, mindspore.intc) >>> print(output) True
-
class
tinyms.primitives.KLDivLoss(*args, **kwargs)[source]¶ Computes the Kullback-Leibler divergence between the target and the output.
Note
Sets input as \(x\), input label as \(y\), output as \(\ell(x, y)\). Let,
\[L = \{l_1,\dots,l_N\}^\top, \quad l_n = y_n \cdot (\log y_n - x_n)\]Then,
\[\begin{split}\ell(x, y) = \begin{cases} L, & \text{if reduction} = \text{'none';}\\ \operatorname{mean}(L), & \text{if reduction} = \text{'mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]- Parameters
reduction (str) – Specifies the reduction to be applied to the output. Its value must be one of ‘none’, ‘mean’, ‘sum’. Default: ‘mean’.
- Inputs:
input_x (Tensor) - The input Tensor. The data type must be float32.
input_y (Tensor) - The label Tensor which has the same shape as input_x. The data type must be float32.
- Outputs:
Tensor or Scalar, if reduction is ‘none’, then output is a tensor and has the same shape as input_x. Otherwise it is a scalar.
- Supported Platforms:
GPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.kldiv_loss = ops.KLDivLoss() ... def construct(self, x, y): ... result = self.kldiv_loss(x, y) ... return result ... >>> net = Net() >>> input_x = Tensor(np.array([0.2, 0.7, 0.1]), mindspore.float32) >>> input_y = Tensor(np.array([0., 1., 0.]), mindspore.float32) >>> output = net(input_x, input_y) >>> print(output) -0.23333333
-
class
tinyms.primitives.L2Loss(*args, **kwargs)[source]¶ Calculates half of the L2 norm of a tensor without using the sqrt.
Set input_x as x and output as loss.
\[loss = sum(x ** 2) / 2\]- Inputs:
input_x (Tensor) - A input Tensor. Data type must be float16 or float32.
- Outputs:
Tensor, has the same dtype as input_x. The output tensor is the value of loss which is a scalar tensor.
- Supported Platforms:
AscendGPU- Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.float16) >>> l2_loss = ops.L2Loss() >>> output = l2_loss(input_x) >>> print(output) 7.0
-
class
tinyms.primitives.L2Normalize(*args, **kwargs)[source]¶ L2 normalization Operator.
This operator will normalize the input using the given axis. The function is shown as follows:
\[\text{output} = \frac{x}{\sqrt{\text{max}(\text{sum} (\text{input_x}^2), \epsilon)}},\]where \(\epsilon\) is epsilon.
- Parameters
- Inputs:
input_x (Tensor) - Input to compute the normalization. Data type must be float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input.
- Supported Platforms:
Ascend
Examples
>>> l2_normalize = ops.L2Normalize() >>> input_x = Tensor(np.random.randint(-256, 256, (2, 3, 4)), mindspore.float32) >>> output = l2_normalize(input_x) >>> print(output.shape) (2, 3, 4)
-
class
tinyms.primitives.LARSUpdate(*args, **kwargs)[source]¶ Conducts LARS (layer-wise adaptive rate scaling) update on the sum of squares of gradient.
- Parameters
- Inputs:
weight (Tensor) - The weight to be updated.
gradient (Tensor) - The gradient of weight, which has the same shape and dtype with weight.
norm_weight (Tensor) - A scalar tensor, representing the sum of squares of weight.
norm_gradient (Tensor) - A scalar tensor, representing the sum of squares of gradient.
weight_decay (Union[Number, Tensor]) - Weight decay. It must be a scalar tensor or number.
learning_rate (Union[Number, Tensor]) - Learning rate. It must be a scalar tensor or number.
- Outputs:
Tensor, represents the new gradient.
- Supported Platforms:
Ascend
Examples
>>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> import mindspore.nn as nn >>> import numpy as np >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.lars = ops.LARSUpdate() ... self.reduce = ops.ReduceSum() ... self.square = ops.Square() ... def construct(self, weight, gradient): ... w_square_sum = self.reduce(self.square(weight)) ... grad_square_sum = self.reduce(self.square(gradient)) ... grad_t = self.lars(weight, gradient, w_square_sum, grad_square_sum, 0.0, 1.0) ... return grad_t ... >>> np.random.seed(0) >>> weight = np.random.random(size=(2, 3)).astype(np.float32) >>> gradient = np.random.random(size=(2, 3)).astype(np.float32) >>> net = Net() >>> output = net(Tensor(weight), Tensor(gradient)) >>> print(output) [[0.00036534 0.00074454 0.00080456] [0.00032014 0.00066101 0.00044157]]
-
class
tinyms.primitives.LRN(*args, **kwargs)[source]¶ Local Response Normalization.
\[b_{c} = a_{c}\left(k + \frac{\alpha}{n} \sum_{c'=\max(0, c-n/2)}^{\min(N-1,c+n/2)}a_{c'}^2\right)^{-\beta}\]- Parameters
depth_radius (int) – Half-width of the 1-D normalization window with the shape of 0-D.
bias (float) – An offset (usually positive to avoid dividing by 0).
alpha (float) – A scale factor, usually positive.
beta (float) – An exponent.
norm_region (str) – Specifies normalization region. Options: “ACROSS_CHANNELS”. Default: “ACROSS_CHANNELS”.
- Inputs:
x (Tensor) - A 4D Tensor with float16 or float32 data type.
- Outputs:
Tensor, with the same shape and data type as the input tensor.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.random.rand(1, 2, 2, 2), mindspore.float32) >>> lrn = ops.LRN() >>> output = lrn(x) >>> print(output.shape) (1, 2, 2, 2)
-
class
tinyms.primitives.LSTM(*args, **kwargs)[source]¶ Performs the Long Short-Term Memory (LSTM) on the input.
For detailed information, please refer to nn.LSTM.
- Supported Platforms:
GPUCPU
Examples
>>> input_size = 10 >>> hidden_size = 2 >>> num_layers = 1 >>> seq_len = 5 >>> batch_size = 2 >>> >>> net = P.LSTM(input_size, hidden_size, num_layers, True, False, 0.0) >>> input = Tensor(np.ones([seq_len, batch_size, input_size]).astype(np.float32)) >>> h0 = Tensor(np.ones([num_layers, batch_size, hidden_size]).astype(np.float32)) >>> c0 = Tensor(np.ones([num_layers, batch_size, hidden_size]).astype(np.float32)) >>> w = Tensor(np.ones([112, 1, 1]).astype(np.float32)) >>> output, hn, cn, _, _ = net(input, h0, c0, w) >>> print(output) [[[0.9640267 0.9640267 ] [0.9640267 0.9640267 ]] [[0.9950539 0.9950539 ] [0.9950539 0.9950539 ]] [[0.99932843 0.99932843] [0.99932843 0.99932843]] [[0.9999084 0.9999084 ] [0.9999084 0.9999084 ]] [[0.9999869 0.9999869 ] [0.9999869 0.9999869 ]]]
-
class
tinyms.primitives.LayerNorm(*args, **kwargs)[source]¶ Applies the Layer Normalization to the input tensor.
This operator will normalize the input tensor on given axis. LayerNorm is described in the paper Layer Normalization.
\[y = \frac{x - mean}{\sqrt{variance + \epsilon}} * \gamma + \beta\]where \(\gamma\) is scale, \(\beta\) is bias, \(\epsilon\) is epsilon.
- Parameters
begin_norm_axis (int) – The begin axis of the input_x to apply LayerNorm, the value must be in [-1, rank(input)). Default: 1.
begin_params_axis (int) – The begin axis of the parameter input (gamma, beta) to apply LayerNorm, the value must be in [-1, rank(input)). Default: 1.
epsilon (float) – A value added to the denominator for numerical stability. Default: 1e-7.
- Inputs:
input_x (Tensor) - Tensor of shape \((N, \ldots)\). The input of LayerNorm.
gamma (Tensor) - Tensor of shape \((P_0, \ldots, P_\text{begin_params_axis})\). The learnable parameter gamma as the scale on norm.
beta (Tensor) - Tensor of shape \((P_0, \ldots, P_\text{begin_params_axis})\). The learnable parameter beta as the scale on norm.
- Outputs:
tuple[Tensor], tuple of 3 tensors, the normalized input and the updated parameters.
output_x (Tensor) - The normalized input, has the same type and shape as the input_x. The shape is \((N, C)\).
mean (Tensor) - Tensor of shape \((C,)\).
variance (Tensor) - Tensor of shape \((C,)\).
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[1, 2, 3], [1, 2, 3]]), mindspore.float32) >>> gamma = Tensor(np.ones([3]), mindspore.float32) >>> beta = Tensor(np.ones([3]), mindspore.float32) >>> layer_norm = ops.LayerNorm() >>> output, mean, variance = layer_norm(input_x, gamma, beta) >>> print(output) [[-0.2247448 1. 2.2247448] [-0.2247448 1. 2.2247448]] >>> print(mean) [[2.] [2.]] >>> print(variance) [[0.6666667] [0.6666667]])
-
class
tinyms.primitives.Less(*args, **kwargs)[source]¶ Computes the boolean value of \(x < y\) element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is bool.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 1, 4]), mindspore.int32) >>> less = ops.Less() >>> output = less(input_x, input_y) >>> print(output) [False False True]
-
class
tinyms.primitives.LessEqual(*args, **kwargs)[source]¶ Computes the boolean value of \(x <= y\) element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool , and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is bool.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 1, 4]), mindspore.int32) >>> less_equal = ops.LessEqual() >>> output = less_equal(input_x, input_y) >>> print(output) [ True False True]
-
class
tinyms.primitives.LinSpace(*args, **kwargs)[source]¶ Generates values in an interval (inclusive of start and stop) and returns the corresponding interpolated array with num number of ticks.
- Inputs:
start (Tensor[float32]) - Start value of interval, With shape of 0-D.
stop (Tensor[float32]) - Last value of interval, With shape of 0-D.
num (int) - Number of ticks in the interval, inclusive of start and stop.
- Outputs:
Tensor, has the same shape as start.
- Supported Platforms:
AscendGPU
Examples
>>> linspace = P.LinSpace() >>> start = Tensor(1, mindspore.float32) >>> stop = Tensor(10, mindspore.float32) >>> num = 5 >>> output = linspace(start, stop, num) >>> print(output) [ 1. 3.25 5.5 7.75 10. ]
-
class
tinyms.primitives.Log(*args, **kwargs)[source]¶ Returns the natural logarithm of a tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor. With float16 or float32 data type. The value must be greater than 0.
- Outputs:
Tensor, has the same shape as the input_x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) >>> log = ops.Log() >>> output = log(input_x) >>> print(output) [0. 0.6931472 1.3862944]
-
class
tinyms.primitives.Log1p(*args, **kwargs)[source]¶ Returns the natural logarithm of one plus the input tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor. With float16 or float32 data type. The value must be greater than -1.
- Outputs:
Tensor, has the same shape as the input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) >>> log1p = ops.Log1p() >>> output = log1p(input_x) >>> print(output) [0.6931472 1.0986123 1.609438 ]
-
class
tinyms.primitives.LogSoftmax(*args, **kwargs)[source]¶ Log Softmax activation function.
Applies the Log Softmax function to the input tensor on the specified axis. Suppose a slice in the given aixs, \(x\) for each element \(x_i\), the Log Softmax function is shown as follows:
\[\text{output}(x_i) = \log \left(\frac{\exp(x_i)} {\sum_{j = 0}^{N-1}\exp(x_j)}\right),\]where \(N\) is the length of the Tensor.
- Parameters
axis (int) – The axis to perform the Log softmax operation. Default: -1.
- Inputs:
logits (Tensor) - The input of Log Softmax, with float16 or float32 data type.
- Outputs:
Tensor, with the same type and shape as the logits.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3, 4, 5]), mindspore.float32) >>> log_softmax = ops.LogSoftmax() >>> output = log_softmax(input_x) >>> print(output) [-4.4519143 -3.4519143 -2.4519143 -1.4519144 -0.4519144]
-
class
tinyms.primitives.LogUniformCandidateSampler(*args, **kwargs)[source]¶ Generates random labels with a log-uniform distribution for sampled_candidates.
Random sampling a tensor of sampled classes from the range of integers [0, range_max).
- Parameters
num_true (int) – The number of target classes per training example. Default: 1.
num_sampled (int) – The number of classes to randomly sample. Default: 5.
unique (bool) – Determines whether sample with rejection. If unique is True, all sampled classes in a batch are unique. Default: True.
range_max (int) – The number of possible classes. When unique is True, range_max must be greater than or equal to num_sampled. Default: 5.
seed (int) – Random seed, must be non-negative.
- Inputs:
true_classes (Tensor) - The target classes. With data type of int64 and shape [batch_size, num_true].
- Outputs:
Tuple of 3 Tensors.
sampled_candidates (Tensor) - A Tensor with shape (num_sampled,) and the same type as true_classes.
true_expected_count (Tensor) - A Tensor with the same shape as true_classes and type float32.
sampled_expected_count (Tensor) - A Tensor with the same shape as sampled_candidates and type float32.
- Supported Platforms:
Ascend
Examples
>>> sampler = ops.LogUniformCandidateSampler(2, 5, True, 5) >>> output1, output2, output3 = sampler(Tensor(np.array([[1, 7], [0, 4], [3, 3]]))) >>> print(output1, output2, output3) [3 2 0 4 1] [[0.92312991 0.49336370] [0.99248987 0.65806371] [0.73553443 0.73553443]] [0.73553443 0.82625800 0.99248987 0.65806371 0.92312991]
-
class
tinyms.primitives.LogicalAnd(*args, **kwargs)[source]¶ Computes the “logical AND” of two tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one bool. When the inputs are two tensors, the shapes of them could be broadcast, and the data types of them must be bool. When the inputs are one tensor and one bool, the bool object could only be a constant, and the data type of the tensor must be bool.
- Inputs:
input_x (Union[Tensor, bool]) - The first input is a bool or a tensor whose data type is bool.
input_y (Union[Tensor, bool]) - The second input is a bool when the first input is a tensor or a tensor whose data type is bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is bool.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([True, False, True]), mindspore.bool_) >>> input_y = Tensor(np.array([True, True, False]), mindspore.bool_) >>> logical_and = ops.LogicalAnd() >>> output = logical_and(input_x, input_y) >>> print(output) [ True False False]
-
class
tinyms.primitives.LogicalNot(*args, **kwargs)[source]¶ Computes the “logical NOT” of a tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor whose dtype is bool.
- Outputs:
Tensor, the shape is the same as the input_x, and the dtype is bool.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([True, False, True]), mindspore.bool_) >>> logical_not = ops.LogicalNot() >>> output = logical_not(input_x) >>> print(output) [False True False]
-
class
tinyms.primitives.LogicalOr(*args, **kwargs)[source]¶ Computes the “logical OR” of two tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one bool. When the inputs are two tensors, the shapes of them could be broadcast, and the data types of them must be bool. When the inputs are one tensor and one bool, the bool object could only be a constant, and the data type of the tensor must be bool.
- Inputs:
input_x (Union[Tensor, bool]) - The first input is a bool or a tensor whose data type is bool.
input_y (Union[Tensor, bool]) - The second input is a bool when the first input is a tensor or a tensor whose data type is bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is bool.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([True, False, True]), mindspore.bool_) >>> input_y = Tensor(np.array([True, True, False]), mindspore.bool_) >>> logical_or = ops.LogicalOr() >>> output = logical_or(input_x, input_y) >>> print(output) [ True True True]
-
class
tinyms.primitives.MakeRefKey(*args, **kwargs)[source]¶ Makes a RefKey instance by string. RefKey stores the name of Parameter, can be passed through the functions, and used for Assign target.
- Parameters
tag (str) – Parameter name to make the RefKey.
- Inputs:
No inputs.
- Outputs:
RefKeyType, made from the Parameter name.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import numpy as np >>> from mindspore import Parameter, Tensor >>> from mindspore import dtype as mstype >>> import mindspore.ops as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.y = Parameter(Tensor(np.ones([2, 3]), mstype.int32), name="y") ... self.make_ref_key = ops.MakeRefKey("y") ... ... def construct(self, x): ... key = self.make_ref_key() ... ref = ops.make_ref(key, x, self.y) ... return ref * x ... >>> x = Tensor(np.array([[1, 2, 3], [4, 5, 6]]), mindspore.int32) >>> net = Net() >>> output = net(x) >>> print(output) [[ 1 4 9] [16 25 36]]
-
class
tinyms.primitives.MatMul(*args, **kwargs)[source]¶ Multiplies matrix a and matrix b.
The rank of input tensors must equal to 2.
- Parameters
- Inputs:
input_x (Tensor) - The first tensor to be multiplied. The shape of the tensor is \((N, C)\). If transpose_a is True, its shape must be \((N, C)\) after transpose.
input_y (Tensor) - The second tensor to be multiplied. The shape of the tensor is \((C, M)\). If transpose_b is True, its shape must be \((C, M)\) after transpose.
- Outputs:
Tensor, the shape of the output tensor is \((N, M)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x1 = Tensor(np.ones(shape=[1, 3]), mindspore.float32) >>> input_x2 = Tensor(np.ones(shape=[3, 4]), mindspore.float32) >>> matmul = ops.MatMul() >>> output = matmul(input_x1, input_x2) >>> print(output) [[3. 3. 3. 3.]]
-
class
tinyms.primitives.MaxPool(*args, **kwargs)[source]¶ Max pooling operation.
Applies a 2D max pooling over an input Tensor which can be regarded as a composition of 2D planes.
Typically the input is of shape \((N_{in}, C_{in}, H_{in}, W_{in})\), MaxPool outputs regional maximum in the \((H_{in}, W_{in})\)-dimension. Given kernel size \(ks = (h_{ker}, w_{ker})\) and stride \(s = (s_0, s_1)\), the operation is as follows.
\[\text{output}(N_i, C_j, h, w) = \max_{m=0, \ldots, h_{ker}-1} \max_{n=0, \ldots, w_{ker}-1} \text{input}(N_i, C_j, s_0 \times h + m, s_1 \times w + n)\]- Parameters
kernel_size (Union[int, tuple[int]]) – The size of kernel used to take the maximum value, is an int number that represents height and width are both kernel_size, or a tuple of two int numbers that represent height and width respectively. Default: 1.
strides (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Default: 1.
pad_mode (str) – The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
format (str) –
The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: ‘NCHW’.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor, with shape \((N, C_{out}, H_{out}, W_{out})\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.arange(1 * 3 * 3 * 4).reshape((1, 3, 3, 4)), mindspore.float32) >>> maxpool_op = ops.MaxPool(pad_mode="VALID", kernel_size=2, strides=1) >>> output = maxpool_op(input_tensor) >>> print(output) [[[[ 5. 6. 7.] [ 9. 10. 11.]] [[17. 18. 19.] [21. 22. 23.]] [[29. 30. 31.] [33. 34. 35.]]]]
-
class
tinyms.primitives.MaxPoolWithArgmax(kernel_size=1, strides=1, pad_mode='valid', data_format='NCHW')[source]¶ Performs max pooling on the input Tensor and returns both max values and indices.
Typically the input is of shape \((N_{in}, C_{in}, H_{in}, W_{in})\), MaxPool outputs regional maximum in the \((H_{in}, W_{in})\)-dimension. Given kernel size \(ks = (h_{ker}, w_{ker})\) and stride \(s = (s_0, s_1)\), the operation is as follows.
\[\text{output}(N_i, C_j, h, w) = \max_{m=0, \ldots, h_{ker}-1} \max_{n=0, \ldots, w_{ker}-1} \text{input}(N_i, C_j, s_0 \times h + m, s_1 \times w + n)\]- Parameters
kernel_size (Union[int, tuple[int]]) – The size of kernel used to take the maximum value and arg value, is an int number that represents height and width are both kernel_size, or a tuple of two int numbers that represent height and width respectively. Default: 1.
strides (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Default: 1.
pad_mode (str) –
The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\). Data type must be float16 or float32.
- Outputs:
Tuple of 2 Tensors, representing the maxpool result and where the max values are generated.
output (Tensor) - Maxpooling result, with shape \((N, C_{out}, H_{out}, W_{out})\). It has the same data type as input.
mask (Tensor) - Max values’ index represented by the mask. Data type is int32.
- Raises
TypeError – If the input data type is not float16 or float32.
- Supported Platforms:
AscendGPU
Examples
>>> input_tensor = Tensor(np.arange(1 * 3 * 3 * 4).reshape((1, 3, 3, 4)), mindspore.float32) >>> maxpool_arg_op = ops.MaxPoolWithArgmax(pad_mode="VALID", kernel_size=2, strides=1) >>> output_tensor, argmax = maxpool_arg_op(input_tensor) >>> print(output_tensor) [[[[ 5. 6. 7.] [ 9. 10. 11.]] [[17. 18. 19.] [21. 22. 23.]] [[29. 30. 31.] [33. 34. 35.]]]]
-
class
tinyms.primitives.Maximum(*args, **kwargs)[source]¶ Computes the maximum of input tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 5.0, 3.0]), mindspore.float32) >>> input_y = Tensor(np.array([4.0, 2.0, 6.0]), mindspore.float32) >>> maximum = ops.Maximum() >>> output = maximum(input_x, input_y) >>> print(output) [4. 5. 6.]
-
class
tinyms.primitives.Merge(*args, **kwargs)[source]¶ Merges all input data to one.
One and only one of the inputs must be selected as the output
- Inputs:
inputs (Union(Tuple, List)) - The data to be merged. All tuple elements must have the same data type.
- Outputs:
tuple. Output is tuple(data, output_index). The data has the same shape of inputs element.
Examples
>>> merge = ops.Merge() >>> input_x = Tensor(np.linspace(0, 8, 8).reshape(2, 4), mindspore.float32) >>> input_y = Tensor(np.random.randint(-4, 4, (2, 4)), mindspore.float32) >>> result = merge((input_x, input_y))
-
class
tinyms.primitives.Meshgrid(*args, **kwargs)[source]¶ Generates coordinate matrices from given coordinate tensors.
Given N one-dimensional coordinate tensors, returns a tuple outputs of N N-D coordinate tensors for evaluating expressions on an N-D grid.
- Parameters
indexing (str) – Either ‘xy’ or ‘ij’. Default: ‘xy’. When the indexing argument is set to ‘xy’ (the default), the broadcasting instructions for the first two dimensions are swapped.
- Inputs:
input (Union[tuple]) - A Tuple of N 1-D Tensor objects. The length of input should be greater than 1
- Outputs:
Tensors, A Tuple of N N-D Tensor objects.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.array([1, 2, 3, 4]).astype(np.int32)) >>> y = Tensor(np.array([5, 6, 7]).astype(np.int32)) >>> z = Tensor(np.array([8, 9, 0, 1, 2]).astype(np.int32)) >>> inputs = (x, y, z) >>> meshgrid = ops.Meshgrid(indexing="xy") >>> output = meshgrid(inputs) >>> print(output) (Tensor(shape=[3, 4, 5], dtype=Int32, value= [[[1, 1, 1, 1, 1], [2, 2, 2, 2, 2], [3, 3, 3, 3, 3], [4, 4, 4, 4, 4]], [[1, 1, 1, 1, 1], [2, 2, 2, 2, 2], [3, 3, 3, 3, 3], [4, 4, 4, 4, 4]], [[1, 1, 1, 1, 1], [2, 2, 2, 2, 2], [3, 3, 3, 3, 3], [4, 4, 4, 4, 4]]]), Tensor(shape=[3, 4, 5], dtype=Int32, value= [[[5, 5, 5, 5, 5], [5, 5, 5, 5, 5], [5, 5, 5, 5, 5], [5, 5, 5, 5, 5]], [[6, 6, 6, 6, 6], [6, 6, 6, 6, 6], [6, 6, 6, 6, 6], [6, 6, 6, 6, 6]], [[7, 7, 7, 7, 7], [7, 7, 7, 7, 7], [7, 7, 7, 7, 7], [7, 7, 7, 7, 7]]]), Tensor(shape=[3, 4, 5], dtype=Int32, value= [[[8, 9, 0, 1, 2], [8, 9, 0, 1, 2], [8, 9, 0, 1, 2], [8, 9, 0, 1, 2]], [[8, 9, 0, 1, 2], [8, 9, 0, 1, 2], [8, 9, 0, 1, 2], [8, 9, 0, 1, 2]], [[8, 9, 0, 1, 2], [8, 9, 0, 1, 2], [8, 9, 0, 1, 2], [8, 9, 0, 1, 2]]]))
-
class
tinyms.primitives.Minimum(*args, **kwargs)[source]¶ Computes the minimum of input tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([1.0, 5.0, 3.0]), mindspore.float32) >>> input_y = Tensor(np.array([4.0, 2.0, 6.0]), mindspore.float32) >>> minimum = ops.Minimum() >>> output = minimum(input_x, input_y) >>> print(output) [1. 2. 3.]
-
class
tinyms.primitives.MirrorPad(*args, **kwargs)[source]¶ Pads the input tensor according to the paddings and mode.
- Parameters
mode (str) – Specifies the padding mode. The optional values are “REFLECT” and “SYMMETRIC”. Default: “REFLECT”.
- Inputs:
input_x (Tensor) - The input tensor.
paddings (Tensor) - The paddings tensor. The value of paddings is a matrix(list), and its shape is (N, 2). N is the rank of input data. All elements of paddings are int type. For the input in the D th dimension, paddings[D, 0] indicates how many sizes to be extended ahead of the input tensor in the D th dimension, and paddings[D, 1] indicates how many sizes to be extended behind the input tensor in the D th dimension.
- Outputs:
Tensor, the tensor after padding.
If mode is “REFLECT”, it uses a way of symmetrical copying through the axis of symmetry to fill in. If the input_x is [[1,2,3], [4,5,6], [7,8,9]] and paddings is [[1,1], [2,2]], then the Outputs is [[6,5,4,5,6,5,4], [3,2,1,2,3,2,1], [6,5,4,5,6,5,4], [9,8,7,8,9,8,7], [6,5,4,5,6,5,4]].
If mode is “SYMMETRIC”, the filling method is similar to the “REFLECT”. It is also copied according to the symmetry axis, except that it includes the symmetry axis. If the input_x is [[1,2,3], [4,5,6], [7,8,9]] and paddings is [[1,1], [2,2]], then the Outputs is [[2,1,1,2,3,3,2], [2,1,1,2,3,3,2], [5,4,4,5,6,6,5], [8,7,7,8,9,9,8], [8,7,7,8,9,9,8]].
- Supported Platforms:
AscendGPU
Examples
>>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> import mindspore.nn as nn >>> import numpy as np >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.pad = ops.MirrorPad(mode="REFLECT") ... def construct(self, x, paddings): ... return self.pad(x, paddings) ... >>> x = np.random.random(size=(2, 3)).astype(np.float32) >>> paddings = Tensor([[1, 1], [2, 2]]) >>> pad = Net() >>> output = pad(Tensor(x), paddings) >>> print(output.shape) (4, 7)
-
class
tinyms.primitives.Mod(*args, **kwargs)[source]¶ Computes the remainder of dividing the first input tensor by the second input tensor element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, both dtypes cannot be bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number]) - The first input is a number or a tensor whose data type is number.
input_y (Union[Tensor, Number]) - When the first input is a tensor, The second input could be a number or a tensor whose data type is number. When the first input is a number, the second input must be a tensor whose data type is number.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
ValueError – When input_x and input_y are not the same dtype.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([-4.0, 5.0, 6.0]), mindspore.float32) >>> input_y = Tensor(np.array([3.0, 2.0, 3.0]), mindspore.float32) >>> mod = ops.Mod() >>> output = mod(input_x, input_y) >>> print(output) [-1. 1. 0.]
-
class
tinyms.primitives.Mul(*args, **kwargs)[source]¶ Multiplies two tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
\[out_{i} = x_{i} * y_{i}\]- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) >>> input_y = Tensor(np.array([4.0, 5.0, 6.0]), mindspore.float32) >>> mul = ops.Mul() >>> output = mul(input_x, input_y) >>> print(output) [ 4. 10. 18.]
-
class
tinyms.primitives.Multinomial(*args, **kwargs)[source]¶ Returns a tensor sampled from the multinomial probability distribution located in the corresponding row of tensor input.
Note
The rows of input do not need to sum to one (in which case we use the values as weights), but must be non-negative, finite and have a non-zero sum.
- Parameters
- Inputs:
input (Tensor[float32]) - the input tensor containing the cumsum of probabilities, must be 1 or 2 dimensions.
num_samples (int32) - number of samples to draw.
- Outputs:
Tensor with the same rows as input, each row has num_samples sampled indices.
- Supported Platforms:
GPU
Examples
>>> input = Tensor([0., 9., 4., 0.], mstype.float32) >>> multinomial = ops.Multinomial(seed=10) >>> output = multinomial(input, 2) >>> print(output) [2 1]
-
class
tinyms.primitives.NMSWithMask(*args, **kwargs)[source]¶ Selects some bounding boxes in descending order of score.
- Parameters
iou_threshold (float) – Specifies the threshold of overlap boxes with respect to IOU. Default: 0.5.
- Raises
ValueError – If the iou_threshold is not a float number, or if the first dimension of input Tensor is less than or equal to 0, or if the data type of the input Tensor is not float16 or float32.
- Inputs:
bboxes (Tensor) - The shape of tensor is \((N, 5)\). Input bounding boxes. N is the number of input bounding boxes. Every bounding box contains 5 values, the first 4 values are the coordinates of bounding box, and the last value is the score of this bounding box. The data type must be float16 or float32.
- Outputs:
tuple[Tensor], tuple of three tensors, they are selected_boxes, selected_idx and selected_mask.
selected_boxes (Tensor) - The shape of tensor is \((N, 5)\). The list of bounding boxes after non-max suppression calculation.
selected_idx (Tensor) - The shape of tensor is \((N,)\). The indexes list of valid input bounding boxes.
selected_mask (Tensor) - The shape of tensor is \((N,)\). A mask list of valid output bounding boxes.
- Supported Platforms:
AscendGPU
Examples
>>> bbox = np.array([[0.4, 0.2, 0.4, 0.3, 0.1], [0.4, 0.3, 0.6, 0.8, 0.7]]) >>> bbox[:, 2] += bbox[:, 0] >>> bbox[:, 3] += bbox[:, 1] >>> inputs = Tensor(bbox, mindspore.float32) >>> nms = ops.NMSWithMask(0.5) >>> output_boxes, indices, mask = nms(inputs) >>> indices_np = indices.asnumpy() >>> print(indices_np[mask.asnumpy()]) [0 1]
-
class
tinyms.primitives.NPUAllocFloatStatus(*args, **kwargs)[source]¶ Allocates a flag to store the overflow status.
The flag is a tensor whose shape is (8,) and data type is mindspore.dtype.float32.
Note
Examples: see NPUGetFloatStatus.
- Outputs:
Tensor, has the shape of (8,).
- Supported Platforms:
Ascend
Examples
>>> alloc_status = ops.NPUAllocFloatStatus() >>> output = alloc_status() >>> print(output) [0. 0. 0. 0. 0. 0. 0. 0.]
-
class
tinyms.primitives.NPUClearFloatStatus(*args, **kwargs)[source]¶ Clears the flag which stores the overflow status.
Note
The flag is in the register on the Ascend device. It will be reset and can not be reused again after the NPUClearFloatStatus is called.
Examples: see NPUGetFloatStatus.
- Inputs:
input_x (Tensor) - The output tensor of NPUAllocFloatStatus. The data type must be float16 or float32.
- Outputs:
Tensor, has the same shape as input_x. All the elements in the tensor will be zero.
- Supported Platforms:
Ascend
Examples
>>> alloc_status = ops.NPUAllocFloatStatus() >>> get_status = ops.NPUGetFloatStatus() >>> clear_status = ops.NPUClearFloatStatus() >>> init = alloc_status() >>> flag = get_status(init) >>> clear_status(init) Tensor(shape=[8], dtype=Float32, value= [ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00]) >>> print(init) [1. 1. 1. 1. 1. 1. 1. 1.]
-
class
tinyms.primitives.NPUGetFloatStatus(*args, **kwargs)[source]¶ Updates the flag which is the output tensor of NPUAllocFloatStatus with the latest overflow status.
The flag is a tensor whose shape is (8,) and data type is mindspore.dtype.float32. If the sum of the flag equals to 0, there is no overflow happened. If the sum of the flag is bigger than 0, there is overflow happened.
- Inputs:
input_x (Tensor) - The output tensor of NPUAllocFloatStatus. The data type must be float16 or float32.
- Outputs:
Tensor, has the same shape as input_x. All the elements in the tensor will be zero.
- Supported Platforms:
Ascend
Examples
>>> alloc_status = ops.NPUAllocFloatStatus() >>> get_status = ops.NPUGetFloatStatus() >>> init = alloc_status() >>> get_status(init) Tensor(shape=[8], dtype=Float32, value= [ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00]) >>> print(init) [1. 1. 1. 1. 1. 1. 1. 1.]
-
class
tinyms.primitives.Neg(*args, **kwargs)[source]¶ Returns a tensor with negative values of the input tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor whose dtype is number.
- Outputs:
Tensor, has the same shape and dtype as input.
- Supported Platforms:
AscendGPUCPU
Examples
>>> neg = ops.Neg() >>> input_x = Tensor(np.array([1, 2, -1, 2, 0, -3.5]), mindspore.float32) >>> output = neg(input_x) >>> print(output) [-1. -2. 1. -2. 0. 3.5]
-
class
tinyms.primitives.NoRepeatNGram(*args, **kwargs)[source]¶ Update log_probs with repeat n-grams.
- Parameters
ngram_size (int) – Size of n-grams, must be greater than 0. Default: 1.
- Inputs:
state_seq (Tensor) - A 3-D tensor with shape: (batch_size, beam_width, m).
log_probs (Tensor) - A 3-D tensor with shape: (batch_size, beam_width, vocab_size). The value of log_probs will be replaced with -FLOAT_MAX when n-grams repeated.
- Outputs:
log_probs (Tensor) - The output Tensor with same shape and type as original log_probs.
- Supported Platforms:
Ascend
Examples
>>> no_repeat_ngram = ops.NoRepeatNGram(ngram_size=3) >>> state_seq = Tensor([[[1, 2, 1, 2, 5, 1, 2], [9, 3, 9, 5, 4, 1, 5]], [[4, 8, 6, 4, 5, 6, 4], [4, 8, 8, 4, 3, 4, 8]]], dtype=mindspore.int32) >>> log_probs = Tensor([[[0.75858542, 0.8437121 , 0.69025469, 0.79379992, 0.27400691, 0.84709179, 0.78771346, 0.68587179, 0.22943851, 0.17682976]], [[0.99401879, 0.77239773, 0.81973878, 0.32085208, 0.59944118, 0.3125177, 0.52604189, 0.77111461, 0.98443699, 0.71532898]]], dtype=mindspore.float32) >>> output = no_repeat_ngram(state_seq, log_probs) >>> print(output) [[[0.75858542 -3.4028235e+38 0.69025469 0.79379992 0.27400691 -3.4028235e+38 0.78771346 0.68587179 0.22943851 0.17682976]] [[0.99401879 0.77239773 0.81973878 0.32085208 0.59944118 -3.4028235e+38 0.52604189 0.77111461 0.98443699 0.71532898]]]
-
class
tinyms.primitives.NotEqual(*args, **kwargs)[source]¶ Computes the non-equivalence of two tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is bool.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> not_equal = ops.NotEqual() >>> output = not_equal(input_x, 2.0) >>> print(output) [ True False True] >>> >>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 2, 4]), mindspore.int32) >>> not_equal = ops.NotEqual() >>> output = not_equal(input_x, input_y) >>> print(output) [False False True]
-
class
tinyms.primitives.OneHot(*args, **kwargs)[source]¶ Computes a one-hot tensor.
Makes a new tensor, whose locations represented by indices in indices take value on_value, while all other locations take value off_value.
Note
If the input indices is rank N, the output will have rank N+1. The new axis is created at dimension axis.
- Parameters
axis (int) – Position to insert the value. e.g. If indices shape is [n, c], and axis is -1 the output shape will be [n, c, depth], If axis is 0 the output shape will be [depth, n, c]. Default: -1.
- Inputs:
indices (Tensor) - A tensor of indices. Tensor of shape \((X_0, \ldots, X_n)\). Data type must be int32.
depth (int) - A scalar defining the depth of the one hot dimension.
on_value (Tensor) - A value to fill in output when indices[j] = i. With data type of float16 or float32.
off_value (Tensor) - A value to fill in output when indices[j] != i. Has the same data type with as on_value.
- Outputs:
Tensor, one-hot tensor. Tensor of shape \((X_0, \ldots, X_{axis}, \text{depth} ,X_{axis+1}, \ldots, X_n)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> indices = Tensor(np.array([0, 1, 2]), mindspore.int32) >>> depth, on_value, off_value = 3, Tensor(1.0, mindspore.float32), Tensor(0.0, mindspore.float32) >>> onehot = ops.OneHot() >>> output = onehot(indices, depth, on_value, off_value) >>> print(output) [[1. 0. 0.] [0. 1. 0.] [0. 0. 1.]]
-
class
tinyms.primitives.Ones(*args, **kwargs)[source]¶ Creates a tensor filled with value ones.
Creates a tensor with shape described by the first argument and fills it with value ones in type of the second argument.
- Inputs:
shape (Union[tuple[int], int]) - The specified shape of output tensor. Only constant positive int is allowed.
type (mindspore.dtype) - The specified type of output tensor. Only constant value is allowed.
- Outputs:
Tensor, has the same type and shape as input shape value.
- Supported Platforms:
AscendGPUCPU
Examples
>>> from mindspore.ops import operations as ops >>> ones = ops.Ones() >>> output = ones((2, 2), mindspore.float32) >>> print(output) [[1. 1.] [1. 1.]]
-
class
tinyms.primitives.OnesLike(*args, **kwargs)[source]¶ Creates a new tensor. The values of all elements are 1.
Returns a tensor of ones with the same shape and type as the input.
- Inputs:
input_x (Tensor) - Input tensor.
- Outputs:
Tensor, has the same shape and type as input_x but filled with ones.
- Supported Platforms:
AscendGPUCPU
Examples
>>> oneslike = ops.OnesLike() >>> x = Tensor(np.array([[0, 1], [2, 1]]).astype(np.int32)) >>> output = oneslike(x) >>> print(output) [[1 1] [1 1]]
-
class
tinyms.primitives.PReLU(*args, **kwargs)[source]¶ Parametric Rectified Linear Unit activation function.
PReLU is described in the paper Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. Defined as follows:
\[prelu(x_i)= \max(0, x_i) + \min(0, w * x_i),\]where \(x_i\) is an element of an channel of the input.
Note
1-dimensional input_x is not supported.
- Inputs:
input_x (Tensor) - Float tensor, representing the output of the preview layer. With data type of float16 or float32.
weight (Tensor) - Float Tensor, w > 0, there are only two shapes are legitimate, 1 or the number of channels of the input. With data type of float16 or float32.
- Outputs:
Tensor, with the same type as input_x.
For detailed information, please refer to nn.PReLU.
- Supported Platforms:
Ascend
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.prelu = ops.PReLU() ... def construct(self, input_x, weight): ... result = self.prelu(input_x, weight) ... return result ... >>> input_x = Tensor(np.random.randint(-3, 3, (2, 3, 2)), mindspore.float32) >>> weight = Tensor(np.array([0.1, 0.6, -0.3]), mindspore.float32) >>> net = Net() >>> output = net(input_x, weight) >>> print(output.shape) (2, 3, 2)
-
tinyms.primitives.Pack(axis=0)[source]¶ Packs a list of tensors in specified axis.
The usage of Pack is deprecated. Please use Stack.
-
class
tinyms.primitives.Pad(*args, **kwargs)[source]¶ Pads the input tensor according to the paddings.
- Parameters
paddings (tuple) – The shape of parameter paddings is (N, 2). N is the rank of input data. All elements of paddings are int type. For the input in D th dimension, paddings[D, 0] indicates how many sizes to be extended ahead of the input tensor in the D th dimension, and paddings[D, 1] indicates how many sizes to be extended behind the input tensor in the D th dimension.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, the tensor after padding.
- Supported Platforms:
AscendGPU
Examples
>>> input_tensor = Tensor(np.array([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]), mindspore.float32) >>> pad_op = ops.Pad(((1, 2), (2, 1))) >>> output = pad_op(input_tensor) >>> print(output) [[ 0. 0. 0. 0. 0. 0. ] [ 0. 0. -0.1 0.3 3.6 0. ] [ 0. 0. 0.4 0.5 -3.2 0. ] [ 0. 0. 0. 0. 0. 0. ] [ 0. 0. 0. 0. 0. 0. ]]
-
class
tinyms.primitives.Padding(*args, **kwargs)[source]¶ Extends the last dimension of the input tensor from 1 to pad_dim_size, by filling with 0.
- Parameters
pad_dim_size (int) – The value of the last dimension of x to be extended, which must be positive.
- Inputs:
x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). The rank of x must be at least 2. The last dimension of x must be 1.
- Outputs:
Tensor, the shape of tensor is \((z_1, z_2, ..., z_N)\).
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.array([[8], [10]]), mindspore.float32) >>> pad_dim_size = 4 >>> output = ops.Padding(pad_dim_size)(x) >>> print(output) [[ 8. 0. 0. 0.] [10. 0. 0. 0.]]
-
class
tinyms.primitives.ParallelConcat(*args, **kwargs)[source]¶ Concats tensor in the first dimension.
Concats input tensors along with the first dimension.
Note
The input tensors are all required to have size 1 in the first dimension.
- Inputs:
values (tuple, list) - A tuple or a list of input tensors. The data type and shape of these tensors must be the same.
- Outputs:
Tensor, data type is the same as values.
- Supported Platforms:
Ascend
Examples
>>> data1 = Tensor(np.array([[0, 1]]).astype(np.int32)) >>> data2 = Tensor(np.array([[2, 1]]).astype(np.int32)) >>> op = ops.ParallelConcat() >>> output = op((data1, data2)) >>> print(output) [[0 1] [2 1]]
-
class
tinyms.primitives.Partial(*args, **kwargs)[source]¶ Makes a partial function instance, used for pynative mode.
- Inputs:
args (Union[FunctionType, Tensor]) - The function and bind arguments.
- Outputs:
FunctionType, partial function binded with arguments.
-
class
tinyms.primitives.Poisson(*args, **kwargs)[source]¶ Produces random non-negative integer values i, distributed according to discrete probability function:
\[\text{P}(i|μ) = \frac{\exp(-μ)μ^{i}}{i!},\]- Parameters
- Inputs:
shape (tuple) - The shape of random tensor to be generated. Only constant value is allowed.
mean (Tensor) - μ parameter the distribution was constructed with. The parameter defines mean number of occurrences of the event. It must be greater than 0. With float32 data type.
- Outputs:
Tensor. Its shape must be the broadcasted shape of shape and the shape of mean. The dtype is int32.
- Supported Platforms:
Ascend
Examples
>>> shape = (4, 1) >>> mean = Tensor(np.array([5.0, 10.0]), mstype.float32) >>> poisson = ops.Poisson(seed=5) >>> output = poisson(shape, mean) >>> result = output.shape >>> print(result) (4, 2)
-
class
tinyms.primitives.PopulationCount(*args, **kwargs)[source]¶ Calculates population count.
- Inputs:
input (Tensor) - The data type must be int16 or uint16.
- Outputs:
Tensor, with the same shape as the input.
- Supported Platforms:
Ascend
Examples
>>> population_count = ops.PopulationCount() >>> x_input = Tensor([0, 1, 3], mindspore.int16) >>> output = population_count(x_input) >>> print(output) [0 1 2]
-
class
tinyms.primitives.Pow(*args, **kwargs)[source]¶ Computes a tensor to the power of the second input.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) >>> input_y = 3.0 >>> pow = ops.Pow() >>> output = pow(input_x, input_y) >>> print(output) [ 1. 8. 64.] >>> >>> input_x = Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) >>> input_y = Tensor(np.array([2.0, 4.0, 3.0]), mindspore.float32) >>> pow = ops.Pow() >>> output = pow(input_x, input_y) >>> print(output) [ 1. 16. 64.]
-
class
tinyms.primitives.Print(*args, **kwargs)[source]¶ Outputs the tensor or string to stdout.
Note
In pynative mode, please use python print function.
- Inputs:
input_x (Union[Tensor, str]) - The graph node to attach to. The input supports multiple strings and tensors which are separated by ‘,’.
- Supported Platforms:
Ascend
Examples
>>> class PrintDemo(nn.Cell): ... def __init__(self): ... super(PrintDemo, self).__init__() ... self.print = ops.Print() ... ... def construct(self, x, y): ... self.print('Print Tensor x and Tensor y:', x, y) ... return x ... >>> x = Tensor(np.ones([2, 1]).astype(np.int32)) >>> y = Tensor(np.ones([2, 2]).astype(np.int32)) >>> net = PrintDemo() >>> result = net(x, y) Print Tensor x and Tensor y: [[1] [1]] [[1 1] [1 1]]
-
class
tinyms.primitives.Pull(*args, **kwargs)[source]¶ Pulls weight from parameter server.
- Inputs:
key (Tensor) - The key of the weight.
weight (Tensor) - The weight to be updated.
- Outputs:
None.
-
class
tinyms.primitives.Push(*args, **kwargs)[source]¶ Pushes the inputs of the corresponding optimizer to parameter server.
- Parameters
optim_type (string) – The optimizer type. Default: ‘ApplyMomentum’.
only_shape_indices (list) – The indices of input of which only shape will be pushed to parameter server. Default: None.
- Inputs:
optim_inputs (tuple) - The inputs for this kind of optimizer.
optim_input_shapes (tuple) - The shapes of the inputs.
- Outputs:
Tensor, the key of the weight which needs to be updated.
-
class
tinyms.primitives.RNNTLoss(*args, **kwargs)[source]¶ Computes the RNNTLoss and its gradient with respect to the softmax outputs.
- Parameters
blank_label (int) – blank label. Default: 0.
- Inputs:
acts (Tensor) - Tensor of shape \((B, T, U, V)\). Data type must be float16 or float32.
labels (Tensor[int32]) - Tensor of shape \((B, U-1)\).
input_lengths (Tensor[int32]) - Tensor of shape \((B,)\).
label_lengths (Tensor[int32]) - Tensor of shape \((B,)\).
- Outputs:
costs (Tensor[int32]) - Tensor of shape \((B,)\).
grads (Tensor[int32]) - Has the same shape as acts.
- Supported Platforms:
Ascend
Examples
>>> B, T, U, V = 1, 2, 3, 5 >>> blank = 0 >>> acts = np.random.random((B, T, U, V)).astype(np.float32) >>> labels = np.array([[1, 2]]).astype(np.int32) >>> input_length = np.array([T] * B).astype(np.int32) >>> label_length = np.array([len(l) for l in labels]).astype(np.int32) >>> rnnt_loss = ops.RNNTLoss(blank_label=0) >>> costs, grads = rnnt_loss(Tensor(acts), Tensor(labels), Tensor(input_length), Tensor(label_length)) >>> print(costs.shape) (1,) >>> print(grads.shape) (1, 2, 3, 5)
-
class
tinyms.primitives.ROIAlign(*args, **kwargs)[source]¶ Computes the Region of Interest (RoI) Align operator.
The operator computes the value of each sampling point by bilinear interpolation from the nearby grid points on the feature map. No quantization is performed on any coordinates involved in the RoI, its bins, or the sampling points. The details of (RoI) Align operator are described in Mask R-CNN.
- Parameters
pooled_height (int) – The output features height.
pooled_width (int) – The output features width.
spatial_scale (float) – A scaling factor that maps the raw image coordinates to the input feature map coordinates. Suppose the height of a RoI is ori_h in the raw image and fea_h in the input feature map, the spatial_scale must be fea_h / ori_h.
sample_num (int) – Number of sampling points. Default: 2.
roi_end_mode (int) – Number must be 0 or 1. Default: 1.
- Inputs:
features (Tensor) - The input features, whose shape must be (N, C, H, W).
rois (Tensor) - The shape is (rois_n, 5). With data type of float16 or float32. rois_n represents the number of RoI. The size of the second dimension must be 5 and the 5 colunms are (image_index, top_left_x, top_left_y, bottom_right_x, bottom_right_y). image_index represents the index of image. top_left_x and top_left_y represent the x, y coordinates of the top left corner of corresponding RoI, respectively. bottom_right_x and bottom_right_y represent the x, y coordinates of the bottom right corner of corresponding RoI, respectively.
- Outputs:
Tensor, the shape is (rois_n, C, pooled_height, pooled_width).
- Supported Platforms:
AscendGPU
Examples
>>> input_tensor = Tensor(np.array([[[[1., 2.], [3., 4.]]]]), mindspore.float32) >>> rois = Tensor(np.array([[0, 0.2, 0.3, 0.2, 0.3]]), mindspore.float32) >>> roi_align = ops.ROIAlign(2, 2, 0.5, 2) >>> output = roi_align(input_tensor, rois) >>> print(output) [[[[1.775 2.025] [2.275 2.525]]]]
-
class
tinyms.primitives.RandomCategorical(*args, **kwargs)[source]¶ Generates random samples from a given categorical distribution tensor.
- Parameters
dtype (mindspore.dtype) – The type of output. Its value must be one of mindspore.int16, mindspore.int32 and mindspore.int64. Default: mindspore.int64.
- Inputs:
logits (Tensor) - The input tensor. 2-D Tensor with shape [batch_size, num_classes].
num_sample (int) - Number of sample to be drawn. Only constant values is allowed.
seed (int) - Random seed. Default: 0. Only constant values is allowed.
- Outputs:
output (Tensor) - The output Tensor with shape [batch_size, num_samples].
- Supported Platforms:
AscendGPU
Examples
>>> class Net(nn.Cell): ... def __init__(self, num_sample): ... super(Net, self).__init__() ... self.random_categorical = ops.RandomCategorical(mindspore.int64) ... self.num_sample = num_sample ... def construct(self, logits, seed=0): ... return self.random_categorical(logits, self.num_sample, seed) ... >>> x = np.random.random((10, 5)).astype(np.float32) >>> net = Net(8) >>> output = net(Tensor(x)) >>> result = output.shape >>> print(result) (10, 8)
-
class
tinyms.primitives.RandomChoiceWithMask(*args, **kwargs)[source]¶ Generates a random sample as index tensor with a mask tensor from a given tensor.
The input must be a tensor of rank not less than 1. If its rank is greater than or equal to 2, the first dimension specifies the number of samples. The index tensor and the mask tensor have the fixed shapes. The index tensor denotes the index of the nonzero sample, while the mask tensor denotes which elements in the index tensor are valid.
- Parameters
- Inputs:
input_x (Tensor[bool]) - The input tensor. The input tensor rank must be greater than or equal to 1 and less than or equal to 5.
- Outputs:
Two tensors, the first one is the index tensor and the other one is the mask tensor.
index (Tensor) - The output shape is 2-D.
mask (Tensor) - The output shape is 1-D.
- Supported Platforms:
AscendGPU
Examples
>>> rnd_choice_mask = ops.RandomChoiceWithMask() >>> input_x = Tensor(np.ones(shape=[240000, 4]).astype(np.bool)) >>> output_y, output_mask = rnd_choice_mask(input_x) >>> result = output_y.shape >>> print(result) (256, 2) >>> result = output_mask.shape >>> print(result) (256,)
-
class
tinyms.primitives.Randperm(*args, **kwargs)[source]¶ Generates random samples from 0 to n-1.
- Parameters
- Inputs:
n (Tensor[int]) - The input tensor with shape: (1,) and the number must be in (0, max_length]. Default: 1.
- Outputs:
output (Tensor) - The output Tensor with shape: (max_length,) and type: dtype.
- Supported Platforms:
Ascend
Examples
>>> randperm = ops.Randperm(max_length=30, pad=-1) >>> n = Tensor([20], dtype=mindspore.int32) >>> output = randperm(n) >>> print(output) [15 6 11 19 14 16 9 5 13 18 4 10 8 0 17 2 14 1 12 3 7 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1]
-
class
tinyms.primitives.Rank(*args, **kwargs)[source]¶ Returns the rank of a tensor.
Returns a 0-D int32 Tensor representing the rank of input; the rank of a tensor is the number of indices required to uniquely select each element of the tensor.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor. 0-D int32 Tensor representing the rank of input, i.e., \(R\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) >>> rank = ops.Rank() >>> output = rank(input_tensor) >>> print(output) 2
-
class
tinyms.primitives.ReLU(*args, **kwargs)[source]¶ Computes ReLU (Rectified Linear Unit) of input tensors element-wise.
It returns \(\max(x,\ 0)\) element-wise.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, with the same type and shape as the input_x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> relu = ops.ReLU() >>> output = relu(input_x) >>> print(output) [[0. 4. 0.] [2. 0. 9.]]
-
class
tinyms.primitives.ReLU6(*args, **kwargs)[source]¶ Computes ReLU (Rectified Linear Unit) upper bounded by 6 of input tensors element-wise.
\[\text{ReLU6}(x) = \min(\max(0,x), 6)\]It returns \(\min(\max(0,x), 6)\) element-wise.
- Inputs:
input_x (Tensor) - The input tensor, with float16 or float32 data type.
- Outputs:
Tensor, with the same type and shape as the input_x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> relu6 = ops.ReLU6() >>> result = relu6(input_x) >>> print(result) [[0. 4. 0.] [2. 0. 6.]]
-
class
tinyms.primitives.ReLUV2(*args, **kwargs)[source]¶ Computes ReLU (Rectified Linear Unit) of input tensors element-wise.
It returns \(\max(x,\ 0)\) element-wise.
- Inputs:
input_x (Tensor) - The input tensor must be a 4-D tensor.
- Outputs:
output (Tensor) - Has the same type and shape as the input_x.
mask (Tensor) - A tensor whose data type must be uint8.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([[[[1, -2], [-3, 4]], [[-5, 6], [7, -8]]]]), mindspore.float32) >>> relu_v2 = ops.ReLUV2() >>> output, mask= relu_v2(input_x) >>> print(output) [[[[1. 0.] [0. 4.]] [[0. 6.] [7. 0.]]]] >>> print(mask) [[[[[1 0] [2 0]] [[2 0] [1 0]]]]]
-
class
tinyms.primitives.RealDiv(*args, **kwargs)[source]¶ Divides the first input tensor by the second input tensor in floating-point type element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) >>> input_y = Tensor(np.array([4.0, 5.0, 6.0]), mindspore.float32) >>> realdiv = ops.RealDiv() >>> output = realdiv(input_x, input_y) >>> print(output) [0.25 0.4 0.5 ]
-
class
tinyms.primitives.Reciprocal(*args, **kwargs)[source]¶ Returns reciprocal of a tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, has the same shape as the input_x.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) >>> reciprocal = ops.Reciprocal() >>> output = reciprocal(input_x) >>> print(output) [1. 0.5 0.25]
-
class
tinyms.primitives.ReduceAll(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by the “logicalAND” of all elements in the dimension.
The dtype of the tensor to be reduced is bool.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default : False, don’t keep these reduced dimensions.
- Inputs:
input_x (Tensor[bool]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, the dtype is bool.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the “logical and” of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[True, False], [True, True]])) >>> op = ops.ReduceAll(keep_dims=True) >>> output = op(input_x, 1) >>> print(output) [[False] [ True]]
-
class
tinyms.primitives.ReduceAny(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by the “logical OR” of all elements in the dimension.
The dtype of the tensor to be reduced is bool.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default : False, don’t keep these reduced dimensions.
- Inputs:
input_x (Tensor[bool]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, the dtype is bool.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the “logical or” of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[True, False], [True, True]])) >>> op = ops.ReduceAny(keep_dims=True) >>> output = op(input_x, 1) >>> print(output) [[ True] [ True]]
-
class
tinyms.primitives.ReduceMax(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by the maximum value in this dimension.
The dtype of the tensor to be reduced is number.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default : False, don’t keep these reduced dimensions.
- Inputs:
input_x (Tensor[Number]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, has the same dtype as the input_x.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the maximum of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.random.randn(3, 4, 5, 6).astype(np.float32)) >>> op = ops.ReduceMax(keep_dims=True) >>> output = op(input_x, 1) >>> result = output.shape >>> print(result) (3, 1, 5, 6)
-
class
tinyms.primitives.ReduceMean(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by averaging all elements in the dimension.
The dtype of the tensor to be reduced is number.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default: False.
- Inputs:
input_x (Tensor[Number]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, has the same dtype as the input_x.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the mean of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.random.randn(3, 4, 5, 6).astype(np.float32)) >>> op = ops.ReduceMean(keep_dims=True) >>> output = op(input_x, 1) >>> result = output.shape >>> print(result) (3, 1, 5, 6)
-
class
tinyms.primitives.ReduceMin(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by the minimum value in the dimension.
The dtype of the tensor to be reduced is number.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default : False, don’t keep these reduced dimensions.
- Inputs:
input_x (Tensor[Number]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, has the same dtype as the input_x.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the minimum of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.random.randn(3, 4, 5, 6).astype(np.float32)) >>> op = ops.ReduceMin(keep_dims=True) >>> output = op(input_x, 1) >>> result = output.shape >>> print(result) (3, 1, 5, 6)
-
class
tinyms.primitives.ReduceOp[source]¶ Operation options for reducing tensors.
There are four kinds of operation options, “SUM”, “MAX”, “MIN”, and “PROD”.
SUM: Take the sum.
MAX: Take the maximum.
MIN: Take the minimum.
PROD: Take the product.
- Supported Platforms:
AscendGPU
-
class
tinyms.primitives.ReduceProd(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by multiplying all elements in the dimension.
The dtype of the tensor to be reduced is number.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default : False, don’t keep these reduced dimensions.
- Inputs:
input_x (Tensor[Number]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, has the same dtype as the input_x.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the product of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.random.randn(3, 4, 5, 6).astype(np.float32)) >>> op = ops.ReduceProd(keep_dims=True) >>> output = op(input_x, 1) >>> result = output.shape >>> print(result) (3, 1, 5, 6)
-
class
tinyms.primitives.ReduceScatter(*args, **kwargs)[source]¶ Reduces and scatters tensors from the specified communication group.
Note
The back propagation of the op is not supported yet. Stay tuned for more. The tensors must have the same shape and format in all processes of the collection.
- Parameters
- Raises
TypeError – If any of operation and group is not a string.
ValueError – If the first dimension of the input cannot be divided by the rank size.
- Supported Platforms:
AscendGPU
Examples
>>> # This example should be run with two devices. Refer to the tutorial > Distributed Training on mindspore.cn. >>> from mindspore import Tensor, context >>> from mindspore.communication import init >>> from mindspore.ops.operations.comm_ops import ReduceOp >>> import mindspore.nn as nn >>> import mindspore.ops.operations as ops >>> import numpy as np >>> >>> context.set_context(mode=context.GRAPH_MODE) >>> init() >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.reducescatter = ops.ReduceScatter(ReduceOp.SUM) ... ... def construct(self, x): ... return self.reducescatter(x) ... >>> input_ = Tensor(np.ones([8, 8]).astype(np.float32)) >>> net = Net() >>> output = net(input_) >>> print(output) [[2. 2. 2. 2. 2. 2. 2. 2.] [2. 2. 2. 2. 2. 2. 2. 2.] [2. 2. 2. 2. 2. 2. 2. 2.] [2. 2. 2. 2. 2. 2. 2. 2.]]
-
class
tinyms.primitives.ReduceSum(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by summing all elements in the dimension.
The dtype of the tensor to be reduced is number.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default: False.
- Inputs:
input_x (Tensor[Number]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, has the same dtype as the input_x.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the sum of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.random.randn(3, 4, 5, 6).astype(np.float32)) >>> op = ops.ReduceSum(keep_dims=True) >>> output = op(input_x, 1) >>> output.shape (3, 1, 5, 6)
-
class
tinyms.primitives.Reshape(*args, **kwargs)[source]¶ Reshapes the input tensor with the same values based on a given shape tuple.
- Raises
ValueError – Given a shape tuple, if it has several -1; or if the product of its elements is less than or equal to 0 or cannot be divided by the product of the input tensor shape; or if it does not match the input’s array size.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
input_shape (tuple[int]) - The input tuple is constructed by multiple integers, i.e., \((y_1, y_2, ..., y_S)\). Only constant value is allowed.
- Outputs:
Tensor, the shape of tensor is \((y_1, y_2, ..., y_S)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]), mindspore.float32) >>> reshape = ops.Reshape() >>> output = reshape(input_tensor, (3, 2)) >>> print(output) [[-0.1 0.3] [ 3.6 0.4] [ 0.5 -3.2]]
-
class
tinyms.primitives.ResizeBilinear(*args, **kwargs)[source]¶ Resizes an image to a certain size using the bilinear interpolation.
The resizing only affects the lower two dimensions which represent the height and width. The input images can be represented by different data types, but the data types of output images are always float32.
- Parameters
size (Union[tuple[int], list[int]]) – A tuple or list of 2 int elements (new_height, new_width), the new size of the images.
align_corners (bool) – If true, rescale input by (new_height - 1) / (height - 1), which exactly aligns the 4 corners of images and resized images. If false, rescale by new_height / height. Default: False.
- Inputs:
input (Tensor) - Image to be resized. Input images must be a 4-D tensor with shape \((batch, channels, height, width)\), with data type of float32 or float16.
- Outputs:
Tensor, resized image. 4-D with shape [batch, channels, new_height, new_width] in float32.
- Supported Platforms:
Ascend
Examples
>>> tensor = Tensor([[[[1, 2, 3, 4, 5], [1, 2, 3, 4, 5]]]], mindspore.float32) >>> resize_bilinear = ops.ResizeBilinear((5, 5)) >>> output = resize_bilinear(tensor) >>> print(output) [[[[1. 2. 3. 4. 5.] [1. 2. 3. 4. 5.] [1. 2. 3. 4. 5.] [1. 2. 3. 4. 5.] [1. 2. 3. 4. 5.]]]]
-
class
tinyms.primitives.ResizeNearestNeighbor(*args, **kwargs)[source]¶ Resizes the input tensor by using the nearest neighbor algorithm.
Resizes the input tensor to a given size by using the nearest neighbor algorithm. The nearest neighbor algorithm selects the value of the nearest point and does not consider the values of neighboring points at all, yielding a piecewise-constant interpolant.
- Parameters
- Inputs:
input_x (Tensor) - The input tensor. The shape of the tensor is \((N, C, H, W)\).
- Outputs:
Tensor, the shape of the output tensor is \((N, C, NEW\_H, NEW\_W)\).
- Supported Platforms:
AscendGPU
Examples
>>> input_tensor = Tensor(np.array([[[[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]]]), mindspore.float32) >>> resize = ops.ResizeNearestNeighbor((2, 2)) >>> output = resize(input_tensor) >>> print(output) [[[[-0.1 0.3] [ 0.4 0.5]]]]
-
class
tinyms.primitives.ReverseSequence(*args, **kwargs)[source]¶ Reverses variable length slices.
- Parameters
- Inputs:
x (Tensor) - The input to reverse, supporting all number types including bool.
seq_lengths (Tensor) - Must be a 1-D vector with int32 or int64 types.
- Outputs:
Reversed tensor with the same shape and data type as input.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]), mindspore.float32) >>> seq_lengths = Tensor(np.array([1, 2, 3])) >>> reverse_sequence = ops.ReverseSequence(seq_dim=1) >>> output = reverse_sequence(x, seq_lengths) >>> print(output) [[1. 2. 3.] [5. 4. 6.] [9. 8. 7.]]
-
class
tinyms.primitives.ReverseV2(*args, **kwargs)[source]¶ Reverses specific dimensions of a tensor.
- Inputs:
input_x (Tensor) - The target tensor.
- Outputs:
Tensor, has the same shape and type as input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([[1, 2, 3, 4], [5, 6, 7, 8]]), mindspore.int32) >>> op = ops.ReverseV2(axis=[1]) >>> output = op(input_x) >>> print(output) [[4 3 2 1] [8 7 6 5]]
-
class
tinyms.primitives.Rint(*args, **kwargs)[source]¶ Returns an integer that is closest to x element-wise.
- Inputs:
input_x (Tensor) - The target tensor, which must be one of the following types: float16, float32.
- Outputs:
Tensor, has the same shape and type as input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([-1.6, -0.1, 1.5, 2.0]), mindspore.float32) >>> op = ops.Rint() >>> output = op(input_x) >>> print(output) [-2. 0. 2. 2.]
-
class
tinyms.primitives.Round(*args, **kwargs)[source]¶ Returns half to even of a tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, has the same shape and type as the input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([0.8, 1.5, 2.3, 2.5, -4.5]), mindspore.float32) >>> round = ops.Round() >>> output = round(input_x) >>> print(output) [ 1. 2. 2. 2. -4.]
-
class
tinyms.primitives.Rsqrt(*args, **kwargs)[source]¶ Computes reciprocal of square root of input tensor element-wise.
- Inputs:
input_x (Tensor) - The input of Rsqrt. Each element must be a non-negative number.
- Outputs:
Tensor, has the same type and shape as input_x.
- Supported Platforms:
AscendGPU
Examples
>>> input_tensor = Tensor([[4, 4], [9, 9]], mindspore.float32) >>> rsqrt = ops.Rsqrt() >>> output = rsqrt(input_tensor) >>> print(output) [[0.5 0.5 ] [0.33333334 0.33333334]]
-
class
tinyms.primitives.SGD(*args, **kwargs)[source]¶ Computes the stochastic gradient descent. Momentum is optional.
Nesterov momentum is based on the formula from paper On the importance of initialization and momentum in deep learning.
Note
For details, please refer to nn.SGD source code.
- Parameters
- Inputs:
parameters (Tensor) - Parameters to be updated. With float16 or float32 data type.
gradient (Tensor) - Gradient, with float16 or float32 data type.
learning_rate (Tensor) - Learning rate, a scalar tensor with float16 or float32 data type. e.g. Tensor(0.1, mindspore.float32)
accum (Tensor) - Accum(velocity) to be updated. With float16 or float32 data type.
momentum (Tensor) - Momentum, a scalar tensor with float16 or float32 data type. e.g. Tensor(0.1, mindspore.float32).
stat (Tensor) - States to be updated with the same shape as gradient, with float16 or float32 data type.
- Outputs:
Tensor, parameters to be updated.
- Supported Platforms:
AscendGPU
Examples
>>> sgd = ops.SGD() >>> parameters = Tensor(np.array([2, -0.5, 1.7, 4]), mindspore.float32) >>> gradient = Tensor(np.array([1, -1, 0.5, 2]), mindspore.float32) >>> learning_rate = Tensor(0.01, mindspore.float32) >>> accum = Tensor(np.array([0.1, 0.3, -0.2, -0.1]), mindspore.float32) >>> momentum = Tensor(0.1, mindspore.float32) >>> stat = Tensor(np.array([1.5, -0.3, 0.2, -0.7]), mindspore.float32) >>> output = sgd(parameters, gradient, learning_rate, accum, momentum, stat) >>> print(output[0]) (Tensor(shape=[4], dtype=Float32, value= [ 1.98989999e+00, -4.90300000e-01, 1.69520009e+00, 3.98009992e+00]),)
-
class
tinyms.primitives.SameTypeShape(*args, **kwargs)[source]¶ Checks whether the data type and shape of two tensors are the same.
- Raises
TypeError – If the data types of two tensors are not the same.
ValueError – If the shapes of two tensors are not the same.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
input_y (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_S)\).
- Outputs:
Tensor, the shape of tensor is \((x_1, x_2, ..., x_R)\), if data type and shape of input_x and input_y are the same.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) >>> input_y = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) >>> output = ops.SameTypeShape()(input_x, input_y) >>> print(output) [[2. 2.] [2. 2.]]
-
class
tinyms.primitives.ScalarCast(*args, **kwargs)[source]¶ Casts the input scalar to another type.
- Inputs:
input_x (scalar) - The input scalar. Only constant value is allowed.
input_y (mindspore.dtype) - The type to be cast. Only constant value is allowed.
- Outputs:
Scalar. The type is the same as the python type corresponding to input_y.
- Supported Platforms:
AscendGPUCPU
Examples
>>> scalar_cast = ops.ScalarCast() >>> output = scalar_cast(255.0, mindspore.int32) >>> print(output) 255
-
class
tinyms.primitives.ScalarSummary(*args, **kwargs)[source]¶ Outputs a scalar to a protocol buffer through a scalar summary operator.
- Inputs:
name (str) - The name of the input variable, it must not be an empty string.
value (Tensor) - The value of scalar, and the shape of value must be [] or [1].
- Supported Platforms:
AscendGPUCPU
Examples
>>> class SummaryDemo(nn.Cell): ... def __init__(self,): ... super(SummaryDemo, self).__init__() ... self.summary = ops.ScalarSummary() ... self.add = ops.Add() ... ... def construct(self, x, y): ... name = "x" ... self.summary(name, x) ... x = self.add(x, y) ... return x ...
-
class
tinyms.primitives.ScalarToArray(*args, **kwargs)[source]¶ Converts a scalar to a Tensor.
- Inputs:
input_x (Union[int, float]) - The input is a scalar. Only constant value is allowed.
- Outputs:
Tensor. 0-D Tensor and the content is the input.
- Supported Platforms:
AscendGPUCPU
Examples
>>> op = ops.ScalarToArray() >>> data = 1.0 >>> output = op(data) >>> print(output) 1.0
-
class
tinyms.primitives.ScalarToTensor(*args, **kwargs)[source]¶ Converts a scalar to a Tensor, and converts the data type to the specified type.
- Inputs:
input_x (Union[int, float]) - The input is a scalar. Only constant value is allowed.
dtype (mindspore.dtype) - The target data type. Default: mindspore.float32. Only constant value is allowed.
- Outputs:
Tensor. 0-D Tensor and the content is the input.
- Supported Platforms:
AscendGPUCPU
Examples
>>> op = ops.ScalarToTensor() >>> data = 1 >>> output = op(data, mindspore.float32) >>> print(output) 1.0
-
class
tinyms.primitives.ScatterAdd(*args, **kwargs)[source]¶ Updates the value of the input tensor through the addition operation.
Using given values to update tensor value through the add operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
\[\begin{split}\begin{array}{l} \text {for each i, ..., j in indices.shape:} \\ input\_x[indices[i, ..., j], :] \mathrel{+}= updates[i, ..., j, :] \end{array}\end{split}\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do add operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor that performs the add operation with input_x, the data type is the same as input_x, the shape is indices_shape + x_shape[1:].
- Outputs:
Parameter, the updated input_x.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Parameter(Tensor(np.array([[0.0, 0.0, 0.0], [0.0, 0.0, 0.0]]), mindspore.float32), name="x") >>> indices = Tensor(np.array([[0, 1], [1, 1]]), mindspore.int32) >>> updates = Tensor(np.ones([2, 2, 3]), mindspore.float32) >>> scatter_add = ops.ScatterAdd() >>> output = scatter_add(input_x, indices, updates) >>> print(output) [[1. 1. 1.] [3. 3. 3.]]
-
class
tinyms.primitives.ScatterDiv(*args, **kwargs)[source]¶ Updates the value of the input tensor through the divide operation.
Using given values to update tensor value through the div operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
\[\begin{split}\begin{array}{l} \text {for each i, ..., j in indices.shape:} \\ input\_x[indices[i, ..., j], :] \mathrel{/}= updates[i, ..., j, :] \end{array}\end{split}\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do div operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor that performs the div operation with input_x, the data type is the same as input_x, the shape is indices_shape + x_shape[1:].
- Outputs:
Parameter, the updated input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([[6.0, 6.0, 6.0], [2.0, 2.0, 2.0]]), mindspore.float32), name="x") >>> indices = Tensor(np.array([0, 1]), mindspore.int32) >>> updates = Tensor(np.array([[2.0, 2.0, 2.0], [2.0, 2.0, 2.0]]), mindspore.float32) >>> scatter_div = ops.ScatterDiv() >>> output = scatter_div(input_x, indices, updates) >>> print(output) [[3. 3. 3.] [1. 1. 1.]]
-
class
tinyms.primitives.ScatterMax(*args, **kwargs)[source]¶ Updates the value of the input tensor through the maximum operation.
Using given values to update tensor value through the max operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
\[\begin{split}\begin{array}{l} \text {for each i, ..., j in indices.shape:} \\ input\_x[indices[i, ..., j], :] = max(input\_x[indices[i, ..., j], :], updates[i, ..., j, :]) \end{array}\end{split}\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: True.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do max operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor that performs the maximum operation with input_x, the data type is the same as input_x, the shape is indices_shape + x_shape[1:].
- Outputs:
Parameter, the updated input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]), mindspore.float32), name="input_x") >>> indices = Tensor(np.array([[0, 0], [1, 1]]), mindspore.int32) >>> updates = Tensor(np.ones([2, 2, 3]) * 88, mindspore.float32) >>> scatter_max = ops.ScatterMax() >>> output = scatter_max(input_x, indices, updates) >>> print(output) [[88. 88. 88.] [88. 88. 88.]]
-
class
tinyms.primitives.ScatterMin(*args, **kwargs)[source]¶ Updates the value of the input tensor through the minimum operation.
Using given values to update tensor value through the min operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
\[\begin{split}\begin{array}{l} \text {for each i, ..., j in indices.shape:} \\ input\_x[indices[i, ..., j], :] = min(input\_x[indices[i, ..., j], :], updates[i, ..., j, :]) \end{array}\end{split}\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do min operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor doing the min operation with input_x, the data type is same as input_x, the shape is indices_shape + x_shape[1:].
- Outputs:
Parameter, the updated input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([[0.0, 1.0, 2.0], [0.0, 0.0, 0.0]]), mindspore.float32), name="input_x") >>> indices = Tensor(np.array([[0, 0], [1, 1]]), mindspore.int32) >>> update = Tensor(np.ones([2, 2, 3]), mindspore.float32) >>> scatter_min = ops.ScatterMin() >>> output = scatter_min(input_x, indices, update) >>> print(output) [[0. 1. 1.] [0. 0. 0.]]
-
class
tinyms.primitives.ScatterMul(*args, **kwargs)[source]¶ Updates the value of the input tensor through the multiply operation.
Using given values to update tensor value through the mul operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
\[\begin{split}\begin{array}{l} \text {for each i, ..., j in indices.shape:} \\ input\_x[indices[i, ..., j], :] \mathrel{*}= updates[i, ..., j, :] \end{array}\end{split}\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do mul operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor doing the mul operation with input_x, the data type is same as input_x, the shape is indices_shape + x_shape[1:].
- Outputs:
Parameter, the updated input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([[1.0, 1.0, 1.0], [2.0, 2.0, 2.0]]), mindspore.float32), name="x") >>> indices = Tensor(np.array([0, 1]), mindspore.int32) >>> updates = Tensor(np.array([[2.0, 2.0, 2.0], [2.0, 2.0, 2.0]]), mindspore.float32) >>> scatter_mul = ops.ScatterMul() >>> output = scatter_mul(input_x, indices, updates) >>> print(output) [[2. 2. 2.] [4. 4. 4.]]
-
class
tinyms.primitives.ScatterNd(*args, **kwargs)[source]¶ Scatters a tensor into a new tensor depending on the specified indices.
Creates an empty tensor with the given shape, and set values by scattering the update tensor depending on indices.
The empty tensor has rank P and indices has rank Q where Q >= 2.
indices has shape \((i_0, i_1, ..., i_{Q-2}, N)\) where N <= P.
The last dimension of indices (with length N ) indicates slices along the N th dimension of the empty tensor.
updates is a tensor of rank Q-1+P-N. Its shape is: \((i_0, i_1, ..., i_{Q-2}, shape_N, ..., shape_{P-1})\).
- Inputs:
indices (Tensor) - The index of scattering in the new tensor with int32 data type. The rank of indices must be at least 2 and indices_shape[-1] <= len(shape).
updates (Tensor) - The source Tensor to be scattered. It has shape indices_shape[:-1] + shape[indices_shape[-1]:].
shape (tuple[int]) - Define the shape of the output tensor, has the same type as indices.
- Outputs:
Tensor, the new tensor, has the same type as update and the same shape as shape.
- Supported Platforms:
AscendGPU
Examples
>>> op = ops.ScatterNd() >>> indices = Tensor(np.array([[0, 1], [1, 1]]), mindspore.int32) >>> updates = Tensor(np.array([3.2, 1.1]), mindspore.float32) >>> shape = (3, 3) >>> output = op(indices, updates, shape) >>> print(output) [[0. 3.2 0. ] [0. 1.1 0. ] [0. 0. 0. ]]
-
class
tinyms.primitives.ScatterNdAdd(*args, **kwargs)[source]¶ Applies sparse addition to individual values or slices in a tensor.
Using given values to update tensor value through the add operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
input_x has rank P and indices has rank Q where Q >= 2.
indices has shape \((i_0, i_1, ..., i_{Q-2}, N)\) where N <= P.
The last dimension of indices (with length N ) indicates slices along the N th dimension of input_x.
updates is a tensor of rank Q-1+P-N. Its shape is: \((i_0, i_1, ..., i_{Q-2}, x\_shape_N, ..., x\_shape_{P-1})\).
Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do add operation whose data type must be mindspore.int32. The rank of indices must be at least 2 and indices_shape[-1] <= len(shape).
updates (Tensor) - The tensor doing the add operation with input_x, the data type is same as input_x, the shape is indices_shape[:-1] + x_shape[indices_shape[-1]:].
- Outputs:
Parameter, the updated input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([1, 2, 3, 4, 5, 6, 7, 8]), mindspore.float32), name="x") >>> indices = Tensor(np.array([[2], [4], [1], [7]]), mindspore.int32) >>> updates = Tensor(np.array([6, 7, 8, 9]), mindspore.float32) >>> scatter_nd_add = ops.ScatterNdAdd() >>> output = scatter_nd_add(input_x, indices, updates) >>> print(output) [ 1. 10. 9. 4. 12. 6. 7. 17.]
-
class
tinyms.primitives.ScatterNdSub(*args, **kwargs)[source]¶ Applies sparse subtraction to individual values or slices in a tensor.
Using given values to update tensor value through the subtraction operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
input_x has rank P and indices has rank Q where Q >= 2.
indices has shape \((i_0, i_1, ..., i_{Q-2}, N)\) where N <= P.
The last dimension of indices (with length N ) indicates slices along the N th dimension of input_x.
updates is a tensor of rank Q-1+P-N. Its shape is: \((i_0, i_1, ..., i_{Q-2}, x\_shape_N, ..., x\_shape_{P-1})\).
Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do add operation whose data type must be mindspore.int32. The rank of indices must be at least 2 and indices_shape[-1] <= len(shape).
updates (Tensor) - The tensor that performs the subtraction operation with input_x, the data type is the same as input_x, the shape is indices_shape[:-1] + x_shape[indices_shape[-1]:].
- Outputs:
Parameter, the updated input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([1, 2, 3, 4, 5, 6, 7, 8]), mindspore.float32), name="x") >>> indices = Tensor(np.array([[2], [4], [1], [7]]), mindspore.int32) >>> updates = Tensor(np.array([6, 7, 8, 9]), mindspore.float32) >>> scatter_nd_sub = ops.ScatterNdSub() >>> output = scatter_nd_sub(input_x, indices, updates) >>> print(output) [ 1. -6. -3. 4. -2. 6. 7. -1.]
-
class
tinyms.primitives.ScatterNdUpdate(*args, **kwargs)[source]¶ Updates tensor values by using input indices and value.
Using given values to update tensor value, along with the input indices.
input_x has rank P and indices has rank Q where Q >= 2.
indices has shape \((i_0, i_1, ..., i_{Q-2}, N)\) where N <= P.
The last dimension of indices (with length N ) indicates slices along the N th dimension of input_x.
updates is a tensor of rank Q-1+P-N. Its shape is: \((i_0, i_1, ..., i_{Q-2}, x\_shape_N, ..., x\_shape_{P-1})\).
Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: True.
- Inputs:
input_x (Parameter) - The target tensor, with data type of Parameter.
indices (Tensor) - The index of input tensor, with int32 data type. The rank of indices must be at least 2 and indices_shape[-1] <= len(shape).
updates (Tensor) - The tensor to be updated to the input tensor, has the same type as input. the shape is indices_shape[:-1] + x_shape[indices_shape[-1]:].
- Outputs:
Tensor, has the same shape and type as input_x.
- Supported Platforms:
AscendCPU
Examples
>>> np_x = np.array([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]) >>> input_x = mindspore.Parameter(Tensor(np_x, mindspore.float32), name="x") >>> indices = Tensor(np.array([[0, 0], [1, 1]]), mindspore.int32) >>> updates = Tensor(np.array([1.0, 2.2]), mindspore.float32) >>> op = ops.ScatterNdUpdate() >>> output = op(input_x, indices, updates) >>> print(output) [[ 1. 0.3 3.6] [ 0.4 2.2 -3.2]]
-
class
tinyms.primitives.ScatterNonAliasingAdd(*args, **kwargs)[source]¶ Applies sparse addition to the input using individual values or slices.
Using given values to update tensor value through the add operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
input_x (Parameter) - The target parameter. The data type must be float16, float32 or int32.
indices (Tensor) - The index to perform the addition operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor that performs the addition operation with input_x, the data type is the same as input_x, the shape is indices_shape[:-1] + x_shape[indices_shape[-1]:].
- Outputs:
Parameter, the updated input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([1, 2, 3, 4, 5, 6, 7, 8]), mindspore.float32), name="x") >>> indices = Tensor(np.array([[2], [4], [1], [7]]), mindspore.int32) >>> updates = Tensor(np.array([6, 7, 8, 9]), mindspore.float32) >>> scatter_non_aliasing_add = ops.ScatterNonAliasingAdd() >>> output = scatter_non_aliasing_add(input_x, indices, updates) >>> print(output) [ 1. 10. 9. 4. 12. 6. 7. 17.]
-
class
tinyms.primitives.ScatterSub(*args, **kwargs)[source]¶ Updates the value of the input tensor through the subtraction operation.
Using given values to update tensor value through the subtraction operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
\[\begin{split}\begin{array}{l} \text {for each i, ..., j in indices.shape:} \\ input\_x[indices[i, ..., j], :] \mathrel{-}= updates[i, ..., j, :] \end{array}\end{split}\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to perform the subtraction operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor that performs the subtraction operation with input_x, the data type is the same as input_x, the shape is indices_shape + x_shape[1:].
- Outputs:
Parameter, the updated input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([[0.0, 0.0, 0.0], [1.0, 1.0, 1.0]]), mindspore.float32), name="x") >>> indices = Tensor(np.array([[0, 1]]), mindspore.int32) >>> updates = Tensor(np.array([[[1.0, 1.0, 1.0], [2.0, 2.0, 2.0]]]), mindspore.float32) >>> scatter_sub = ops.ScatterSub() >>> output = scatter_sub(input_x, indices, updates) >>> print(output) [[-1. -1. -1.] [-1. -1. -1.]]
-
class
tinyms.primitives.ScatterUpdate(*args, **kwargs)[source]¶ Updates tensor values by using input indices and value.
Using given values to update tensor value, along with the input indices.
\[\begin{split}\begin{array}{l} \text {for each i, ..., j in indices.shape:} \\ input\_x[indices[i, ..., j], :] = updates[i, ..., j, :] \end{array}\end{split}\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: True.
- Inputs:
input_x (Parameter) - The target tensor, with data type of Parameter.
indices (Tensor) - The index of input tensor. With int32 data type. If there are duplicates in indices, the order for updating is undefined.
updates (Tensor) - The tensor to update the input tensor, has the same type as input, and updates.shape = indices.shape + input_x.shape[1:].
- Outputs:
Tensor, has the same shape and type as input_x.
- Supported Platforms:
AscendGPU
Examples
>>> np_x = np.array([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]) >>> input_x = mindspore.Parameter(Tensor(np_x, mindspore.float32), name="x") >>> indices = Tensor(np.array([0, 1]), mindspore.int32) >>> np_updates = np.array([[2.0, 1.2, 1.0], [3.0, 1.2, 1.0]]) >>> updates = Tensor(np_updates, mindspore.float32) >>> op = ops.ScatterUpdate() >>> output = op(input_x, indices, updates) >>> print(output) [[2. 1.2 1. ] [3. 1.2 1. ]]
-
class
tinyms.primitives.Select(*args, **kwargs)[source]¶ Returns the selected elements, either from input \(x\) or input \(y\), depending on the condition.
Given a tensor as input, this operation inserts a dimension of 1 at the dimension, if both \(x\) and \(y\) are none, the operation returns the coordinates of the true element in the condition, the coordinates are returned as a two-dimensional tensor, where the first dimension (row) represents the number of true elements and the second dimension (columns) represents the coordinates of the true elements. Keep in mind that the shape of the output tensor can vary depending on how many true values are in the input. Indexes are output in row-first order.
If neither is None, \(x\) and \(y\) must have the same shape. If \(x\) and \(y\) are scalars, the conditional tensor must be a scalar. If \(x\) and \(y\) are higher-dimensional vectors, the condition must be a vector whose size matches the first dimension of \(x\), or must have the same shape as \(y\).
The conditional tensor acts as an optional compensation (mask), which determines whether the corresponding element / row in the output must be selected from \(x\) (if true) or \(y\) (if false) based on the value of each element.
If condition is a vector, then \(x\) and \(y\) are higher-dimensional matrices, then it chooses to copy that row (external dimensions) from \(x\) and \(y\). If condition has the same shape as \(x\) and \(y\), you can choose to copy these elements from \(x\) and \(y\).
- Inputs:
input_cond (Tensor[bool]) - The shape is \((x_1, x_2, ..., x_N, ..., x_R)\). The condition tensor, decides which element is chosen.
input_x (Tensor) - The shape is \((x_1, x_2, ..., x_N, ..., x_R)\). The first input tensor.
input_y (Tensor) - The shape is \((x_1, x_2, ..., x_N, ..., x_R)\). The second input tensor.
- Outputs:
Tensor, has the same shape as input_x. The shape is \((x_1, x_2, ..., x_N, ..., x_R)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> select = ops.Select() >>> input_cond = Tensor([True, False]) >>> input_x = Tensor([2,3], mindspore.float32) >>> input_y = Tensor([1,2], mindspore.float32) >>> output = select(input_cond, input_x, input_y) >>> print(output) [2. 2.]
-
class
tinyms.primitives.Shape(*args, **kwargs)[source]¶ Returns the shape of the input tensor.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
tuple[int], the output tuple is constructed by multiple integers, \((x_1, x_2, ..., x_R)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.ones(shape=[3, 2, 1]), mindspore.float32) >>> shape = ops.Shape() >>> output = shape(input_tensor) >>> print(output) (3, 2, 1)
-
class
tinyms.primitives.Sigmoid(*args, **kwargs)[source]¶ Sigmoid activation function.
Computes Sigmoid of input element-wise. The Sigmoid function is defined as:
\[\text{sigmoid}(x_i) = \frac{1}{1 + \exp(-x_i)},\]where \(x_i\) is the element of the input.
- Inputs:
input_x (Tensor) - The input of Sigmoid, data type must be float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_x.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3, 4, 5]), mindspore.float32) >>> sigmoid = ops.Sigmoid() >>> output = sigmoid(input_x) >>> print(output) [0.7310586 0.880797 0.95257413 0.98201376 0.9933072 ]
-
class
tinyms.primitives.SigmoidCrossEntropyWithLogits(*args, **kwargs)[source]¶ Uses the given logits to compute sigmoid cross entropy.
Sets input logits as X, input label as Y, output as loss. Then,
\[p_{ij} = sigmoid(X_{ij}) = \frac{1}{1 + e^{-X_{ij}}}\]\[loss_{ij} = -[Y_{ij} * ln(p_{ij}) + (1 - Y_{ij})ln(1 - p_{ij})]\]- Inputs:
logits (Tensor) - Input logits.
label (Tensor) - Ground truth label.
- Outputs:
Tensor, with the same shape and type as input logits.
- Supported Platforms:
AscendGPUCPU
Examples
>>> logits = Tensor(np.array([[-0.8, 1.2, 0.7], [-0.1, -0.4, 0.7]]).astype(np.float32)) >>> labels = Tensor(np.array([[0.3, 0.8, 1.2], [-0.6, 0.1, 2.2]]).astype(np.float32)) >>> sigmoid = ops.SigmoidCrossEntropyWithLogits() >>> output = sigmoid(logits, labels) >>> print(output) [[ 0.6111007 0.5032824 0.26318604] [ 0.58439666 0.5530153 -0.4368139 ]]
-
class
tinyms.primitives.Sign(*args, **kwargs)[source]¶ Performs sign on the tensor element-wise.
Note
\[sign(x) = \begin{cases} -1, &if\ x < 0 \cr 0, &if\ x = 0 \cr 1, &if\ x > 0\end{cases}\]- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, has the same shape and type as the input_x.
- Supported Platforms:
AscendCPU
Examples
>>> input_x = Tensor(np.array([[2.0, 0.0, -1.0]]), mindspore.float32) >>> sign = ops.Sign() >>> output = sign(input_x) >>> print(output) [[ 1. 0. -1.]]
-
class
tinyms.primitives.Sin(*args, **kwargs)[source]¶ Computes sine of the input element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Supported Platforms:
AscendGPU
Examples
>>> sin = ops.Sin() >>> input_x = Tensor(np.array([0.62, 0.28, 0.43, 0.62]), mindspore.float32) >>> output = sin(input_x) >>> print(output) [0.5810352 0.27635565 0.41687083 0.5810352 ]
-
class
tinyms.primitives.Sinh(*args, **kwargs)[source]¶ Computes hyperbolic sine of the input element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Supported Platforms:
Ascend
Examples
>>> sinh = ops.Sinh() >>> input_x = Tensor(np.array([0.62, 0.28, 0.43, 0.62]), mindspore.float32) >>> output = sinh(input_x) >>> print(output) [0.6604918 0.28367308 0.44337422 0.6604918 ]
-
class
tinyms.primitives.Size(*args, **kwargs)[source]¶ Returns the size of a tensor.
Returns an int scalar representing the elements size of input, the total number of elements in the tensor.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
int, a scalar representing the elements size of input_x, tensor is the number of elements in a tensor, \(size=x_1*x_2*...x_R\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) >>> size = ops.Size() >>> output = size(input_tensor) >>> print(output) 4
-
class
tinyms.primitives.Slice(*args, **kwargs)[source]¶ Slices a tensor in the specified shape.
- Inputs:
x (Tensor): The target tensor.
begin (tuple, list): The beginning of the slice. Only constant value is allowed.
size (tuple, list): The size of the slice. Only constant value is allowed.
- Outputs:
Tensor, the shape is : input size, the data type is the same as input x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> data = Tensor(np.array([[[1, 1, 1], [2, 2, 2]], ... [[3, 3, 3], [4, 4, 4]], ... [[5, 5, 5], [6, 6, 6]]]).astype(np.int32)) >>> slice = ops.Slice() >>> output = slice(data, (1, 0, 0), (1, 1, 3)) >>> print(output) [[[3 3 3]]]
-
class
tinyms.primitives.SmoothL1Loss(*args, **kwargs)[source]¶ Computes smooth L1 loss, a robust L1 loss.
SmoothL1Loss is a Loss similar to MSELoss but less sensitive to outliers as described in the Fast R-CNN by Ross Girshick.
Note
Sets input prediction as X, input target as Y, output as loss. Then,
\[\text{SmoothL1Loss} = \begin{cases} \frac{0.5 x^{2}}{\text{beta}}, &if \left |x \right | < \text{beta} \cr \left |x \right|-0.5 \text{beta}, &\text{otherwise}\end{cases}\]- Parameters
beta (float) – A parameter used to control the point where the function will change from quadratic to linear. Default: 1.0.
- Inputs:
prediction (Tensor) - Predict data. Data type must be float16 or float32.
target (Tensor) - Ground truth data, with the same type and shape as prediction.
- Outputs:
Tensor, with the same type and shape as prediction.
- Supported Platforms:
AscendGPU
Examples
>>> loss = ops.SmoothL1Loss() >>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32) >>> output = loss(input_data, target_data) >>> print(output) [0. 0. 0.5]
-
class
tinyms.primitives.Softmax(*args, **kwargs)[source]¶ Softmax operation.
Applies the Softmax operation to the input tensor on the specified axis. Suppose a slice in the given aixs \(x\), then for each element \(x_i\), the Softmax function is shown as follows:
\[\text{output}(x_i) = \frac{exp(x_i)}{\sum_{j = 0}^{N-1}\exp(x_j)},\]where \(N\) is the length of the tensor.
- Inputs:
logits (Tensor) - The input of Softmax, with float16 or float32 data type.
- Outputs:
Tensor, with the same type and shape as the logits.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3, 4, 5]), mindspore.float32) >>> softmax = ops.Softmax() >>> output = softmax(input_x) >>> print(output) [0.01165623 0.03168492 0.08612854 0.23412167 0.6364086 ]
-
class
tinyms.primitives.SoftmaxCrossEntropyWithLogits(*args, **kwargs)[source]¶ Gets the softmax cross-entropy value between logits and labels with one-hot encoding.
Note
Sets input logits as X, input label as Y, output as loss. Then,
\[p_{ij} = softmax(X_{ij}) = \frac{\exp(x_i)}{\sum_{j = 0}^{N-1}\exp(x_j)}\]\[loss_{ij} = -\sum_j{Y_{ij} * ln(p_{ij})}\]- Inputs:
logits (Tensor) - Input logits, with shape \((N, C)\). Data type must be float16 or float32.
labels (Tensor) - Ground truth labels, with shape \((N, C)\), has the same data type with logits.
- Outputs:
Tuple of 2 tensors, the loss shape is (N,), and the dlogits with the same shape as logits.
- Supported Platforms:
AscendGPUCPU
Examples
>>> logits = Tensor([[2, 4, 1, 4, 5], [2, 1, 2, 4, 3]], mindspore.float32) >>> labels = Tensor([[0, 0, 0, 0, 1], [0, 0, 0, 1, 0]], mindspore.float32) >>> softmax_cross = ops.SoftmaxCrossEntropyWithLogits() >>> loss, dlogits = softmax_cross(logits, labels) >>> print(loss) [0.5899297 0.52374405] >>> print(dlogits) [[ 0.02760027 0.20393994 0.01015357 0.20393994 -0.44563377] [ 0.08015892 0.02948882 0.08015892 -0.4077012 0.21789455]]
-
class
tinyms.primitives.Softplus(*args, **kwargs)[source]¶ Softplus activation function.
Softplus is a smooth approximation to the ReLU function. The function is shown as follows:
\[\text{output} = \log(1 + \exp(\text{input_x})),\]- Inputs:
input_x (Tensor) - The input tensor whose data type must be float.
- Outputs:
Tensor, with the same type and shape as the input_x.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3, 4, 5]), mindspore.float32) >>> softplus = ops.Softplus() >>> output = softplus(input_x) >>> print(output) [1.3132615 2.126928 3.0485873 4.01815 5.0067153]
-
class
tinyms.primitives.Softsign(*args, **kwargs)[source]¶ Softsign activation function.
The function is shown as follows:
\[\text{SoftSign}(x) = \frac{x}{ 1 + |x|}\]- Inputs:
input_x (Tensor) - The input tensor whose data type must be float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([0, -1, 2, 30, -30]), mindspore.float32) >>> softsign = ops.Softsign() >>> output = softsign(input_x) >>> print(output) [ 0. -0.5 0.6666667 0.9677419 -0.9677419]
-
class
tinyms.primitives.Sort(*args, **kwargs)[source]¶ Sorts the elements of the input tensor along a given dimension in ascending order by value.
- Parameters
- Inputs:
x (Tensor) - The input to sort, with float16 or float32 data type.
- Outputs:
y1 (Tensor) - A tensor whose values are the sorted values, with the same shape and data type as input.
y2 (Tensor) - The indices of the elements in the original input tensor. Data type is int32.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.array([[8, 2, 1], [5, 9, 3], [4, 6, 7]]), mindspore.float16) >>> sort = ops.Sort() >>> output = sort(x) >>> print(output) (Tensor(shape=[3, 3], dtype=Float16, value= [[ 1.0000e+00, 2.0000e+00, 8.0000e+00], [ 3.0000e+00, 5.0000e+00, 9.0000e+00], [ 4.0000e+00, 6.0000e+00, 7.0000e+00]]), Tensor(shape=[3, 3], dtype=Int32, value= [[2, 1, 0], [2, 0, 1], [0, 1, 2]]))
-
class
tinyms.primitives.SpaceToBatch(*args, **kwargs)[source]¶ Divides spatial dimensions into blocks and combines the block size with the original batch.
This operation will divide spatial dimensions (H, W) into blocks with block_size, the output tensor’s H and W dimension is the corresponding number of blocks after division. The output tensor’s batch dimension is the product of the original batch and the square of block_size. Before division, the spatial dimensions of the input are zero padded according to paddings if necessary.
- Parameters
block_size (int) – The block size of dividing blocks with value greater than 2.
paddings (Union[tuple, list]) – The padding values for H and W dimension, containing 2 subtraction lists. Each subtraction list contains 2 integer value. All values must be greater than 0. paddings[i] specifies the paddings for the spatial dimension i, which corresponds to the input dimension i+2. It is required that input_shape[i+2]+paddings[i][0]+paddings[i][1] is divisible by block_size.
- Inputs:
input_x (Tensor) - The input tensor. It must be a 4-D tensor.
- Outputs:
Tensor, the output tensor with the same data type as input. Assume input shape is \((n, c, h, w)\) with \(block\_size\) and \(paddings\). The shape of the output tensor will be \((n', c', h', w')\), where
\(n' = n*(block\_size*block\_size)\)
\(c' = c\)
\(h' = (h+paddings[0][0]+paddings[0][1])//block\_size\)
\(w' = (w+paddings[1][0]+paddings[1][1])//block\_size\)
- Supported Platforms:
Ascend
Examples
>>> block_size = 2 >>> paddings = [[0, 0], [0, 0]] >>> space_to_batch = ops.SpaceToBatch(block_size, paddings) >>> input_x = Tensor(np.array([[[[1, 2], [3, 4]]]]), mindspore.float32) >>> output = space_to_batch(input_x) >>> print(output) [[[[1.]]] [[[2.]]] [[[3.]]] [[[4.]]]]
-
class
tinyms.primitives.SpaceToBatchND(*args, **kwargs)[source]¶ Divides spatial dimensions into blocks and combines the block size with the original batch.
This operation will divide spatial dimensions (H, W) into blocks with block_shape, the output tensor’s H and W dimension is the corresponding number of blocks after division. The output tensor’s batch dimension is the product of the original batch and the product of block_shape. Before division, the spatial dimensions of the input are zero padded according to paddings if necessary.
- Parameters
block_shape (Union[list(int), tuple(int), int]) – The block shape of dividing block with all value greater than 1. If block_shape is a tuple or list, the length of block_shape is M corresponding to the number of spatial dimensions. If block_shape is a int, the block size of M dimendions are the same, equal to block_shape. M must be 2.
paddings (Union[tuple, list]) – The padding values for H and W dimension, containing 2 subtraction list. Each contains 2 integer value. All values must be greater than 0. paddings[i] specifies the paddings for the spatial dimension i, which corresponds to the input dimension i+2. It is required that input_shape[i+2]+paddings[i][0]+paddings[i][1] is divisible by block_shape[i].
- Inputs:
input_x (Tensor) - The input tensor. It must be a 4-D tensor.
- Outputs:
Tensor, the output tensor with the same data type as input. Assume input shape is \((n, c, h, w)\) with \(block\_shape\) and \(padddings\). The shape of the output tensor will be \((n', c', h', w')\), where
\(n' = n*(block\_shape[0]*block\_shape[1])\)
\(c' = c\)
\(h' = (h+paddings[0][0]+paddings[0][1])//block\_shape[0]\)
\(w' = (w+paddings[1][0]+paddings[1][1])//block\_shape[1]\)
- Supported Platforms:
Ascend
Examples
>>> block_shape = [2, 2] >>> paddings = [[0, 0], [0, 0]] >>> space_to_batch_nd = ops.SpaceToBatchND(block_shape, paddings) >>> input_x = Tensor(np.array([[[[1, 2], [3, 4]]]]), mindspore.float32) >>> output = space_to_batch_nd(input_x) >>> print(output) [[[[1.]]] [[[2.]]] [[[3.]]] [[[4.]]]]
-
class
tinyms.primitives.SpaceToDepth(*args, **kwargs)[source]¶ Rearranges blocks of spatial data into depth.
The output tensor’s height dimension is \(height / block\_size\).
The output tensor’s weight dimension is \(weight / block\_size\).
The depth of output tensor is \(block\_size * block\_size * input\_depth\).
The input tensor’s height and width must be divisible by block_size. The data format is “NCHW”.
- Parameters
block_size (int) – The block size used to divide spatial data. It must be >= 2.
- Inputs:
x (Tensor) - The target tensor.
- Outputs:
Tensor, the same data type as x. It must be a 4-D tensor.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.random.rand(1,3,2,2), mindspore.float32) >>> block_size = 2 >>> space_to_depth = ops.SpaceToDepth(block_size) >>> output = space_to_depth(x) >>> print(output.shape) (1, 12, 1, 1)
-
class
tinyms.primitives.SparseApplyAdagrad(*args, **kwargs)[source]¶ Updates relevant entries according to the adagrad scheme.
\[accum += grad * grad\]\[var -= lr * grad * (1 / sqrt(accum))\]Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
- Inputs:
var (Parameter) - Variable to be updated. The data type must be float16 or float32.
accum (Parameter) - Accumulation to be updated. The shape and data type must be the same as var.
grad (Tensor) - Gradient. The shape must be the same as var’s shape except the first dimension. Gradients has the same data type as var.
indices (Tensor) - A vector of indices into the first dimension of var and accum. The shape of indices must be the same as grad in first dimension, the type must be int32.
- Outputs:
Tuple of 2 tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as accum.
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.sparse_apply_adagrad = ops.SparseApplyAdagrad(lr=1e-8) ... self.var = Parameter(Tensor(np.ones([1, 1, 1]).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.ones([1, 1, 1]).astype(np.float32)), name="accum") ... def construct(self, grad, indices): ... out = self.sparse_apply_adagrad(self.var, self.accum, grad, indices) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> grad = Tensor(np.random.rand(1, 1, 1).astype(np.float32)) >>> indices = Tensor([0], mstype.int32) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[1, 1, 1], dtype=Float32, value= [[[1.00000000e+00]]]), Tensor(shape=[1, 1, 1], dtype=Float32, value= [[[1.00000000e+00]]]))
-
class
tinyms.primitives.SparseApplyAdagradV2(*args, **kwargs)[source]¶ Updates relevant entries according to the adagrad scheme, one more epsilon attribute than SparseApplyAdagrad.
\[accum += grad * grad\]\[var -= lr * grad * \frac{1}{\sqrt{accum} + \epsilon}\]Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
- Inputs:
var (Parameter) - Variable to be updated. The data type must be float16 or float32.
accum (Parameter) - Accumulation to be updated. The shape and data type must be the same as var.
grad (Tensor) - Gradient. The shape must be the same as var’s shape except the first dimension. Gradients has the same data type as var.
indices (Tensor) - A vector of indices into the first dimension of var and accum. The shape of indices must be the same as grad in first dimension, the type must be int32.
- Outputs:
Tuple of 2 tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as accum.
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.sparse_apply_adagrad_v2 = ops.SparseApplyAdagradV2(lr=1e-8, epsilon=1e-6) ... self.var = Parameter(Tensor(np.ones([1, 1, 1]).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.ones([1, 1, 1]).astype(np.float32)), name="accum") ... ... def construct(self, grad, indices): ... out = self.sparse_apply_adagrad_v2(self.var, self.accum, grad, indices) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> grad = Tensor(np.random.rand(1, 1, 1).astype(np.float32)) >>> indices = Tensor([0], mstype.int32) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[1, 1, 1], dtype=Float32, value= [[[1.00000000e+00]]]), Tensor(shape=[1, 1, 1], dtype=Float32, value= [[[1.30119634e+00]]]))
-
class
tinyms.primitives.SparseApplyFtrl(*args, **kwargs)[source]¶ Updates relevant entries according to the FTRL-proximal scheme.
All of inputs except indices comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
lr (float) – The learning rate value, must be positive.
l1 (float) – l1 regularization strength, must be greater than or equal to zero.
l2 (float) – l2 regularization strength, must be greater than or equal to zero.
lr_power (float) – Learning rate power controls how the learning rate decreases during training, must be less than or equal to zero. Use fixed learning rate if lr_power is zero.
use_locking (bool) – Use locks for updating operation if true . Default: False.
- Inputs:
var (Parameter) - The variable to be updated. The data type must be float16 or float32.
accum (Parameter) - The accumulation to be updated, must be same data type and shape as var.
linear (Parameter) - the linear coefficient to be updated, must be the same data type and shape as var.
grad (Tensor) - A tensor of the same type as var, for the gradient.
indices (Tensor) - A tensor of indices in the first dimension of var and accum. The shape of indices must be the same as grad in the first dimension. If there are duplicates in indices, the behavior is undefined. The type must be int32 or int64.
- Outputs:
var (Tensor) - Tensor, has the same shape and data type as var.
accum (Tensor) - Tensor, has the same shape and data type as accum.
linear (Tensor) - Tensor, has the same shape and data type as linear.
- Supported Platforms:
AscendGPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Parameter >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class SparseApplyFtrlNet(nn.Cell): ... def __init__(self): ... super(SparseApplyFtrlNet, self).__init__() ... self.sparse_apply_ftrl = ops.SparseApplyFtrl(lr=0.01, l1=0.0, l2=0.0, lr_power=-0.5) ... self.var = Parameter(Tensor(np.random.rand(1, 1).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(1, 1).astype(np.float32)), name="accum") ... self.linear = Parameter(Tensor(np.random.rand(1, 1).astype(np.float32)), name="linear") ... ... def construct(self, grad, indices): ... out = self.sparse_apply_ftrl(self.var, self.accum, self.linear, grad, indices) ... return out ... >>> np.random.seed(0) >>> net = SparseApplyFtrlNet() >>> grad = Tensor(np.random.rand(1, 1).astype(np.float32)) >>> indices = Tensor(np.ones([1]), mindspore.int32) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[1, 1], dtype=Float32, value= [[5.48813522e-01]]), Tensor(shape=[1, 1], dtype=Float32, value= [[7.15189338e-01]]), Tensor(shape=[1, 1], dtype=Float32, value= [[6.02763355e-01]]))
-
class
tinyms.primitives.SparseApplyFtrlV2(*args, **kwargs)[source]¶ Updates relevant entries according to the FTRL-proximal scheme. This class has one more attribute, named l2_shrinkage, than class SparseApplyFtrl.
All of inputs except indices comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
lr (float) – The learning rate value, must be positive.
l1 (float) – l1 regularization strength, must be greater than or equal to zero.
l2 (float) – l2 regularization strength, must be greater than or equal to zero.
l2_shrinkage (float) – L2 shrinkage regularization.
lr_power (float) – Learning rate power controls how the learning rate decreases during training, must be less than or equal to zero. Use fixed learning rate if lr_power is zero.
use_locking (bool) – If True, the var and accumulation tensors will be protected from being updated. Default: False.
- Inputs:
var (Parameter) - The variable to be updated. The data type must be float16 or float32.
accum (Parameter) - The accumulation to be updated, must be same data type and shape as var.
linear (Parameter) - the linear coefficient to be updated, must be same data type and shape as var.
grad (Tensor) - A tensor of the same type as var, for the gradient.
indices (Tensor) - A vector of indices in the first dimension of var and accum. The shape of indices must be the same as grad in the first dimension. The type must be int32.
- Outputs:
Tuple of 3 Tensor, the updated parameters.
var (Tensor) - Tensor, has the same shape and data type as var.
accum (Tensor) - Tensor, has the same shape and data type as accum.
linear (Tensor) - Tensor, has the same shape and data type as linear.
- Supported Platforms:
Ascend
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Parameter >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class SparseApplyFtrlV2Net(nn.Cell): ... def __init__(self): ... super(SparseApplyFtrlV2Net, self).__init__() ... self.sparse_apply_ftrl_v2 = ops.SparseApplyFtrlV2(lr=0.01, l1=0.0, l2=0.0, ... l2_shrinkage=0.0, lr_power=-0.5) ... self.var = Parameter(Tensor(np.random.rand(1, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(1, 2).astype(np.float32)), name="accum") ... self.linear = Parameter(Tensor(np.random.rand(1, 2).astype(np.float32)), name="linear") ... ... def construct(self, grad, indices): ... out = self.sparse_apply_ftrl_v2(self.var, self.accum, self.linear, grad, indices) ... return out ... >>> np.random.seed(0) >>> net = SparseApplyFtrlV2Net() >>> grad = Tensor(np.random.rand(1, 2).astype(np.float32)) >>> indices = Tensor(np.ones([1]), mindspore.int32) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[1, 2], dtype=Float32, value= [[ 5.48813522e-01, 7.15189338e-01]]), Tensor(shape=[1, 2], dtype=Float32, value= [[ 6.02763355e-01, 5.44883192e-01]]), Tensor(shape=[1, 2], dtype=Float32, value= [[ 4.23654795e-01, 6.45894110e-01]]))
-
class
tinyms.primitives.SparseApplyProximalAdagrad(*args, **kwargs)[source]¶ Updates relevant entries according to the proximal adagrad algorithm. Compared with ApplyProximalAdagrad, an additional index tensor is input.
\[accum += grad * grad\]\[\text{prox_v} = var - lr * grad * \frac{1}{\sqrt{accum}}\]\[var = \frac{sign(\text{prox_v})}{1 + lr * l2} * \max(\left| \text{prox_v} \right| - lr * l1, 0)\]Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – If true, the var and accum tensors will be protected from being updated. Default: False.
- Inputs:
var (Parameter) - Variable tensor to be updated. The data type must be float16 or float32.
accum (Parameter) - Variable tensor to be updated, has the same dtype as var.
lr (Union[Number, Tensor]) - The learning rate value, must be a float number or a scalar tensor with float16 or float32 data type.
l1 (Union[Number, Tensor]) - l1 regularization strength, must be a float number or a scalar tensor with float16 or float32 data type.
l2 (Union[Number, Tensor]) - l2 regularization strength, must be a float number or a scalar tensor with float16 or float32 data type..
grad (Tensor) - A tensor of the same type as var, for the gradient.
indices (Tensor) - A tensor of indices in the first dimension of var and accum. If there are duplicates in indices, the behavior is undefined. Must be one of the following types: int32, int64.
- Outputs:
Tuple of 2 tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as accum.
- Supported Platforms:
AscendGPU
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.sparse_apply_proximal_adagrad = ops.SparseApplyProximalAdagrad() ... self.var = Parameter(Tensor(np.random.rand(1, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(1, 2).astype(np.float32)), name="accum") ... self.lr = 0.01 ... self.l1 = 0.0 ... self.l2 = 0.0 ... def construct(self, grad, indices): ... out = self.sparse_apply_proximal_adagrad(self.var, self.accum, self.lr, self.l1, ... self.l2, grad, indices) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> grad = Tensor(np.random.rand(1, 2).astype(np.float32)) >>> indices = Tensor(np.ones((1,), np.int32)) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[1, 2], dtype=Float32, value= [[ 5.48813522e-01, 7.15189338e-01]]), Tensor(shape=[1, 2], dtype=Float32, value= [[ 6.02763355e-01, 5.44883192e-01]]))
-
class
tinyms.primitives.SparseGatherV2(*args, **kwargs)[source]¶ Returns a slice of input tensor based on the specified indices and axis.
- Inputs:
input_params (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). The original Tensor.
input_indices (Tensor) - The shape of tensor is \((y_1, y_2, ..., y_S)\). Specifies the indices of elements of the original Tensor, must be in the range [0, input_param.shape[axis]).
axis (int) - Specifies the dimension index to gather indices.
- Outputs:
Tensor, the shape of tensor is \((z_1, z_2, ..., z_N)\).
- Supported Platforms:
AscendGPU
Examples
>>> input_params = Tensor(np.array([[1, 2, 7, 42], [3, 4, 54, 22], [2, 2, 55, 3]]), mindspore.float32) >>> input_indices = Tensor(np.array([1, 2]), mindspore.int32) >>> axis = 1 >>> out = ops.SparseGatherV2()(input_params, input_indices, axis) >>> print(out) [[2. 7.] [4. 54.] [2. 55.]]
-
class
tinyms.primitives.SparseSoftmaxCrossEntropyWithLogits(*args, **kwargs)[source]¶ Computes the softmax cross-entropy value between logits and sparse encoding labels.
Note
Sets input logits as X, input label as Y, output as loss. Then,
\[p_{ij} = softmax(X_{ij}) = \frac{\exp(x_i)}{\sum_{j = 0}^{N-1}\exp(x_j)}\]\[loss_{ij} = \begin{cases} -ln(p_{ij}), &j = y_i \cr -ln(1 - p_{ij}), & j \neq y_i \end{cases}\]\[loss = \sum_{ij} loss_{ij}\]- Parameters
is_grad (bool) – If true, this operation returns the computed gradient. Default: False.
- Inputs:
logits (Tensor) - Input logits, with shape \((N, C)\). Data type must be float16 or float32.
labels (Tensor) - Ground truth labels, with shape \((N)\). Data type must be int32 or int64.
- Outputs:
Tensor, if is_grad is False, the output tensor is the value of loss which is a scalar tensor; if is_grad is True, the output tensor is the gradient of input with the same shape as logits.
- Supported Platforms:
GPUCPU
Examples
Please refer to the usage in nn.SoftmaxCrossEntropyWithLogits source code.
-
class
tinyms.primitives.SparseToDense(*args, **kwargs)[source]¶ Converts a sparse representation into a dense tensor.
- Inputs:
indices (Tensor) - The indices of sparse representation.
values (Tensor) - Values corresponding to each row of indices.
dense_shape (tuple) - An int tuple which specifies the shape of dense tensor.
- Returns
Tensor, the shape of tensor is dense_shape.
Examples
>>> indices = Tensor([[0, 1], [1, 2]]) >>> values = Tensor([1, 2], dtype=ms.float32) >>> dense_shape = (3, 4) >>> out = ops.SparseToDense()(indices, values, dense_shape)
-
class
tinyms.primitives.Split(*args, **kwargs)[source]¶ Splits the input tensor into output_num of tensors along the given axis and output numbers.
- Parameters
- Raises
ValueError – If axis is out of the range [-len(input_x.shape), len(input_x.shape)), or if the output_num is less than or equal to 0.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
tuple[Tensor], the shape of each output tensor is the same, which is \((y_1, y_2, ..., y_S)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> split = ops.Split(1, 2) >>> x = Tensor(np.array([[1, 1, 1, 1], [2, 2, 2, 2]]), mindspore.int32) >>> output = split(x) >>> print(output) (Tensor(shape=[2, 2], dtype=Int32, value= [[1, 1], [2, 2]]), Tensor(shape=[2, 2], dtype=Int32, value= [[1, 1], [2, 2]]))
-
class
tinyms.primitives.Sqrt(*args, **kwargs)[source]¶ Returns square root of a tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor whose dtype is number.
- Outputs:
Tensor, has the same shape as the input_x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 4.0, 9.0]), mindspore.float32) >>> sqrt = ops.Sqrt() >>> output = sqrt(input_x) >>> print(output) [1. 2. 3.]
-
class
tinyms.primitives.Square(*args, **kwargs)[source]¶ Returns square of a tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor whose dtype is number.
- Outputs:
Tensor, has the same shape and dtype as the input_x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) >>> square = ops.Square() >>> output = square(input_x) >>> print(output) [1. 4. 9.]
-
class
tinyms.primitives.SquareSumAll(*args, **kwargs)[source]¶ Returns the square sum of a tensor element-wise
- Inputs:
input_x1 (Tensor) - The input tensor. The data type must be float16 or float32.
input_x2 (Tensor) - The input tensor has the same type and shape as the input_x1.
Note
SquareSumAll only supports float16 and float32 data type.
- Outputs:
output_y1 (Tensor) - The same type as the input_x1.
output_y2 (Tensor) - The same type as the input_x1.
- Supported Platforms:
Ascend
Examples
>>> input_x1 = Tensor(np.array([0, 0, 2, 0]), mindspore.float32) >>> input_x2 = Tensor(np.array([0, 0, 2, 4]), mindspore.float32) >>> square_sum_all = ops.SquareSumAll() >>> output = square_sum_all(input_x1, input_x2) >>> print(output) (Tensor(shape=[], dtype=Float32, value= 4), Tensor(shape=[], dtype=Float32, value= 20))
-
class
tinyms.primitives.SquaredDifference(*args, **kwargs)[source]¶ Subtracts the second input tensor from the first input tensor element-wise and returns square of it.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number, or a bool, or a tensor whose data type is float16, float32, int32 or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number, or a bool when the first input is a tensor or a tensor whose data type is float16, float32, int32 or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) >>> input_y = Tensor(np.array([2.0, 4.0, 6.0]), mindspore.float32) >>> squared_difference = ops.SquaredDifference() >>> output = squared_difference(input_x, input_y) >>> print(output) [1. 4. 9.]
-
class
tinyms.primitives.Squeeze(*args, **kwargs)[source]¶ Returns a tensor with the same type but dimensions of 1 are removed based on axis.
Note
The dimension index starts at 0 and must be in the range [-input.ndim, input.ndim.
- Raises
ValueError – If the corresponding dimension of the specified axis does not equal to 1.
- Parameters
axis (Union[int, tuple(int)]) – Specifies the dimension indexes of shape to be removed, which will remove all the dimensions that are equal to 1. If specified, it must be int32 or int64. Default: (), an empty tuple.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, the shape of tensor is \((x_1, x_2, ..., x_S)\).
- Supported Platforms:
AscendGPU
Examples
>>> input_tensor = Tensor(np.ones(shape=[3, 2, 1]), mindspore.float32) >>> squeeze = ops.Squeeze(2) >>> output = squeeze(input_tensor) >>> print(output) [[1. 1.] [1. 1.] [1. 1.]]
-
class
tinyms.primitives.Stack(*args, **kwargs)[source]¶ Stacks a list of tensors in specified axis.
Stacks the list of input tensors with the same rank R, output is a tensor of rank (R+1).
Given input tensors of shape \((x_1, x_2, ..., x_R)\). Set the number of input tensors as N. If \(0 \le axis\), the shape of the output tensor is \((x_1, x_2, ..., x_{axis}, N, x_{axis+1}, ..., x_R)\).
- Parameters
axis (int) – Dimension to stack. Default: 0. Negative values wrap around. The range is [-(R+1), R+1).
- Inputs:
input_x (Union[tuple, list]) - A Tuple or list of Tensor objects with the same shape and type.
- Outputs:
Tensor. A stacked Tensor with the same type as input_x.
- Raises
TypeError – If the data types of elements in input_x are not the same.
ValueError – If the length of input_x is not greater than 1; or if axis is out of the range [-(R+1), R+1); or if the shapes of elements in input_x are not the same.
- Supported Platforms:
AscendGPU
Examples
>>> data1 = Tensor(np.array([0, 1]).astype(np.float32)) >>> data2 = Tensor(np.array([2, 3]).astype(np.float32)) >>> stack = ops.Stack() >>> output = stack([data1, data2]) >>> print(output) [[0. 1.] [2. 3.]]
-
class
tinyms.primitives.StandardLaplace(*args, **kwargs)[source]¶ Generates random numbers according to the Laplace random number distribution (mean=0, lambda=1). It is defined as:
\[\text{f}(x;0,1) = \frac{1}{2}\exp(-|x|),\]- Inputs:
shape (tuple) - The shape of random tensor to be generated. Only constant value is allowed.
- Outputs:
Tensor. The shape that the input ‘shape’ denotes. The dtype is float32.
- Supported Platforms:
Ascend
Examples
>>> shape = (4, 16) >>> stdlaplace = ops.StandardLaplace(seed=2) >>> output = stdlaplace(shape) >>> result = output.shape >>> print(result) (4, 16)
-
class
tinyms.primitives.StandardNormal(*args, **kwargs)[source]¶ Generates random numbers according to the standard Normal (or Gaussian) random number distribution.
- Parameters
- Inputs:
shape (tuple) - The shape of random tensor to be generated. Only constant value is allowed.
- Outputs:
Tensor. The shape is the same as the input shape. The dtype is float32.
- Supported Platforms:
AscendGPUCPU
Examples
>>> shape = (4, 16) >>> stdnormal = ops.StandardNormal(seed=2) >>> output = stdnormal(shape) >>> result = output.shape >>> print(result) (4, 16)
-
class
tinyms.primitives.StridedSlice(*args, **kwargs)[source]¶ Extracts a strided slice of a tensor.
Given an input tensor, this operation inserts a dimension of length 1 at the dimension. This operation extracts a fragment of size (end-begin)/stride from the given ‘input_tensor’. Starting from the beginning position, the fragment continues adding stride to the index until all dimensions are not less than the ending position.
Note
The stride may be negative value, which causes reverse slicing. The shape of begin, end and strides must be the same.
- Parameters
- Inputs:
input_x (Tensor) - The input Tensor.
begin (tuple[int]) - A tuple which represents the location where to start. Only constant value is allowed.
end (tuple[int]) - A tuple or which represents the maximum location where to end. Only constant value is allowed.
strides (tuple[int]) - A tuple which represents the stride is continuously added before reaching the maximum location. Only constant value is allowed.
- Outputs:
Tensor. The output is explained by following example.
In the 0th dimension, begin is 1, end is 2, and strides is 1, because \(1+1=2\geq2\), the interval is \([1,2)\). Thus, return the element with \(index = 1\) in 0th dimension, i.e., [[3, 3, 3], [4, 4, 4]].
In the 1st dimension, similarly, the interval is \([0,1)\). Based on the return value of the 0th dimension, return the element with \(index = 0\), i.e., [3, 3, 3].
In the 2nd dimension, similarly, the interval is \([0,3)\). Based on the return value of the 1st dimension, return the element with \(index = 0,1,2\), i.e., [3, 3, 3].
Finally, the output is [3, 3, 3].
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor([[[1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]], ... [[5, 5, 5], [6, 6, 6]]], mindspore.float32) >>> slice = ops.StridedSlice() >>> output = slice(input_x, (1, 0, 0), (2, 1, 3), (1, 1, 1)) >>> print(output) [[[3. 3. 3.]]]
-
class
tinyms.primitives.Sub(*args, **kwargs)[source]¶ Subtracts the second input tensor from the first input tensor element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number, or a bool, or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number, or a bool when the first input is a tensor, or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([4, 5, 6]), mindspore.int32) >>> sub = ops.Sub() >>> output = sub(input_x, input_y) >>> print(output) [-3 -3 -3]
-
class
tinyms.primitives.Tan(*args, **kwargs)[source]¶ Computes tangent of input_x element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). Data type must be float16, float32 or int32.
- Outputs:
Tensor, has the same shape as input_x.
- Supported Platforms:
Ascend
Examples
>>> tan = ops.Tan() >>> input_x = Tensor(np.array([-1.0, 0.0, 1.0]), mindspore.float32) >>> output = tan(input_x) >>> print(output) [-1.5574081 0. 1.5574081]
-
class
tinyms.primitives.Tanh(*args, **kwargs)[source]¶ Tanh activation function.
Computes hyperbolic tangent of input element-wise. The Tanh function is defined as:
\[tanh(x_i) = \frac{\exp(x_i) - \exp(-x_i)}{\exp(x_i) + \exp(-x_i)} = \frac{\exp(2x_i) - 1}{\exp(2x_i) + 1},\]where \(x_i\) is an element of the input Tensor.
- Inputs:
input_x (Tensor) - The input of Tanh.
- Outputs:
Tensor, with the same type and shape as the input_x.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3, 4, 5]), mindspore.float32) >>> tanh = ops.Tanh() >>> output = tanh(input_x) >>> print(output) [0.7615941 0.9640276 0.9950547 0.9993293 0.9999092]
-
tinyms.primitives.TensorAdd()[source]¶ Adds two input tensors element-wise.
The usage of TensorAdd is deprecated. Please use Add.
-
class
tinyms.primitives.TensorScatterUpdate(*args, **kwargs)[source]¶ Updates tensor values using given values, along with the input indices.
- Inputs:
input_x (Tensor) - The target tensor. The dimension of input_x must be equal to indices.shape[-1].
indices (Tensor) - The index of input tensor whose data type is int32.
update (Tensor) - The tensor to update the input tensor, has the same type as input, and update.shape = indices.shape[:-1] + input_x.shape[indices.shape[-1]:].
- Outputs:
Tensor, has the same shape and type as input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]), mindspore.float32) >>> indices = Tensor(np.array([[0, 0], [1, 1]]), mindspore.int32) >>> update = Tensor(np.array([1.0, 2.2]), mindspore.float32) >>> op = ops.TensorScatterUpdate() >>> output = op(input_x, indices, update) >>> print(output) [[ 1. 0.3 3.6] [ 0.4 2.2 -3.2]]
-
class
tinyms.primitives.TensorSummary(*args, **kwargs)[source]¶ Outputs a tensor to a protocol buffer through a tensor summary operator.
- Inputs:
name (str) - The name of the input variable.
value (Tensor) - The value of tensor, and the rank of tensor must be greater than 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> class SummaryDemo(nn.Cell): ... def __init__(self,): ... super(SummaryDemo, self).__init__() ... self.summary = ops.TensorSummary() ... self.add = ops.Add() ... ... def construct(self, x, y): ... x = self.add(x, y) ... name = "x" ... self.summary(name, x) ... return x ...
-
class
tinyms.primitives.Tile(*args, **kwargs)[source]¶ Replicates a tensor with given multiples times.
Creates a new tensor by replicating input multiples times. The dimension of output tensor is the larger of the input tensor dimension and the length of multiples.
- Inputs:
input_x (Tensor) - 1-D or higher Tensor. Set the shape of input tensor as \((x_1, x_2, ..., x_S)\).
multiples (tuple[int]) - The input tuple is constructed by multiple integers, i.e., \((y_1, y_2, ..., y_S)\). The length of multiples cannot be smaller than the length of the shape of input_x. Only constant value is allowed.
- Outputs:
Tensor, has the same data type as the input_x.
If the length of multiples is the same as the length of shape of input_x, then the shape of their corresponding positions can be multiplied, and the shape of Outputs is \((x_1*y_1, x_2*y_2, ..., x_S*y_R)\).
If the length of multiples is larger than the length of shape of input_x, fill in multiple 1 in the length of the shape of input_x until their lengths are consistent. Such as set the shape of input_x as \((1, ..., x_1, x_2, ..., x_S)\), then the shape of their corresponding positions can be multiplied, and the shape of Outputs is \((1*y_1, ..., x_S*y_R)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> tile = ops.Tile() >>> input_x = Tensor(np.array([[1, 2], [3, 4]]), mindspore.float32) >>> multiples = (2, 3) >>> output = tile(input_x, multiples) >>> print(output) [[1. 2. 1. 2. 1. 2.] [3. 4. 3. 4. 3. 4.] [1. 2. 1. 2. 1. 2.] [3. 4. 3. 4. 3. 4.]]
-
class
tinyms.primitives.TopK(*args, **kwargs)[source]¶ Finds values and indices of the k largest entries along the last dimension.
- Parameters
sorted (bool) – If true, the obtained elements will be sorted by the values in descending order. Default: False.
- Inputs:
input_x (Tensor) - Input to be computed, data type must be float16, float32 or int32.
k (int) - The number of top elements to be computed along the last dimension, constant input is needed.
- Outputs:
Tuple of 2 tensors, the values and the indices.
values (Tensor) - The k largest elements in each slice of the last dimensional.
indices (Tensor) - The indices of values within the last dimension of input.
- Supported Platforms:
AscendGPU
Examples
>>> topk = ops.TopK(sorted=True) >>> input_x = Tensor([1, 2, 3, 4, 5], mindspore.float16) >>> k = 3 >>> values, indices = topk(input_x, k) >>> print((values, indices)) (Tensor(shape=[3], dtype=Float16, value= [ 5.0000e+00, 4.0000e+00, 3.0000e+00]), Tensor(shape=[3], dtype=Int32, value= [4, 3, 2]))
-
class
tinyms.primitives.Transpose(*args, **kwargs)[source]¶ Permutes the dimensions of the input tensor according to input permutation.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
input_perm (tuple[int]) - The permutation to be converted. The input tuple is constructed by multiple indexes. The length of input_perm and the shape of input_x must be the same. Only constant value is allowed. Must be in the range [0, rank(input_x)).
- Outputs:
Tensor, the type of output tensor is the same as input_x and the shape of output tensor is decided by the shape of input_x and the value of input_perm.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]]), mindspore.float32) >>> perm = (0, 2, 1) >>> transpose = ops.Transpose() >>> output = transpose(input_tensor, perm) >>> print(output) [[[ 1. 4.] [ 2. 5.] [ 3. 6.]] [[ 7. 10.] [ 8. 11.] [ 9. 12.]]]
-
class
tinyms.primitives.TruncateDiv(*args, **kwargs)[source]¶ Divides the first input tensor by the second input tensor element-wise for integer types, negative numbers will round fractional quantities towards zero.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number, or a bool, or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number, or a bool when the first input is a tensor, or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([2, 4, -1]), mindspore.int32) >>> input_y = Tensor(np.array([3, 3, 3]), mindspore.int32) >>> truncate_div = ops.TruncateDiv() >>> output = truncate_div(input_x, input_y) >>> print(output) [0 1 0]
-
class
tinyms.primitives.TruncateMod(*args, **kwargs)[source]¶ Returns the remainder of division element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number, or a bool, or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number, or a bool when the first input is a tensor, or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([2, 4, -1]), mindspore.int32) >>> input_y = Tensor(np.array([3, 3, 3]), mindspore.int32) >>> truncate_mod = ops.TruncateMod() >>> output = truncate_mod(input_x, input_y) >>> print(output) [ 2 1 -1]
-
class
tinyms.primitives.TruncatedNormal(*args, **kwargs)[source]¶ Returns a tensor of the specified shape filled with truncated normal values.
The generated values follow a normal distribution.
- Parameters
seed (int) – A integer number used to create random seed. Default: 0.
dtype (
mindspore.dtype) – Data type. Default: mindspore.float32.
- Inputs:
shape (tuple[int]) - The shape of the output tensor, is a tuple of positive integer.
- Outputs:
Tensor, the data type of output tensor is the same as attribute dtype.
Examples
>>> shape = (1, 2, 3) >>> truncated_normal = ops.TruncatedNormal() >>> output = truncated_normal(shape)
-
class
tinyms.primitives.TupleToArray(*args, **kwargs)[source]¶ Converts a tuple to a tensor.
If the type of the first number in the tuple is integer, the data type of the output tensor is int. Otherwise, the data type of the output tensor is float.
- Inputs:
input_x (tuple) - A tuple of numbers. These numbers have the same type. Only constant value is allowed.
- Outputs:
Tensor, if the input tuple contains N numbers, then the shape of the output tensor is (N,).
- Supported Platforms:
AscendGPUCPU
Examples
>>> type = ops.TupleToArray()((1,2,3)) >>> print(type) [1 2 3]
-
class
tinyms.primitives.UniformCandidateSampler(*args, **kwargs)[source]¶ Uniform candidate sampler.
This function samples a set of classes(sampled_candidates) from [0, range_max-1] based on uniform distribution. If unique=True, candidates are drawn without replacement, else unique=False with replacement.
- Parameters
num_true (int) – The number of target classes in each training example.
num_sampled (int) – The number of classes to randomly sample. The sampled_candidates will have a shape of num_sampled. If unique=True, num_sampled must be less than or equal to range_max.
unique (bool) – Whether all sampled classes in a batch are unique.
range_max (int) – The number of possible classes, must be non-negative.
seed (int) – Used for random number generation, must be non-negative. If seed has a value of 0, seed will be replaced with a randomly generated value. Default: 0.
remove_accidental_hits (bool) – Whether accidental hit is removed. Default: False.
- Inputs:
true_classes (Tensor) - A Tensor. The target classes with a Tensor shape of (batch_size, num_true).
- Outputs:
sampled_candidates (Tensor) - The sampled_candidates is independent of the true classes. Shape: (num_sampled, ).
true_expected_count (Tensor) - The expected counts under the sampling distribution of each of true_classes. Shape: (batch_size, num_true).
sampled_expected_count (Tensor) - The expected counts under the sampling distribution of each of sampled_candidates. Shape: (num_sampled, ).
- Supported Platforms:
GPU
Examples
>>> sampler = ops.UniformCandidateSampler(1, 3, False, 4) >>> output1, output2, output3 = sampler(Tensor(np.array([[1], [3], [4], [6], [3]], dtype=np.int32))) >>> print(output1, output2, output3) [1, 1, 3], [[0.75], [0.75], [0.75], [0.75], [0.75]], [0.75, 0.75, 0.75]
-
class
tinyms.primitives.UniformInt(*args, **kwargs)[source]¶ Produces random integer values i, uniformly distributed on the closed interval [minval, maxval), that is, distributed according to the discrete probability function:
\[\text{P}(i|a,b) = \frac{1}{b-a+1},\]Note
The number in tensor minval must be strictly less than maxval at any position after broadcasting.
- Parameters
- Inputs:
shape (tuple) - The shape of random tensor to be generated. Only constant value is allowed.
minval (Tensor) - The distribution parameter, a. It defines the minimum possibly generated value, with int32 data type. Only one number is supported.
maxval (Tensor) - The distribution parameter, b. It defines the maximum possibly generated value, with int32 data type. Only one number is supported.
- Outputs:
Tensor. The shape is the same as the input ‘shape’, and the data type is int32.
- Supported Platforms:
AscendGPU
Examples
>>> shape = (2, 4) >>> minval = Tensor(1, mstype.int32) >>> maxval = Tensor(5, mstype.int32) >>> uniform_int = ops.UniformInt(seed=10) >>> output = uniform_int(shape, minval, maxval) >>> result = output.shape >>> print(result) (2, 4)
-
class
tinyms.primitives.UniformReal(*args, **kwargs)[source]¶ Produces random floating-point values i, uniformly distributed to the interval [0, 1).
- Parameters
- Inputs:
shape (tuple) - The shape of random tensor to be generated. Only constant value is allowed.
- Outputs:
Tensor. The shape that the input ‘shape’ denotes. The dtype is float32.
- Supported Platforms:
AscendGPU
Examples
>>> shape = (2, 2) >>> uniformreal = ops.UniformReal(seed=2) >>> output = uniformreal(shape) >>> result = output.shape >>> print(result) (2, 2)
-
class
tinyms.primitives.Unique(*args, **kwargs)[source]¶ Returns the unique elements of input tensor and also return a tensor containing the index of each value of input tensor corresponding to the output unique tensor.
- Inputs:
x (Tensor) - The input tensor.
- Outputs:
Tuple, containing Tensor objects (y, idx), `y is a tensor with the same type as x, and contains the unique elements in x, sorted in ascending order. idx is a tensor containing indices of elements in the input corresponding to the output tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> x = Tensor(np.array([1, 2, 5, 2]), mindspore.int32) >>> output = ops.Unique()(x) >>> print(output) (Tensor(shape=[3], dtype=Int32, value= [1, 2, 5]), Tensor(shape=[4], dtype=Int32, value= [0, 1, 2, 1])) >>> >>> # note that for GPU, this operator must be wrapped inside a model, and executed in graph mode. >>> class UniqueNet(nn.Cell): ... def __init__(self): ... super(UniqueNet, self).__init__() ... self.unique_op = P.Unique() ... ... def construct(self, x): ... output, indices = self.unique_op(x) ... return output, indices ... >>> x = Tensor(np.array([1, 2, 5, 2]), mindspore.int32) >>> context.set_context(mode=context.GRAPH_MODE, device_target="GPU") >>> net = UniqueNet() >>> output = net(x) >>> print(output) (Tensor(shape=[3], dtype=Int32, value= [1, 2, 5]), Tensor(shape=[4], dtype=Int32, value= [0, 1, 2, 1]))
-
class
tinyms.primitives.UniqueWithPad(*args, **kwargs)[source]¶ Returns unique elements and relative indexes in 1-D tensor, filled with padding num.
- Inputs:
x (Tensor) - The tensor need to be unique. Must be 1-D vector with types: int32, int64.
pad_num (int) - Pad num.
- Outputs:
tuple(Tensor), tuple of 2 tensors, y and idx. - y (Tensor) - The unique elements filled with pad_num, the shape and type same as x. - idx (Tensor) - The index of each value of x in the unique output y, the shape and type same as x.
- Supported Platforms:
AscendCPU
Examples
>>> x = Tensor(np.array([1, 1, 5, 5, 4, 4, 3, 3, 2, 2,]), mindspore.int32) >>> pad_num = 8 >>> output = ops.UniqueWithPad()(x, pad_num) >>> print(output) (Tensor(shape=[10], dtype=Int32, value= [1, 5, 4, 3, 2, 8, 8, 8, 8, 8]), Tensor(shape=[10], dtype=Int32, value= [0, 0, 1, 1, 2, 2, 3, 3, 4, 4]))
-
tinyms.primitives.Unpack(axis=0)[source]¶ Unpacks tensor in specified axis.
The usage of Unpack is deprecated. Please use Unstack.
-
class
tinyms.primitives.UnsortedSegmentMax(*args, **kwargs)[source]¶ Computes the maximum along segments of a tensor.
- Inputs:
input_x (Tensor) - The shape is \((x_1, x_2, ..., x_R)\). The data type must be float16, float32 or int32.
segment_ids (Tensor) - A 1-D tensor whose shape is \((x_1)\), the value must be >= 0. The data type must be int32.
num_segments (int) - The value specifies the number of distinct segment_ids.
Note
If the segment_id i is absent in the segment_ids, then output[i] will be filled with the minimum value of the input_x’s type.
- Outputs:
Tensor, set the number of num_segments as N, the shape is \((N, x_2, ..., x_R)\).
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[1, 2, 3], [4, 5, 6], [4, 2, 1]]).astype(np.float32)) >>> segment_ids = Tensor(np.array([0, 1, 1]).astype(np.int32)) >>> num_segments = 2 >>> unsorted_segment_max = ops.UnsortedSegmentMax() >>> output = unsorted_segment_max(input_x, segment_ids, num_segments) >>> print(output) [[1. 2. 3.] [4. 5. 6.]]
-
class
tinyms.primitives.UnsortedSegmentMin(*args, **kwargs)[source]¶ Computes the minimum of a tensor along segments.
- Inputs:
input_x (Tensor) - The shape is \((x_1, x_2, ..., x_R)\). The data type must be float16, float32 or int32.
segment_ids (Tensor) - A 1-D tensor whose shape is \((x_1)\), the value must be >= 0. The data type must be int32.
num_segments (int) - The value specifies the number of distinct segment_ids.
Note
If the segment_id i is absent in the segment_ids, then output[i] will be filled with the maximum value of the input_x’s type.
- Outputs:
Tensor, set the number of num_segments as N, the shape is \((N, x_2, ..., x_R)\).
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[1, 2, 3], [4, 5, 6], [4, 2, 1]]).astype(np.float32)) >>> segment_ids = Tensor(np.array([0, 1, 1]).astype(np.int32)) >>> num_segments = 2 >>> unsorted_segment_min = ops.UnsortedSegmentMin() >>> output = unsorted_segment_min(input_x, segment_ids, num_segments) >>> print(output) [[1. 2. 3.] [4. 2. 1.]]
-
class
tinyms.primitives.UnsortedSegmentProd(*args, **kwargs)[source]¶ Computes the product of a tensor along segments.
- Inputs:
input_x (Tensor) - The shape is \((x_1, x_2, ..., x_R)\). With float16, float32 or int32 data type.
segment_ids (Tensor) - A 1-D tensor whose shape is \((x_1)\), the value must be >= 0. Data type must be int32.
num_segments (int) - The value specifies the number of distinct segment_ids, must be greater than 0.
- Outputs:
Tensor, set the number of num_segments as N, the shape is \((N, x_2, ..., x_R)\).
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([[1, 2, 3], [4, 5, 6], [4, 2, 1]]).astype(np.float32)) >>> segment_ids = Tensor(np.array([0, 1, 0]).astype(np.int32)) >>> num_segments = 2 >>> unsorted_segment_prod = ops.UnsortedSegmentProd() >>> output = unsorted_segment_prod(input_x, segment_ids, num_segments) >>> print(output) [[4. 4. 3.] [4. 5. 6.]]
-
class
tinyms.primitives.UnsortedSegmentSum(*args, **kwargs)[source]¶ Computes the sum of a tensor along segments.
Calculates a tensor such that \(\text{output}[i] = \sum_{segment\_ids[j] == i} \text{data}[j, \ldots]\), where \(j\) is a tuple describing the index of element in data. segment_ids selects which elements in data to sum up. Segment_ids does not need to be sorted, and it does not need to cover all values in the entire valid value range.
Note
If the segment_id i is absent in the segment_ids, then output[i] will be filled with 0.
If the sum of the given segment_ids \(i\) is empty, then \(\text{output}[i] = 0\). If the given segment_ids is negative, the value will be ignored. ‘num_segments’ must be equal to the number of different segment_ids.
- Inputs:
input_x (Tensor) - The shape is \((x_1, x_2, ..., x_R)\).
segment_ids (Tensor) - Set the shape as \((x_1, x_2, ..., x_N)\), where 0 < N <= R. Type must be int.
num_segments (int) - Set \(z\) as num_segments.
- Outputs:
Tensor, the shape is \((z, x_{N+1}, ..., x_R)\).
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor([1, 2, 3, 4], mindspore.float32) >>> segment_ids = Tensor([0, 0, 1, 2], mindspore.int32) >>> num_segments = 4 >>> output = ops.UnsortedSegmentSum()(input_x, segment_ids, num_segments) >>> print(output) [3. 3. 4. 0.]
-
class
tinyms.primitives.Unstack(*args, **kwargs)[source]¶ Unstacks tensor in specified axis.
Unstacks a tensor of rank R along axis dimension, output tensors will have rank (R-1).
Given a tensor of shape \((x_1, x_2, ..., x_R)\). If \(0 \le axis\), the shape of tensor in output is \((x_1, x_2, ..., x_{axis}, x_{axis+2}, ..., x_R)\).
This is the opposite of pack.
- Parameters
axis (int) – Dimension along which to pack. Default: 0. Negative values wrap around. The range is [-R, R).
- Inputs:
input_x (Tensor) - The shape is \((x_1, x_2, ..., x_R)\). A tensor to be unstacked and the rank of the tensor must be greater than 0.
- Outputs:
A tuple of tensors, the shape of each objects is the same.
- Raises
ValueError – If axis is out of the range [-len(input_x.shape), len(input_x.shape)).
- Supported Platforms:
AscendGPU
Examples
>>> unstack = ops.Unstack() >>> input_x = Tensor(np.array([[1, 1, 1, 1], [2, 2, 2, 2]])) >>> output = unstack(input_x) >>> print(output) (Tensor(shape=[4], dtype=Int32, value= [1, 1, 1, 1]), Tensor(shape=[4], dtype=Int32, value= [2, 2, 2, 2]))
-
class
tinyms.primitives.Xdivy(*args, **kwargs)[source]¶ Divides the first input tensor by the second input tensor element-wise. Returns zero when x is zero.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number, or a bool, or a tensor whose data type is float16, float32 or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number, or a bool when the first input is a tensor, or a tensor whose data type is float16, float32 or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([2, 4, -1]), mindspore.float32) >>> input_y = Tensor(np.array([2, 2, 2]), mindspore.float32) >>> xdivy = ops.Xdivy() >>> output = xdivy(input_x, input_y) >>> print(output) [ 1. 2. -0.5]
-
class
tinyms.primitives.Xlogy(*args, **kwargs)[source]¶ Computes the first input tensor multiplied by the logarithm of second input tensor element-wise. Returns zero when x is zero.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is float16, float32 or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is float16, float32 or bool. The value must be positive.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([-5, 0, 4]), mindspore.float32) >>> input_y = Tensor(np.array([2, 2, 2]), mindspore.float32) >>> xlogy = ops.Xlogy() >>> output = xlogy(input_x, input_y) >>> print(output) [-3.465736 0. 2.7725887]
-
class
tinyms.primitives.Zeros(*args, **kwargs)[source]¶ Creates a tensor filled with value zeros.
Creates a tensor with shape described by the first argument and fills it with value zeros in type of the second argument.
- Inputs:
shape (Union[tuple[int], int]) - The specified shape of output tensor. Only constant positive int is allowed.
type (mindspore.dtype) - The specified type of output tensor. Only constant value is allowed.
- Outputs:
Tensor, has the same type and shape as input shape value.
- Supported Platforms:
AscendGPUCPU
Examples
>>> from mindspore.ops import operations as ops >>> zeros = ops.Zeros() >>> output = zeros((2, 2), mindspore.float32) >>> print(output) [[0. 0.] [0. 0.]]
-
class
tinyms.primitives.ZerosLike(*args, **kwargs)[source]¶ Creates a new tensor. All elements value are 0.
Returns a tensor of zeros with the same shape and data type as the input tensor.
- Inputs:
input_x (Tensor) - Input tensor.
- Outputs:
Tensor, has the same shape and data type as input_x but filled with zeros.
- Supported Platforms:
AscendGPUCPU
Examples
>>> zeroslike = ops.ZerosLike() >>> x = Tensor(np.array([[0, 1], [2, 1]]).astype(np.float32)) >>> output = zeroslike(x) >>> print(output) [[0. 0.] [0. 0.]]
tinyms.layers¶
Layer module contains pre-defined building blocks or computing units to construct neural networks.
The high-level components (Layers) used to construct the neural network.
-
class
tinyms.layers.Layer(auto_prefix=True, flags=None)[source]¶ Base class for all neural networks.
A ‘Layer’ could be a single neural network layer, such as conv2d, relu, batch_norm, etc. or a composition of cells to constructing a network.
Note
In general, the autograd algorithm will automatically generate the implementation of the gradient function, but if back-propagation(bprop) method is implemented, the gradient function will be replaced by the bprop. The bprop implementation will receive a Tensor dout containing the gradient of the loss w.r.t. the output, and a Tensor out containing the forward result. The bprop needs to compute the gradient of the loss w.r.t. the inputs, gradient of the loss w.r.t. Parameter variables are not supported currently. The bprop method must contain the self parameter.
- Parameters
auto_prefix (bool) – Recursively generate namespaces. Default: True.
Examples
>>> from tinyms import layers, primitives as P >>> >>> class MyNet(layers.Layer): ... def __init__(self): ... super(MyNet, self).__init__() ... self.relu = P.ReLU() ... ... def construct(self, x): ... return self.relu(x)
-
property
bprop_debug¶ Get whether cell custom bprop debug is enabled.
-
cast_param(param)¶ Cast parameter according to auto mix precision level in pynative mode.
- Parameters
param (Parameter) – The parameter to cast.
-
cells()¶ Returns an iterator over immediate cells.
-
cells_and_names(cells=None, name_prefix='')¶ Returns an iterator over all cells in the network.
Includes the cell’s name and itself.
- Parameters
Examples
>>> n = Net() >>> names = [] >>> for m in n.cells_and_names(): ... if m[0]: ... names.append(m[0])
-
compile_and_run(*inputs)¶ Compiles and runs cell.
- Parameters
inputs (tuple) – Input parameters.
- Returns
Object, the result of executing.
-
construct(*inputs, **kwargs)¶ Defines the computation to be performed. This method must be overridden by all subclasses.
Note
The outermost net only supports tensor inputs by default. If want to support non tensor inputs, set the property support_non_tensor_inputs to the True. Refer to the property support_non_tensor_inputs description.
- Returns
Tensor, returns the computed result.
-
exec_checkpoint_graph()¶ Executes saving checkpoint graph operation.
-
extend_repr()¶ Sets the extended representation of the Cell.
To print customized extended information, re-implement this method in your own cells.
-
generate_scope()¶ Generate the scope for each cell object in the network.
-
get_func_graph_proto()¶ Return graph binary proto.
-
get_parameters(expand=True)¶ Returns an iterator over cell parameters.
Yields parameters of this cell. If expand is True, yield parameters of this cell and all subcells.
- Parameters
expand (bool) – If true, yields parameters of this cell and all subcells. Otherwise, only yield parameters that are direct members of this cell. Default: True.
Examples
>>> net = Net() >>> parameters = [] >>> for item in net.get_parameters(): ... parameters.append(item)
-
get_scope()¶ Returns the scope of a cell object in one network.
-
init_parameters_data(auto_parallel_mode=False)¶ Initialize all parameters and replace the original saved parameters in cell.
Notes
trainable_params() and other similar interfaces may return different parameter instance after init_parameters_data, do not save these result.
- Parameters
auto_parallel_mode (bool) – If running in auto_parallel_mode.
- Returns
Dict[Parameter, Parameter], returns a dict of original parameter and replaced parameter.
-
insert_child_to_cell(child_name, child_cell)¶ Adds a child cell to the current cell with a given name.
-
insert_param_to_cell(param_name, param, check_name=True)¶ Adds a parameter to the current cell.
Inserts a parameter with given name to the cell. Please refer to the usage in source code of mindspore.nn.Cell.__setattr__.
- Parameters
- Raises
KeyError – If the name of parameter is null or contains dot.
AttributeError – If user did not call init() first.
TypeError – If the type of parameter is not Parameter.
-
load_parameter_slice(params)¶ Replace parameters with sliced tensors by parallel strategies.
Please refer to the usage in source code of mindspore.common._Executor.compile.
- Parameters
params (dict) – The parameters dictionary used for initializing the data graph.
-
name_cells()¶ Returns an iterator over all cells in the network.
Include name of the cell and cell itself.
-
property
param_prefix¶ Param prefix is the prefix of current cell’s direct child parameter.
-
parameters_and_names(name_prefix='', expand=True)¶ Returns an iterator over cell parameters.
Includes the parameter’s name and itself.
- Parameters
Examples
>>> n = Net() >>> names = [] >>> for m in n.parameters_and_names(): ... if m[0]: ... names.append(m[0])
-
parameters_dict(recurse=True)¶ Gets parameters dictionary.
Gets the parameters dictionary of this cell.
- Parameters
recurse (bool) – Whether contains the parameters of subcells. Default: True.
- Returns
OrderedDict, return parameters dictionary.
-
register_backward_hook(fn)¶ Set the cell backward hook function. Note that this function is only supported in Pynative Mode.
Note
fn must be defined as the following code. cell_name is the name of registered cell. grad_input is gradient passed to the cell. grad_output is the gradient computed and passed to the next cell or primitve, which may be modified and returned. hook_fn(cell_name, grad_input, grad_output) -> Tensor or None.
- Parameters
fn (function) – Specifies the hook function with grad as input.
-
set_auto_parallel()¶ Set the cell to auto parallel mode.
Note
If a cell needs to use the auto parallel or semi auto parallel mode for training, evaluation or prediction, this interface needs to be called by the cell.
-
set_broadcast_flag(mode=True)¶ Set the cell to data_parallel mode.
The cell can be accessed as an attribute using the given name.
- Parameters
mode (bool) – Specifies whether the model is data_parallel. Default: True.
-
set_grad(requires_grad=True)¶ Sets the cell flag for gradient.
- Parameters
requires_grad (bool) – Specifies if the net need to grad, if it is True, cell will construct backward network in pynative mode. Default: True.
-
set_parallel_input_with_inputs(*inputs)¶ Slice inputs tensors by parallel strategies, and set the sliced inputs to _parallel_input_run
- Parameters
inputs (tuple) – inputs of construct method.
-
set_param_ps(recurse=True, init_in_server=False)¶ Set whether the trainable parameters are updated by parameter server and whether the trainable parameters are initialized on server.
Note
It only works when a running task is in the parameter server mode.
-
set_train(mode=True)¶ Sets the cell to training mode.
The cell itself and all children cells will be set to training mode.
- Parameters
mode (bool) – Specifies whether the model is training. Default: True.
-
property
support_non_tensor_inputs¶ Whether support non tensor inputs in outermost net in GRAPH MODE. This property only used in forward net, and is not supported in grad net. The default value of the property is the False, that is, it does not support passing non tensor inputs to the outermost net. If you want to support, set the property to the True.
-
to_float(dst_type)¶ Add cast on all inputs of cell and child cells to run with certain float type.
If dst_type is mindspore.dtype.float16, all the inputs of Cell including input, Parameter, Tensor as const will be cast to float16. Please refer to the usage in source code of mindspore.train.amp.build_train_network.
Note
Multiple calls will overwrite.
- Parameters
dst_type (
mindspore.dtype) – Transfer Cell to Run with dst_type. dst_type can be mindspore.dtype.float16 or mindspore.dtype.float32.- Raises
ValueError – If dst_type is not float32 nor float16.
-
trainable_params(recurse=True)¶ Returns all trainable parameters.
Returns a list of all trainable parmeters.
- Parameters
recurse (bool) – Whether contains the trainable parameters of subcells. Default: True.
- Returns
List, the list of trainable parameters.
-
untrainable_params(recurse=True)¶ Returns all untrainable parameters.
Returns a list of all untrainable parameters.
- Parameters
recurse (bool) – Whether contains the untrainable parameters of subcells. Default: True.
- Returns
List, the list of untrainable parameters.
-
update_cell_prefix()¶ Update the all child cells’ self.param_prefix.
After being invoked, it can get all the cell’s children’s name prefix by ‘_param_prefix’.
-
update_cell_type(cell_type)¶ The current cell type is updated when a quantization aware training network is encountered.
After being invoked, it can set the cell type to ‘cell_type’.
Layer module contains pre-defined building blocks or computing units to construct neural networks.
The high-level components (Layers) used to construct the neural network.
-
class
tinyms.layers.SequentialLayer(*args)[source]¶ Sequential layer container.
A list of Layers will be added to it in the order they are passed in the constructor. Alternatively, an ordered dict of cells can also be passed in.
- Parameters
args (Union[list, OrderedDict]) – List of subclass of Layer.
- Raises
TypeError – If the type of the argument is not list or OrderedDict.
- Inputs:
input (Tensor) - Tensor with shape according to the first Cell in the sequence.
- Outputs:
Tensor, the output Tensor with shape depending on the input and defined sequence of Layers.
Examples
>>> import tinyms as ts >>> from tinyms.layers import SequentialLayer, Conv2d, ReLU >>> >>> seq_layer = SequentialLayer([Conv2d(3, 2, 3, pad_mode='valid', weight_init="ones"), ReLU()]) >>> x = ts.ones([1, 3, 4, 4]) >>> print(seq_layer(x)) [[[[27. 27.] [27. 27.]] [[27. 27.] [27. 27.]]]]
-
class
tinyms.layers.LayerList(*args)[source]¶ Holds Layers in a list.
LayerList can be used like a regular Python list, support ‘__getitem__’, ‘__setitem__’, ‘__delitem__’, ‘__len__’, ‘__iter__’ and ‘__iadd__’, but layers it contains are properly registered, and will be visible by all Layer methods.
- Parameters
args (list, optional) – List of subclass of Layer.
Examples
>>> from tinyms.layers import LayerList, Conv2d, BatchNorm2d, ReLU >>> >>> conv = nn.Conv2d(100, 20, 3) >>> layers = LayerList([BatchNorm2d(20)]) >>> layers.insert(0, Conv2d(100, 20, 3)) >>> layers.append(ReLU()) >>> layers LayerList< (0): Conv2d<input_channels=100, ..., bias_init=None> (1): BatchNorm2d<num_features=20, ..., moving_variance=Parameter (name=variance)> (2): ReLU<> >
-
class
tinyms.layers.Softmax(axis=-1)[source]¶ Softmax activation function.
Applies the Softmax function to an n-dimensional input Tensor.
The input is a Tensor of logits transformed with exponential function and then normalized to lie in range [0, 1] and sum up to 1.
Softmax is defined as:
\[\text{softmax}(x_{i}) = \frac{\exp(x_i)}{\sum_{j=0}^{n-1}\exp(x_j)},\]where \(x_{i}\) is the \(i\)-th slice in the given dimension of the input Tensor.
- Parameters
axis (Union[int, tuple[int]]) – The axis to apply Softmax operation, -1 means the last dimension. Default: -1.
- Inputs:
x (Tensor) - The input of Softmax.
- Outputs:
Tensor, which has the same type and shape as x with values in the range[0,1].
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> softmax = nn.Softmax() >>> output = softmax(input_x) >>> print(output) [0.03168 0.01166 0.0861 0.636 0.2341 ]
-
class
tinyms.layers.LogSoftmax(axis=-1)[source]¶ LogSoftmax activation function.
Applies the LogSoftmax function to n-dimensional input tensor.
The input is transformed by the Softmax function and then by the log function to lie in range[-inf,0).
Logsoftmax is defined as:
\[\text{logsoftmax}(x_i) = \log \left(\frac{\exp(x_i)}{\sum_{j=0}^{n-1} \exp(x_j)}\right),\]where \(x_{i}\) is the \(i\)-th slice in the given dimension of the input Tensor.
- Parameters
axis (int) – The axis to apply LogSoftmax operation, -1 means the last dimension. Default: -1.
- Inputs:
x (Tensor) - The input of LogSoftmax.
- Outputs:
Tensor, which has the same type and shape as the input as x with values in the range[-inf,0).
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> log_softmax = nn.LogSoftmax() >>> output = log_softmax(input_x) >>> print(output) [[-5.00672150e+00 -6.72150636e-03 -1.20067215e+01] [-7.00091219e+00 -1.40009127e+01 -9.12250078e-04]]
-
class
tinyms.layers.ReLU[source]¶ Rectified Linear Unit activation function.
Applies the rectified linear unit function element-wise.
\[\text{ReLU}(x) = (x)^+ = \max(0, x),\]It returns element-wise \(\max(0, x)\), specially, the neurons with the negative output will be suppressed and the active neurons will stay the same.
The picture about ReLU looks like this ReLU.
- Inputs:
input_data (Tensor) - The input of ReLU.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([-1, 2, -3, 2, -1]), mindspore.float16) >>> relu = nn.ReLU() >>> output = relu(input_x) >>> print(output) [0. 2. 0. 2. 0.]
-
class
tinyms.layers.ReLU6[source]¶ Compute ReLU6 activation function.
ReLU6 is similar to ReLU with a upper limit of 6, which if the inputs are greater than 6, the outputs will be suppressed to 6. It computes element-wise as
\[\min(\max(0, x), 6).\]The input is a Tensor of any valid shape.
- Inputs:
input_data (Tensor) - The input of ReLU6.
- Outputs:
Tensor, which has the same type as input_data.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> relu6 = nn.ReLU6() >>> output = relu6(input_x) >>> print(output) [0. 0. 0. 2. 1.]
-
class
tinyms.layers.Tanh[source]¶ Tanh activation function.
Applies the Tanh function element-wise, returns a new tensor with the hyperbolic tangent of the elements of input, The input is a Tensor with any valid shape.
Tanh function is defined as:
\[tanh(x_i) = \frac{\exp(x_i) - \exp(-x_i)}{\exp(x_i) + \exp(-x_i)} = \frac{\exp(2x_i) - 1}{\exp(2x_i) + 1},\]where \(x_i\) is an element of the input Tensor.
- Inputs:
input_data (Tensor) - The input of Tanh.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3, 2, 1]), mindspore.float16) >>> tanh = nn.Tanh() >>> output = tanh(input_x) >>> print(output) [0.7617 0.964 0.995 0.964 0.7617]
-
class
tinyms.layers.GELU[source]¶ Gaussian error linear unit activation function.
Applies GELU function to each element of the input. The input is a Tensor with any valid shape.
GELU is defined as:
\[GELU(x_i) = x_i*P(X < x_i),\]where \(P\) is the cumulative distribution function of standard Gaussian distribution and \(x_i\) is the element of the input.
The picture about GELU looks like this GELU.
- Inputs:
input_data (Tensor) - The input of GELU.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> gelu = nn.GELU() >>> output = gelu(input_x) >>> print(output) [[-1.5880802e-01 3.9999299e+00 -3.1077917e-21] [ 1.9545976e+00 -2.2918017e-07 9.0000000e+00]]
-
class
tinyms.layers.FastGelu[source]¶ Fast Gaussian error linear unit activation function.
Applies FastGelu function to each element of the input. The input is a Tensor with any valid shape.
FastGelu is defined as:
\[FastGelu(x_i) = \frac {x_i} {1 + \exp(-1.702 * \left| x_i \right|)} * \exp(0.851 * (x_i - \left| x_i \right|))\]where \(x_i\) is the element of the input.
- Inputs:
input_data (Tensor) - The input of FastGelu with data type of float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> fast_gelu = nn.FastGelu() >>> output = fast_gelu(input_x) >>> print(output) [[-1.5420423e-01 3.9955850e+00 -9.7664279e-06] [ 1.9356586e+00 -1.0070159e-03 8.9999981e+00]]
-
class
tinyms.layers.Sigmoid[source]¶ Sigmoid activation function.
Applies sigmoid-type activation element-wise.
Sigmoid function is defined as:
\[\text{sigmoid}(x_i) = \frac{1}{1 + \exp(-x_i)},\]where \(x_i\) is the element of the input.
The picture about Sigmoid looks like this Sigmoid.
- Inputs:
input_data (Tensor) - The input of Tanh.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> sigmoid = nn.Sigmoid() >>> output = sigmoid(input_x) >>> print(output) [0.2688 0.11914 0.5 0.881 0.7305 ]
-
class
tinyms.layers.PReLU(channel=1, w=0.25)[source]¶ PReLU activation function.
Applies the PReLU function element-wise.
PReLU is defined as:
\[prelu(x_i)= \max(0, x_i) + w * \min(0, x_i),\]where \(x_i\) is an element of an channel of the input.
Here \(w\) is a learnable parameter with a default initial value 0.25. Parameter \(w\) has dimensionality of the argument channel. If called without argument channel, a single parameter \(w\) will be shared across all channels.
The picture about PReLU looks like this PReLU.
- Parameters
- Inputs:
input_data (Tensor) - The input of PReLU.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([[[[0.1, 0.6], [0.9, 0.9]]]]), mindspore.float32) >>> prelu = nn.PReLU() >>> output = prelu(input_x) >>> print(output) [[[[0.1 0.6] [0.9 0.9]]]]
-
tinyms.layers.get_activation(name)[source]¶ Gets the activation function.
- Parameters
name (str) – The name of the activation function.
- Returns
Function, the activation function.
Examples
>>> sigmoid = nn.get_activation('sigmoid')
-
class
tinyms.layers.LeakyReLU(alpha=0.2)[source]¶ Leaky ReLU activation function.
LeakyReLU is similar to ReLU, but LeakyReLU has a slope that makes it not equal to 0 at x < 0. The activation function is defined as:
\[\text{leaky_relu}(x) = \begin{cases}x, &\text{if } x \geq 0; \cr \text{alpha} * x, &\text{otherwise.}\end{cases}\]See https://ai.stanford.edu/~amaas/papers/relu_hybrid_icml2013_final.pdf
- Inputs:
input_x (Tensor) - The input of LeakyReLU.
- Outputs:
Tensor, has the same type and shape as the input_x.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> leaky_relu = nn.LeakyReLU() >>> output = leaky_relu(input_x) >>> print(output) [[-0.2 4. -1.6] [ 2. -1. 9. ]]
-
class
tinyms.layers.HSigmoid[source]¶ Hard sigmoid activation function.
Applies hard sigmoid activation element-wise. The input is a Tensor with any valid shape.
Hard sigmoid is defined as:
\[\text{hsigmoid}(x_{i}) = max(0, min(1, \frac{x_{i} + 3}{6})),\]where \(x_{i}\) is the \(i\)-th slice in the given dimension of the input Tensor.
- Inputs:
input_data (Tensor) - The input of HSigmoid.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Supported Platforms:
GPU
Examples
>>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> hsigmoid = nn.HSigmoid() >>> result = hsigmoid(input_x) >>> print(result) [0.3333 0.1666 0.5 0.833 0.6665]
-
class
tinyms.layers.HSwish[source]¶ Hard swish activation function.
Applies hswish-type activation element-wise. The input is a Tensor with any valid shape.
Hard swish is defined as:
\[\text{hswish}(x_{i}) = x_{i} * \frac{ReLU6(x_{i} + 3)}{6},\]where \(x_{i}\) is the \(i\)-th slice in the given dimension of the input Tensor.
- Inputs:
input_data (Tensor) - The input of HSwish.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Supported Platforms:
GPU
Examples
>>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> hswish = nn.HSwish() >>> result = hswish(input_x) >>> print(result) [-0.3333 -0.3333 0 1.666 0.6665]
-
class
tinyms.layers.ELU(alpha=1.0)[source]¶ Exponential Linear Uint activation function.
Applies the exponential linear unit function element-wise. The activation function is defined as:
\[E_{i} = \begin{cases} x, &\text{if } x \geq 0; \cr \text{alpha} * (\exp(x_i) - 1), &\text{otherwise.} \end{cases}\]The picture about ELU looks like this ELU.
- Parameters
alpha (float) – The coefficient of negative factor whose type is float. Default: 1.0.
- Inputs:
input_data (Tensor) - The input of ELU.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float32) >>> elu = nn.ELU() >>> result = elu(input_x) >>> print(result) [-0.63212055 -0.86466473 0. 2. 1.]
-
class
tinyms.layers.LogSigmoid[source]¶ Logsigmoid activation function.
Applies logsigmoid activation element-wise. The input is a Tensor with any valid shape.
Logsigmoid is defined as:
\[\text{logsigmoid}(x_{i}) = log(\frac{1}{1 + \exp(-x_i)}),\]where \(x_{i}\) is the element of the input.
- Inputs:
input_data (Tensor) - The input of LogSigmoid.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.LogSigmoid() >>> input_x = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) >>> output = net(input_x) >>> print(output) [-0.31326166 -0.12692806 -0.04858734]
-
class
tinyms.layers.BatchNorm1d(num_features, eps=1e-05, momentum=0.9, affine=True, gamma_init='ones', beta_init='zeros', moving_mean_init='zeros', moving_var_init='ones', use_batch_statistics=None)[source]¶ Batch normalization layer over a 2D input.
Batch Normalization is widely used in convolutional networks. This layer applies Batch Normalization over a 2D input (a mini-batch of 1D inputs) to reduce internal covariate shift as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. It rescales and recenters the feature using a mini-batch of data and the learned parameters which can be described in the following formula.
\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]Note
The implementation of BatchNorm is different in graph mode and pynative mode, therefore the mode is not recommended to be changed after net was initialized.
- Parameters
num_features (int) – C from an expected input of size (N, C).
eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.
momentum (float) – A floating hyperparameter of the momentum for the running_mean and running_var computation. Default: 0.9.
affine (bool) – A bool value. When set to True, gamma and beta can be learned. Default: True.
gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’.
beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’.
moving_mean_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving mean. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’.
moving_var_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving variance. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’.
use_batch_statistics (bool) – If true, use the mean value and variance value of current batch data. If false, use the mean value and variance value of specified value. If None, the training process will use the mean and variance of current batch data and track the running mean and variance, the evaluation process will use the running mean and variance. Default: None.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in})\).
- Outputs:
Tensor, the normalized, scaled, offset tensor, of shape \((N, C_{out})\).
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.BatchNorm1d(num_features=4) >>> np.random.seed(0) >>> input = Tensor(np.random.randint(0, 255, [2, 4]), mindspore.float32) >>> output = net(input) >>> print(output) [[171.99915 46.999763 116.99941 191.99904 ] [ 66.999664 250.99875 194.99902 102.99948 ]]
-
class
tinyms.layers.BatchNorm2d(num_features, eps=1e-05, momentum=0.9, affine=True, gamma_init='ones', beta_init='zeros', moving_mean_init='zeros', moving_var_init='ones', use_batch_statistics=None, data_format='NCHW')[source]¶ Batch normalization layer over a 4D input.
Batch Normalization is widely used in convolutional networks. This layer applies Batch Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension) to avoid internal covariate shift as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. It rescales and recenters the feature using a mini-batch of data and the learned parameters which can be described in the following formula.
\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]Note
The implementation of BatchNorm is different in graph mode and pynative mode, therefore that mode can not be changed after net was initialized. Note that the formula for updating the running_mean and running_var is \(\hat{x}_\text{new} = (1 - \text{momentum}) \times x_t + \text{momentum} \times \hat{x}\), where \(\hat{x}\) is the estimated statistic and \(x_t\) is the new observed value.
- Parameters
num_features (int) – C from an expected input of size (N, C, H, W).
eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.
momentum (float) – A floating hyperparameter of the momentum for the running_mean and running_var computation. Default: 0.9.
affine (bool) – A bool value. When set to True, gamma and beta can be learned. Default: True.
gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’.
beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’.
moving_mean_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving mean. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’.
moving_var_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving variance. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’.
use_batch_statistics (bool) – If true, use the mean value and variance value of current batch data. If false, use the mean value and variance value of specified value. If None, the training process will use the mean and variance of current batch data and track the running mean and variance, the evaluation process will use the running mean and variance. Default: None.
data_format (str) – The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: ‘NCHW’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor, the normalized, scaled, offset tensor, of shape \((N, C_{out}, H_{out}, W_{out})\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.BatchNorm2d(num_features=3) >>> np.random.seed(0) >>> input = Tensor(np.random.randint(0, 255, [1, 3, 2, 2]), mindspore.float32) >>> output = net(input) >>> print(output) [[[[171.99915 46.999763 ] [116.99941 191.99904 ]] [[ 66.999664 250.99875 ] [194.99902 102.99948 ]] [[ 8.999955 210.99895 ] [ 20.999895 241.9988 ]]]]
-
class
tinyms.layers.LayerNorm(normalized_shape, begin_norm_axis=-1, begin_params_axis=-1, gamma_init='ones', beta_init='zeros', epsilon=1e-07)[source]¶ Applies Layer Normalization over a mini-batch of inputs.
Layer normalization is widely used in recurrent neural networks. It applies normalization on a mini-batch of inputs for each single training case as described in the paper Layer Normalization. Unlike batch normalization, layer normalization performs exactly the same computation at training and testing time. It can be described using the following formula. It is applied across all channels and pixel but only one batch size.
\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]- Parameters
normalized_shape (Union(tuple[int], list[int]) – The normalization is performed over axis begin_norm_axis … R - 1.
begin_norm_axis (int) – The first normalization dimension: normalization will be performed along dimensions begin_norm_axis: rank(inputs), the value should be in [-1, rank(input)). Default: -1.
begin_params_axis (int) – The first parameter(beta, gamma)dimension: scale and centering parameters will have dimensions begin_params_axis: rank(inputs) and will be broadcast with the normalized inputs accordingly, the value should be in [-1, rank(input)). Default: -1.
gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’.
beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’.
epsilon (float) – A value added to the denominator for numerical stability. Default: 1e-7.
- Inputs:
input_x (Tensor) - The shape of ‘input_x’ is \((x_1, x_2, ..., x_R)\), and input_shape[begin_norm_axis:] is equal to normalized_shape.
- Outputs:
Tensor, the normalized and scaled offset tensor, has the same shape and data type as the input_x.
- Supported Platforms:
AscendGPU
Examples
>>> x = Tensor(np.ones([20, 5, 10, 10]), mindspore.float32) >>> shape1 = x.shape[1:] >>> m = nn.LayerNorm(shape1, begin_norm_axis=1, begin_params_axis=1) >>> output = m(x).shape >>> print(output) (20, 5, 10, 10)
-
class
tinyms.layers.GroupNorm(num_groups, num_channels, eps=1e-05, affine=True, gamma_init='ones', beta_init='zeros')[source]¶ Group Normalization over a mini-batch of inputs.
Group normalization is widely used in recurrent neural networks. It applies normalization on a mini-batch of inputs for each single training case as described in the paper Group Normalization. Group normalization divides the channels into groups and computes within each group the mean and variance for normalization, and it performs very stable over a wide range of batch size. It can be described using the following formula.
\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]- Parameters
num_groups (int) – The number of groups to be divided along the channel dimension.
num_channels (int) – The number of channels per group.
eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.
affine (bool) – A bool value, this layer will have learnable affine parameters when set to true. Default: True.
gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’. If gamma_init is a Tensor, the shape must be [num_channels].
beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’. If beta_init is a Tensor, the shape must be [num_channels].
- Inputs:
input_x (Tensor) - The input feature with shape [N, C, H, W].
- Outputs:
Tensor, the normalized and scaled offset tensor, has the same shape and data type as the input_x.
- Supported Platforms:
AscendGPU
Examples
>>> goup_norm_op = nn.GroupNorm(2, 2) >>> x = Tensor(np.ones([1, 2, 4, 4], np.float32)) >>> output = goup_norm_op(x) >>> print(output) [[[[0. 0. 0. 0.] [0. 0. 0. 0.] [0. 0. 0. 0.] [0. 0. 0. 0.]] [[0. 0. 0. 0.] [0. 0. 0. 0.] [0. 0. 0. 0.] [0. 0. 0. 0.]]]]
-
class
tinyms.layers.GlobalBatchNorm(num_features, eps=1e-05, momentum=0.9, affine=True, gamma_init='ones', beta_init='zeros', moving_mean_init='zeros', moving_var_init='ones', use_batch_statistics=None, device_num_each_group=2)[source]¶ Global normalization layer over a N-dimension input.
Global Normalization is cross device synchronized batch normalization. The implementation of Batch Normalization only normalizes the data within each device. Global normalization will normalize the input within the group. It has been described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. It rescales and recenters the feature using a mini-batch of data and the learned parameters which can be described in the following formula.
\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]Note
Currently, GlobalBatchNorm only supports 2D and 4D inputs.
- Parameters
num_features (int) – C from an expected input of size (N, C, H, W).
device_num_each_group (int) – The number of devices in each group. Default: 2.
eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.
momentum (float) – A floating hyperparameter of the momentum for the running_mean and running_var computation. Default: 0.9.
affine (bool) – A bool value. When set to True, gamma and beta can be learned. Default: True.
gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’.
beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’.
moving_mean_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving mean. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’.
moving_var_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving variance. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’.
use_batch_statistics (bool) – If true, use the mean value and variance value of current batch data. If false, use the mean value and variance value of specified value. If None, training process will use the mean and variance of current batch data and track the running mean and variance, eval process will use the running mean and variance. Default: None.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor, the normalized, scaled, offset tensor, of shape \((N, C_{out}, H_{out}, W_{out})\).
- Supported Platforms:
Ascend
Examples
>>> # This example should be run with multiple processes. >>> # Please refer to the tutorial > Distributed Training on mindspore.cn. >>> import numpy as np >>> from mindspore.communication import init >>> from mindspore import context >>> from mindspore.context import ParallelMode >>> from mindspore import nn, Tensor >>> from mindspore.common import dtype as mstype >>> >>> context.set_context(mode=context.GRAPH_MODE) >>> init() >>> context.reset_auto_parallel_context() >>> context.set_auto_parallel_context(parallel_mode=ParallelMode.DATA_PARALLEL) >>> np.random.seed(0) >>> global_bn_op = nn.GlobalBatchNorm(num_features=3, device_num_each_group=2) >>> input = Tensor(np.random.randint(0, 255, [1, 3, 2, 2]), mstype.float32) >>> output = global_bn_op(input) >>> print(output) [[[[171.99915 46.999763] [116.99941 191.99904 ]] [[ 66.999664 250.99875 ] [194.99902 102.99948 ]] [[ 8.999955 210.99895 ] [ 20.9999895 241.9988 ]]]]
-
class
tinyms.layers.Conv2d(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros', data_format='NCHW')[source]¶ 2D convolution layer.
Applies a 2D convolution over an input tensor which is typically of shape \((N, C_{in}, H_{in}, W_{in})\), where \(N\) is batch size, \(C_{in}\) is channel number, and \(H_{in}, W_{in})\) are height and width. For each batch of shape \((C_{in}, H_{in}, W_{in})\), the formula is defined as:
\[out_j = \sum_{i=0}^{C_{in} - 1} ccor(W_{ij}, X_i) + b_j,\]where \(ccor\) is the cross-correlation operator, \(C_{in}\) is the input channel number, \(j\) ranges from \(0\) to \(C_{out} - 1\), \(W_{ij}\) corresponds to the \(i\)-th channel of the \(j\)-th filter and \(out_{j}\) corresponds to the \(j\)-th channel of the output. \(W_{ij}\) is a slice of kernel and it has shape \((\text{ks_h}, \text{ks_w})\), where \(\text{ks_h}\) and \(\text{ks_w}\) are the height and width of the convolution kernel. The full kernel has shape \((C_{out}, C_{in} // \text{group}, \text{ks_h}, \text{ks_w})\), where group is the group number to split the input in the channel dimension.
If the ‘pad_mode’ is set to be “valid”, the output height and width will be \(\left \lfloor{1 + \frac{H_{in} + 2 \times \text{padding} - \text{ks_h} - (\text{ks_h} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) and \(\left \lfloor{1 + \frac{W_{in} + 2 \times \text{padding} - \text{ks_w} - (\text{ks_w} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) respectively.
The first introduction can be found in paper Gradient Based Learning Applied to Document Recognition.
- Parameters
in_channels (int) – The number of input channel \(C_{in}\).
out_channels (int) – The number of output channel \(C_{out}\).
kernel_size (Union[int, tuple[int]]) – The data type is int or a tuple of 2 integers. Specifies the height and width of the 2D convolution window. Single int means the value is for both the height and the width of the kernel. A tuple of 2 ints means the first value is for the height and the other is for the width of the kernel.
stride (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Default: 1.
pad_mode (str) –
Specifies padding mode. The optional values are “same”, “valid”, “pad”. Default: “same”.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side. If this mode is set, padding must be 0.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded. If this mode is set, padding must be 0.
pad: Implicit paddings on both sides of the input. The number of padding will be padded to the input Tensor borders. padding must be greater than or equal to 0.
padding (Union[int, tuple[int]]) – Implicit paddings on both sides of the input. If padding is one integer, the paddings of top, bottom, left and right are the same, equal to padding. If padding is a tuple with four integers, the paddings of top, bottom, left and right will be equal to padding[0], padding[1], padding[2], and padding[3] accordingly. Default: 0.
dilation (Union[int, tuple[int]]) – The data type is int or a tuple of 2 integers. Specifies the dilation rate to use for dilated convolution. If set to be \(k > 1\), there will be \(k - 1\) pixels skipped for each sampling location. Its value must be greater or equal to 1 and bounded by the height and width of the input. Default: 1.
group (int) – Splits filter into groups, in_ channels and out_channels must be divisible by the number of groups. If the group is equal to in_channels and out_channels, this 2D convolution layer also can be called 2D depthwise convolution layer. Default: 1.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the convolution kernel. It can be a Tensor, a string, an Initializer or a number. When a string is specified, values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as well as constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’ and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values of Initializer for more details. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. Possible Initializer and string are the same as ‘weight_init’. Refer to the values of Initializer for more details. Default: ‘zeros’.
data_format (str) – The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: ‘NCHW’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\) or \((N, H_{in}, W_{in}, C_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, H_{out}, W_{out})\) or \((N, H_{out}, W_{out}, C_{out})\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.Conv2d(120, 240, 4, has_bias=False, weight_init='normal') >>> input = Tensor(np.ones([1, 120, 1024, 640]), mindspore.float32) >>> output = net(input).shape >>> print(output) (1, 240, 1024, 640)
-
class
tinyms.layers.Conv2dTranspose(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros')[source]¶ 2D transposed convolution layer.
Compute a 2D transposed convolution, which is also known as a deconvolution (although it is not an actual deconvolution).
Input is typically of shape \((N, C, H, W)\), where \(N\) is batch size and \(C\) is channel number.
If the ‘pad_mode’ is set to be “pad”, the height and width of output are defined as:
\[ \begin{align}\begin{aligned}H_{out} = (H_{in} - 1) \times \text{stride} - 2 \times \text{padding} + \text{dilation} \times (\text{ks_h} - 1) + 1\\W_{out} = (W_{in} - 1) \times \text{stride} - 2 \times \text{padding} + \text{dilation} \times (\text{ks_w} - 1) + 1\end{aligned}\end{align} \]where \(\text{ks_h}\) is the height of the convolution kernel and \(\text{ks_w}\) is the width of the convolution kernel.
- Parameters
in_channels (int) – The number of channels in the input space.
out_channels (int) – The number of channels in the output space.
kernel_size (Union[int, tuple]) – int or a tuple of 2 integers, which specifies the height and width of the 2D convolution window. Single int means the value is for both the height and the width of the kernel. A tuple of 2 ints means the first value is for the height and the other is for the width of the kernel.
stride (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Its value must be equal to or greater than 1. Default: 1.
pad_mode (str) –
Select the mode of the pad. The optional values are “pad”, “same”, “valid”. Default: “same”.
pad: Implicit paddings on both sides of the input.
same: Adopted the way of completion.
valid: Adopted the way of discarding.
padding (Union[int, tuple[int]]) – Implicit paddings on both sides of the input. If padding is one integer, the paddings of top, bottom, left and right are the same, equal to padding. If padding is a tuple with four integers, the paddings of top, bottom, left and right will be equal to padding[0], padding[1], padding[2], and padding[3] accordingly. Default: 0.
dilation (Union[int, tuple[int]]) – The data type is int or a tuple of 2 integers. Specifies the dilation rate to use for dilated convolution. If set to be \(k > 1\), there will be \(k - 1\) pixels skipped for each sampling location. Its value must be greater than or equal to 1 and bounded by the height and width of the input. Default: 1.
group (int) – Splits filter into groups, in_channels and out_channels must be divisible by the number of groups. This does not support for Davinci devices when group > 1. Default: 1.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the convolution kernel. It can be a Tensor, a string, an Initializer or a number. When a string is specified, values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as well as constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’ and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values of Initializer for more details. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. Possible Initializer and string are the same as ‘weight_init’. Refer to the values of Initializer for more details. Default: ‘zeros’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, H_{out}, W_{out})\).
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.Conv2dTranspose(3, 64, 4, has_bias=False, weight_init='normal', pad_mode='pad') >>> input = Tensor(np.ones([1, 3, 16, 50]), mindspore.float32) >>> output = net(input).shape >>> print(output) (1, 64, 19, 53)
-
class
tinyms.layers.Conv1d(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros')[source]¶ 1D convolution layer.
Applies a 1D convolution over an input tensor which is typically of shape \((N, C_{in}, W_{in})\), where \(N\) is batch size and \(C_{in}\) is channel number. For each batch of shape \((C_{in}, W_{in})\), the formula is defined as:
\[out_j = \sum_{i=0}^{C_{in} - 1} ccor(W_{ij}, X_i) + b_j,\]where \(ccor\) is the cross correlation operator, \(C_{in}\) is the input channel number, \(j\) ranges from \(0\) to \(C_{out} - 1\), \(W_{ij}\) corresponds to the \(i\)-th channel of the \(j\)-th filter and \(out_{j}\) corresponds to the \(j\)-th channel of the output. \(W_{ij}\) is a slice of kernel and it has shape \((\text{ks_w})\), where \(\text{ks_w}\) is the width of the convolution kernel. The full kernel has shape \((C_{out}, C_{in} // \text{group}, \text{ks_w})\), where group is the group number to split the input in the channel dimension.
If the ‘pad_mode’ is set to be “valid”, the output width will be \(\left \lfloor{1 + \frac{W_{in} + 2 \times \text{padding} - \text{ks_w} - (\text{ks_w} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) respectively.
The first introduction of convolution layer can be found in paper Gradient Based Learning Applied to Document Recognition.
- Parameters
in_channels (int) – The number of input channel \(C_{in}\).
out_channels (int) – The number of output channel \(C_{out}\).
kernel_size (int) – The data type is int. Specifies the width of the 1D convolution window.
stride (int) – The distance of kernel moving, an int number that represents the width of movement. Default: 1.
pad_mode (str) –
Specifies padding mode. The optional values are “same”, “valid”, “pad”. Default: “same”.
same: Adopts the way of completion. The output width will be the same as the input. The total number of padding will be calculated in the horizontal direction and evenly distributed to left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side. If this mode is set, padding must be 0.
valid: Adopts the way of discarding. The possible largest width of the output will be returned without padding. Extra pixels will be discarded. If this mode is set, padding must be 0.
pad: Implicit paddings on both sides of the input. The number of padding will be padded to the input Tensor borders. padding must be greater than or equal to 0.
padding (int) – Implicit paddings on both sides of the input. Default: 0.
dilation (int) – The data type is int. Specifies the dilation rate to use for dilated convolution. If set to be \(k > 1\), there will be \(k - 1\) pixels skipped for each sampling location. Its value must be greater or equal to 1 and bounded by the height and width of the input. Default: 1.
group (int) – Splits filter into groups, in_ channels and out_channels must be divisible by the number of groups. Default: 1.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – An initializer for the convolution kernel. It can be a Tensor, a string, an Initializer or a number. When a string is specified, values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as well as constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’ and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values of Initializer for more details. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. Possible Initializer and string are the same as ‘weight_init’. Refer to the values of Initializer for more details. Default: ‘zeros’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, W_{out})\).
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.Conv1d(120, 240, 4, has_bias=False, weight_init='normal') >>> input = Tensor(np.ones([1, 120, 640]), mindspore.float32) >>> output = net(input).shape >>> print(output) (1, 240, 640)
-
class
tinyms.layers.Conv1dTranspose(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros')[source]¶ 1D transposed convolution layer.
Compute a 1D transposed convolution, which is also known as a deconvolution (although it is not an actual deconvolution).
Input is typically of shape \((N, C, W)\), where \(N\) is batch size and \(C\) is channel number.
If the ‘pad_mode’ is set to be “pad”, the width of output is defined as:
\[W_{out} = (W_{in} - 1) \times \text{stride} - 2 \times \text{padding} + \text{dilation} \times (\text{ks_w} - 1) + 1\]where \(\text{ks_w}\) is the width of the convolution kernel.
- Parameters
in_channels (int) – The number of channels in the input space.
out_channels (int) – The number of channels in the output space.
kernel_size (int) – int, which specifies the width of the 1D convolution window.
stride (int) – The distance of kernel moving, an int number that represents the width of movement. Default: 1.
pad_mode (str) –
Select the mode of the pad. The optional values are “pad”, “same”, “valid”. Default: “same”.
pad: Implicit paddings on both sides of the input.
same: Adopted the way of completion.
valid: Adopted the way of discarding.
padding (int) – Implicit paddings on both sides of the input. Default: 0.
dilation (int) – The data type is int. Specifies the dilation rate to use for dilated convolution. If set to be \(k > 1\), there will be \(k - 1\) pixels skipped for each sampling location. Its value must be greater or equal to 1 and bounded by the width of the input. Default: 1.
group (int) – Splits filter into groups, in_channels and out_channels must be divisible by the number of groups. This is not support for Davinci devices when group > 1. Default: 1.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the convolution kernel. It can be a Tensor, a string, an Initializer or a numbers.Number. When a string is specified, values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as well as constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’ and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values of Initializer for more details. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. Possible Initializer and string are the same as ‘weight_init’. Refer to the values of Initializer for more details. Default: ‘zeros’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, W_{out})\).
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.Conv1dTranspose(3, 64, 4, has_bias=False, weight_init='normal', pad_mode='pad') >>> input = Tensor(np.ones([1, 3, 50]), mindspore.float32) >>> output = net(input).shape >>> print(output) (1, 64, 53)
-
class
tinyms.layers.LSTM(input_size, hidden_size, num_layers=1, has_bias=True, batch_first=False, dropout=0, bidirectional=False)[source]¶ Stacked LSTM (Long Short-Term Memory) layers.
Apply LSTM layer to the input.
There are two pipelines connecting two consecutive cells in a LSTM model; one is cell state pipeline and the other is hidden state pipeline. Denote two consecutive time nodes as \(t-1\) and \(t\). Given an input \(x_t\) at time \(t\), an hidden state \(h_{t-1}\) and an cell state \(c_{t-1}\) of the layer at time \({t-1}\), the cell state and hidden state at time \(t\) is computed using an gating mechanism. Input gate \(i_t\) is designed to protect the cell from perturbation by irrelevant inputs. Forget gate \(f_t\) affords protection of the cell by forgetting some information in the past, which is stored in \(h_{t-1}\). Output gate \(o_t\) protects other units from perturbation by currently irrelevant memory contents. Candidate cell state \(\tilde{c}_t\) is calculated with the current input, on which the input gate will be applied. Finally, current cell state \(c_{t}\) and hidden state \(h_{t}\) are computed with the calculated gates and cell states. The complete formulation is as follows.
\[\begin{split}\begin{array}{ll} \\ i_t = \sigma(W_{ix} x_t + b_{ix} + W_{ih} h_{(t-1)} + b_{ih}) \\ f_t = \sigma(W_{fx} x_t + b_{fx} + W_{fh} h_{(t-1)} + b_{fh}) \\ \tilde{c}_t = \tanh(W_{cx} x_t + b_{cx} + W_{ch} h_{(t-1)} + b_{ch}) \\ o_t = \sigma(W_{ox} x_t + b_{ox} + W_{oh} h_{(t-1)} + b_{oh}) \\ c_t = f_t * c_{(t-1)} + i_t * \tilde{c}_t \\ h_t = o_t * \tanh(c_t) \\ \end{array}\end{split}\]Here \(\sigma\) is the sigmoid function, and \(*\) is the Hadamard product. \(W, b\) are learnable weights between the output and the input in the formula. For instance, \(W_{ix}, b_{ix}\) are the weight and bias used to transform from input \(x\) to \(i\). Details can be found in paper LONG SHORT-TERM MEMORY and Long Short-Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling.
- Parameters
input_size (int) – Number of features of input.
hidden_size (int) – Number of features of hidden layer.
num_layers (int) – Number of layers of stacked LSTM . Default: 1.
has_bias (bool) – Whether the cell has bias b_ih and b_hh. Default: True.
batch_first (bool) – Specifies whether the first dimension of input is batch_size. Default: False.
dropout (float, int) – If not 0, append Dropout layer on the outputs of each LSTM layer except the last layer. Default 0. The range of dropout is [0.0, 1.0].
bidirectional (bool) – Specifies whether it is a bidirectional LSTM. Default: False.
- Inputs:
input (Tensor) - Tensor of shape (seq_len, batch_size, input_size) or (batch_size, seq_len, input_size).
hx (tuple) - A tuple of two Tensors (h_0, c_0) both of data type mindspore.float32 or mindspore.float16 and shape (num_directions * num_layers, batch_size, hidden_size). Data type of hx must be the same as input.
- Outputs:
Tuple, a tuple contains (output, (h_n, c_n)).
output (Tensor) - Tensor of shape (seq_len, batch_size, num_directions * hidden_size).
hx_n (tuple) - A tuple of two Tensor (h_n, c_n) both of shape (num_directions * num_layers, batch_size, hidden_size).
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.LSTM(10, 16, 2, has_bias=True, batch_first=True, bidirectional=False) >>> input = Tensor(np.ones([3, 5, 10]).astype(np.float32)) >>> h0 = Tensor(np.ones([1 * 2, 3, 16]).astype(np.float32)) >>> c0 = Tensor(np.ones([1 * 2, 3, 16]).astype(np.float32)) >>> output, (hn, cn) = net(input, (h0, c0)) >>> print(output.shape) (3, 5, 16)
-
class
tinyms.layers.LSTMCell(input_size, hidden_size, has_bias=True, batch_first=False, dropout=0, bidirectional=False)[source]¶ LSTM (Long Short-Term Memory) layer.
Apply LSTM layer to the input.
There are two pipelines connecting two consecutive cells in a LSTM model; one is cell state pipeline and the other is hidden state pipeline. Denote two consecutive time nodes as \(t-1\) and \(t\). Given an input \(x_t\) at time \(t\), an hidden state \(h_{t-1}\) and an cell state \(c_{t-1}\) of the layer at time \({t-1}\), the cell state and hidden state at time \(t\) is computed using an gating mechanism. Input gate \(i_t\) is designed to protect the cell from perturbation by irrelevant inputs. Forget gate \(f_t\) affords protection of the cell by forgetting some information in the past, which is stored in \(h_{t-1}\). Output gate \(o_t\) protects other units from perturbation by currently irrelevant memory contents. Candidate cell state \(\tilde{c}_t\) is calculated with the current input, on which the input gate will be applied. Finally, current cell state \(c_{t}\) and hidden state \(h_{t}\) are computed with the calculated gates and cell states. The complete formulation is as follows.
\[\begin{split}\begin{array}{ll} \\ i_t = \sigma(W_{ix} x_t + b_{ix} + W_{ih} h_{(t-1)} + b_{ih}) \\ f_t = \sigma(W_{fx} x_t + b_{fx} + W_{fh} h_{(t-1)} + b_{fh}) \\ \tilde{c}_t = \tanh(W_{cx} x_t + b_{cx} + W_{ch} h_{(t-1)} + b_{ch}) \\ o_t = \sigma(W_{ox} x_t + b_{ox} + W_{oh} h_{(t-1)} + b_{oh}) \\ c_t = f_t * c_{(t-1)} + i_t * \tilde{c}_t \\ h_t = o_t * \tanh(c_t) \\ \end{array}\end{split}\]Here \(\sigma\) is the sigmoid function, and \(*\) is the Hadamard product. \(W, b\) are learnable weights between the output and the input in the formula. For instance, \(W_{ix}, b_{ix}\) are the weight and bias used to transform from input \(x\) to \(i\). Details can be found in paper LONG SHORT-TERM MEMORY and Long Short-Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling.
LSTMCell is a single-layer RNN, you can achieve multi-layer RNN by stacking LSTMCell.
- Parameters
input_size (int) – Number of features of input.
hidden_size (int) – Number of features of hidden layer.
has_bias (bool) – Whether the cell has bias b_ih and b_hh. Default: True.
batch_first (bool) – Specifies whether the first dimension of input is batch_size. Default: False.
dropout (float, int) – If not 0, append Dropout layer on the outputs of each LSTM layer except the last layer. Default 0. The range of dropout is [0.0, 1.0].
bidirectional (bool) – Specifies whether this is a bidirectional LSTM. If set True, number of directions will be 2 otherwise number of directions is 1. Default: False.
- Inputs:
input (Tensor) - Tensor of shape (seq_len, batch_size, input_size).
h - data type mindspore.float32 or mindspore.float16 and shape (num_directions, batch_size, hidden_size).
c - data type mindspore.float32 or mindspore.float16 and shape (num_directions, batch_size, hidden_size). Data type of h’ and ‘c’ must be the same of `input.
w - data type mindspore.float32 or mindspore.float16 and shape (weight_size, 1, 1). The value of weight_size depends on input_size, hidden_size and bidirectional
- Outputs:
output, h_n, c_n, ‘reserve’, ‘state’.
output (Tensor) - Tensor of shape (seq_len, batch_size, num_directions * hidden_size).
h - A Tensor with shape (num_directions, batch_size, hidden_size).
c - A Tensor with shape (num_directions, batch_size, hidden_size).
reserve - reserved
state - reserved
- Supported Platforms:
GPUCPU
Examples
>>> net = nn.LSTMCell(10, 12, has_bias=True, batch_first=True, bidirectional=False) >>> input = Tensor(np.ones([3, 5, 10]).astype(np.float32)) >>> h = Tensor(np.ones([1, 3, 12]).astype(np.float32)) >>> c = Tensor(np.ones([1, 3, 12]).astype(np.float32)) >>> w = Tensor(np.ones([1152, 1, 1]).astype(np.float32)) >>> output, h, c, _, _ = net(input, h, c, w) >>> print(output.shape) (3, 5, 12)
-
class
tinyms.layers.Dropout(keep_prob=0.5, dtype=mindspore.float32)[source]¶ Dropout layer for the input.
Randomly set some elements of the input tensor to zero with probability \(1 - keep\_prob\) during training using samples from a Bernoulli distribution.
The outputs are scaled by a factor of \(\frac{1}{keep\_prob}\) during training so that the output layer remains at a similar scale. During inference, this layer returns the same tensor as the input.
This technique is proposed in paper Dropout: A Simple Way to Prevent Neural Networks from Overfitting and proved to be effective to reduce over-fitting and prevents neurons from co-adaptation. See more details in Improving neural networks by preventing co-adaptation of feature detectors.
Note
Each channel will be zeroed out independently on every construct call.
- Parameters
keep_prob (float) – The keep rate, greater than 0 and less equal than 1. E.g. rate=0.9, dropping out 10% of input units. Default: 0.5.
dtype (
mindspore.dtype) – Data type of input. Default: mindspore.float32.
- Raises
ValueError – If keep_prob is not in range (0, 1].
- Inputs:
input (Tensor) - The input tensor.
- Outputs:
Tensor, output tensor with the same shape as the input.
- Supported Platforms:
AscendGPUCPU
Examples
>>> x = Tensor(np.ones([2, 2, 3]), mindspore.float32) >>> net = nn.Dropout(keep_prob=0.8) >>> net.set_train() Dropout<keep_prob=0.8> >>> output = net(x) >>> print(output.shape) (2, 2, 3)
-
class
tinyms.layers.Flatten[source]¶ Flatten layer for the input.
Flattens a tensor without changing dimension of batch size on the 0-th axis.
- Inputs:
input (Tensor) - Tensor of shape \((N, \ldots)\) to be flattened.
- Outputs:
Tensor, the shape of the output tensor is \((N, X)\), where \(X\) is the product of the remaining dimensions.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input = Tensor(np.array([[[1.2, 1.2], [2.1, 2.1]], [[2.2, 2.2], [3.2, 3.2]]]), mindspore.float32) >>> net = nn.Flatten() >>> output = net(input) >>> print(output) [[1.2 1.2 2.1 2.1] [2.2 2.2 3.2 3.2]]
-
class
tinyms.layers.Dense(in_channels, out_channels, weight_init='normal', bias_init='zeros', has_bias=True, activation=None)[source]¶ The dense connected layer.
Applies dense connected layer for the input. This layer implements the operation as:
\[\text{outputs} = \text{activation}(\text{inputs} * \text{kernel} + \text{bias}),\]where \(\text{activation}\) is the activation function passed as the activation argument (if passed in), \(\text{kernel}\) is a weight matrix with the same data type as the inputs created by the layer, and \(\text{bias}\) is a bias vector with the same data type as the inputs created by the layer (only if has_bias is True).
- Parameters
in_channels (int) – The number of channels in the input space.
out_channels (int) – The number of channels in the output space.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – The trainable weight_init parameter. The dtype is same as input x. The values of str refer to the function initializer. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – The trainable bias_init parameter. The dtype is same as input x. The values of str refer to the function initializer. Default: ‘zeros’.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: True.
activation (Union[str, Cell, Primitive]) – activate function applied to the output of the fully connected layer, eg. ‘ReLU’.Default: None.
- Raises
ValueError – If weight_init or bias_init shape is incorrect.
- Inputs:
input (Tensor) - Tensor of shape \((*, in\_channels)\).
- Outputs:
Tensor of shape \((*, out\_channels)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input = Tensor(np.array([[180, 234, 154], [244, 48, 247]]), mindspore.float32) >>> net = nn.Dense(3, 4) >>> output = net(input) >>> print(output.shape) (2, 4)
-
class
tinyms.layers.ClipByNorm(axis=None)[source]¶ Clips tensor values to a maximum \(L_2\)-norm.
The output of this layer remains the same if the \(L_2\)-norm of the input tensor is not greater than the argument clip_norm. Otherwise the tensor will be normalized as:
\[\text{output}(X) = \frac{\text{clip_norm} * X}{L_2(X)},\]where \(L_2(X)\) is the \(L_2\)-norm of \(X\).
- Parameters
axis (Union[None, int, tuple(int)]) – Compute the L2-norm along the Specific dimension. Default: None, all dimensions to calculate.
- Inputs:
input (Tensor) - Tensor of shape N-D. The type must be float32 or float16.
clip_norm (Tensor) - A scalar Tensor of shape \(()\) or \((1)\). Or a tensor shape can be broadcast to input shape.
- Outputs:
Tensor, clipped tensor with the same shape as the input, whose type is float32.
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.ClipByNorm() >>> input = Tensor(np.random.randint(0, 10, [4, 16]), mindspore.float32) >>> clip_norm = Tensor(np.array([100]).astype(np.float32)) >>> output = net(input, clip_norm) >>> print(output.shape) (4, 16)
-
class
tinyms.layers.Norm(axis=(), keep_dims=False)[source]¶ Computes the norm of vectors, currently including Euclidean norm, i.e., \(L_2\)-norm.
\[norm(x) = \sqrt{\sum_{i=1}^{n} (x_i^2)}\]- Parameters
- Inputs:
input (Tensor) - Tensor which is not empty.
- Outputs:
Tensor, output tensor with dimensions in ‘axis’ reduced to 1 will be returned if ‘keep_dims’ is True; otherwise a Tensor with dimensions in ‘axis’ removed is returned.
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.Norm(axis=0) >>> input = Tensor(np.array([[4, 4, 9, 1], [2, 1, 3, 6]]), mindspore.float32) >>> output = net(input) >>> print(output) [4.472136 4.1231055 9.486833 6.0827627]
-
class
tinyms.layers.OneHot(axis=-1, depth=1, on_value=1.0, off_value=0.0, dtype=mindspore.float32)[source]¶ Returns a one-hot tensor.
The locations represented by indices in argument indices take value on_value, while all other locations take value off_value.
Note
If the input indices is rank \(N\), the output will have rank \(N+1\). The new axis is created at dimension axis.
If indices is a scalar, the output shape will be a vector of length depth.
If indices is a vector of length features, the output shape will be:
features * depth if axis == -1 depth * features if axis == 0
If indices is a matrix with shape [batch, features], the output shape will be:
batch * features * depth if axis == -1 batch * depth * features if axis == 1 depth * batch * features if axis == 0
- Parameters
axis (int) – Features x depth if axis is -1, depth x features if axis is 0. Default: -1.
depth (int) – A scalar defining the depth of the one hot dimension. Default: 1.
on_value (float) – A scalar defining the value to fill in output[i][j] when indices[j] = i. Default: 1.0.
off_value (float) – A scalar defining the value to fill in output[i][j] when indices[j] != i. Default: 0.0.
dtype (
mindspore.dtype) – Data type of ‘on_value’ and ‘off_value’, not the data type of indices. Default: mindspore.float32.
- Inputs:
indices (Tensor) - A tensor of indices of data type mindspore.int32 and arbitrary shape.
- Outputs:
Tensor, the one-hot tensor of data type dtype with dimension at axis expanded to depth and filled with on_value and off_value.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.OneHot(depth=4, axis=1) >>> indices = Tensor([[1, 3], [0, 2]], dtype=mindspore.int32) >>> output = net(indices) >>> print(output) [[[0. 0.] [1. 0.] [0. 0.] [0. 1.]] [[1. 0.] [0. 0.] [0. 1.] [0. 0.]]]
-
class
tinyms.layers.Pad(paddings, mode='CONSTANT')[source]¶ Pads the input tensor according to the paddings and mode.
- Parameters
paddings (tuple) –
The shape of parameter paddings is (N, 2). N is the rank of input data. All elements of paddings are int type. For D th dimension of input, paddings[D, 0] indicates how many sizes to be extended ahead of the D th dimension of the input tensor, and paddings[D, 1] indicates how many sizes to be extended behind of the D th dimension of the input tensor. The padded size of each dimension D of the output is:
paddings[D, 0] + input_x.dim_size(D) + paddings[D, 1]
mode (str) – Specifies padding mode. The optional values are “CONSTANT”, “REFLECT”, “SYMMETRIC”. Default: “CONSTANT”.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, the tensor after padding.
If mode is “CONSTANT”, it fills the edge with 0, regardless of the values of the input_x. If the input_x is [[1,2,3], [4,5,6], [7,8,9]] and paddings is [[1,1], [2,2]], then the Outputs is [[0,0,0,0,0,0,0], [0,0,1,2,3,0,0], [0,0,4,5,6,0,0], [0,0,7,8,9,0,0], [0,0,0,0,0,0,0]].
If mode is “REFLECT”, it uses a way of symmetrical copying through the axis of symmetry to fill in. If the input_x is [[1,2,3], [4,5,6], [7,8,9]] and paddings is [[1,1], [2,2]], then the Outputs is [[6,5,4,5,6,5,4], [3,2,1,2,3,2,1], [6,5,4,5,6,5,4], [9,8,7,8,9,8,7], [6,5,4,5,6,5,4]].
If mode is “SYMMETRIC”, the filling method is similar to the “REFLECT”. It is also copied according to the symmetry axis, except that it includes the symmetry axis. If the input_x is [[1,2,3], [4,5,6], [7,8,9]] and paddings is [[1,1], [2,2]], then the Outputs is [[2,1,1,2,3,3,2], [2,1,1,2,3,3,2], [5,4,4,5,6,6,5], [8,7,7,8,9,9,8], [8,7,7,8,9,9,8]].
- Supported Platforms:
AscendGPU
Examples
>>> from mindspore import Tensor >>> from mindspore.ops import operations as P >>> import mindspore.nn as nn >>> import numpy as np >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.pad = nn.Pad(paddings=((1, 1), (2, 2)), mode="CONSTANT") ... def construct(self, x): ... return self.pad(x) >>> x = np.array([[0.3, 0.5, 0.2], [0.5, 0.7, 0.3]], dtype=np.float32) >>> pad = Net() >>> output = pad(Tensor(x)) >>> print(output) [[0. 0. 0. 0. 0. 0. 0. ] [0. 0. 0.3 0.5 0.2 0. 0. ] [0. 0. 0.5 0.7 0.3 0. 0. ] [0. 0. 0. 0. 0. 0. 0. ]]
-
class
tinyms.layers.Unfold(ksizes, strides, rates, padding='valid')[source]¶ Extract patches from images. The input tensor must be a 4-D tensor and the data format is NCHW.
- Parameters
ksizes (Union[tuple[int], list[int]]) – The size of sliding window, must be a tuple or a list of integers, and the format is [1, ksize_row, ksize_col, 1].
strides (Union[tuple[int], list[int]]) – Distance between the centers of the two consecutive patches, must be a tuple or list of int, and the format is [1, stride_row, stride_col, 1].
rates (Union[tuple[int], list[int]]) – In each extracted patch, the gap between the corresponding dimension pixel positions, must be a tuple or a list of integers, and the format is [1, rate_row, rate_col, 1].
padding (str) –
The type of padding algorithm, is a string whose value is “same” or “valid”, not case sensitive. Default: “valid”.
same: Means that the patch can take the part beyond the original image, and this part is filled with 0.
valid: Means that the taken patch area must be completely covered in the original image.
- Inputs:
input_x (Tensor) - A 4-D tensor whose shape is [in_batch, in_depth, in_row, in_col] and data type is number.
- Outputs:
Tensor, a 4-D tensor whose data type is same as input_x, and the shape is [out_batch, out_depth, out_row, out_col] where out_batch is the same as the in_batch.
out_depth = ksize_row * ksize_col * in_depth out_row = (in_row - (ksize_row + (ksize_row - 1) * (rate_row - 1))) // stride_row + 1 out_col = (in_col - (ksize_col + (ksize_col - 1) * (rate_col - 1))) // stride_col + 1
- Supported Platforms:
Ascend
Examples
>>> net = Unfold(ksizes=[1, 2, 2, 1], strides=[1, 2, 2, 1], rates=[1, 2, 2, 1]) >>> image = Tensor(np.ones([2, 3, 6, 6]), dtype=mstype.float16) >>> output = net(image) >>> print(output.shape) (2, 12, 2, 2)
-
class
tinyms.layers.Tril[source]¶ Returns a tensor with elements above the kth diagonal zeroed.
- Inputs:
x (Tensor) - The input tensor.
k (Int) - The index of diagonal. Default: 0
- Outputs:
Tensor, has the same type as input x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> x = Tensor(np.array([[1, 2], [3, 4]])) >>> tril = nn.Tril() >>> result = tril(x) >>> print(result) [[1 0] [3 4]]
-
class
tinyms.layers.Triu[source]¶ Returns a tensor with elements below the kth diagonal zeroed.
- Inputs:
x (Tensor) - The input tensor.
k (Int) - The index of diagonal. Default: 0
- Outputs:
Tensor, has the same type as input x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> x = Tensor(np.array([[1, 2], [3, 4]])) >>> triu = nn.Triu() >>> result = triu(x) >>> print(result) [[1 2] [0 4]]
-
class
tinyms.layers.ResizeBilinear[source]¶ Samples the input tensor to the given size or scale_factor by using bilinear interpolate.
- Inputs:
x (Tensor) - Tensor to be resized. Input tensor must be a 4-D tensor with shape: math:(batch, channels, height, width), with data type of float16 or float32.
size (Union[tuple[int], list[int]]): A tuple or list of 2 int elements ‘(new_height, new_width)’, the new size of the tensor. One and only one of size and scale_factor can be set to None. Default: None.
scale_factor (int): The scale factor of new size of the tensor. The value should be positive integer. One and only one of size and scale_factor can be set to None. Default: None.
align_corners (bool): If true, rescale input by ‘(new_height - 1) / (height - 1)’, which exactly aligns the 4 corners of images and resized images. If false, rescale by ‘new_height / height’. Default: False.
- Outputs:
Resized tensor. If size is set, the result is 4-D tensor with shape:math:(batch, channels, new_height, new_width) in float32. If scale is set, the result is 4-D tensor with shape:math:(batch, channels, scale_factor * height, scale_factor * width) in float32
- Supported Platforms:
Ascend
Examples
>>> tensor = Tensor([[[[1, 2, 3, 4], [5, 6, 7, 8]]]], mindspore.float32) >>> resize_bilinear = nn.ResizeBilinear() >>> result = resize_bilinear(tensor, size=(5,5)) >>> print(result.shape) (1, 1, 5, 5)
-
class
tinyms.layers.MatrixDiag[source]¶ Returns a batched diagonal tensor with a given batched diagonal values.
Assume x has \(k\) dimensions \([I, J, K, ..., N]\), then the output is a tensor of rank \(k+1\) with dimensions \([I, J, K, ..., N, N]\) where:
output[i, j, k, ..., m, n] = 1{m=n} * x[i, j, k, ..., n]
- Inputs:
x (Tensor) - The diagonal values. It can be one of the following data types: float32, float16, int32, int8, and uint8.
- Outputs:
Tensor, has the same type as input x. The shape must be x.shape + (x.shape[-1], ).
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.array([1, -1]), mstype.float32) >>> matrix_diag = nn.MatrixDiag() >>> output = matrix_diag(x) >>> print(output) [[ 1. 0.] [ 0. -1.]]
-
class
tinyms.layers.MatrixDiagPart[source]¶ Returns the batched diagonal part of a batched tensor.
Assume x has \(k\) dimensions \([I, J, K, ..., M, N]\), then the output is a tensor of rank \(k-1\) with dimensions \([I, J, K, ..., min(M, N)]\) where:
output[i, j, k, ..., n] = x[i, j, k, ..., n, n]
- Inputs:
x (Tensor) - The batched tensor. It can be one of the following data types: float32, float16, int32, int8, and uint8.
- Outputs:
Tensor, has the same type as input x. The shape must be x.shape[:-2] + [min(x.shape[-2:])].
- Supported Platforms:
Ascend
Examples
>>> x = Tensor([[[-1, 0], [0, 1]], [[-1, 0], [0, 1]], [[-1, 0], [0, 1]]], mindspore.float32) >>> matrix_diag_part = nn.MatrixDiagPart() >>> output = matrix_diag_part(x) >>> print(output) [[-1. 1.] [-1. 1.] [-1. 1.]]
-
class
tinyms.layers.MatrixSetDiag[source]¶ Modifies the batched diagonal part of a batched tensor.
Assume x has \(k+1\) dimensions \([I, J, K, ..., M, N]\) and diagonal has \(k\) dimensions \([I, J, K, ..., min(M, N)]\). Then the output is a tensor of rank \(k+1\) with dimensions \([I, J, K, ..., M, N]\) where:
output[i, j, k, ..., m, n] = diagnoal[i, j, k, ..., n] for m == n output[i, j, k, ..., m, n] = x[i, j, k, ..., m, n] for m != n
- Inputs:
x (Tensor) - The batched tensor. Rank k+1, where k >= 1. It can be one of the following data types: float32, float16, int32, int8, and uint8.
diagonal (Tensor) - The diagonal values. Must have the same type as input x. Rank k, where k >= 1.
- Outputs:
Tensor, has the same type and shape as input x.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor([[[-1, 0], [0, 1]], [[-1, 0], [0, 1]], [[-1, 0], [0, 1]]], mindspore.float32) >>> diagonal = Tensor([[-1., 2.], [-1., 1.], [-1., 1.]], mindspore.float32) >>> matrix_set_diag = nn.MatrixSetDiag() >>> output = matrix_set_diag(x, diagonal) >>> print(output) [[[-1. 0.] [ 0. 2.]] [[-1. 0.] [ 0. 1.]] [[-1. 0.] [ 0. 1.]]]
-
class
tinyms.layers.L1Regularizer(scale)[source]¶ Apply l1 regularization to weights
l1 regularization makes weights sparsity
Note
scale(regularization factor) should be a number which greater than 0
- Parameters
scale (int, float) – l1 regularization factor which greater than 0.
- Raises
ValueError – If scale(regularization factor) is not greater than 0. If scale(regularization factor) is math.inf or math.nan.
- Inputs:
weights (Tensor) - The input tensor
- Outputs:
Tensor, which dtype is higher precision data type between mindspore.float32 and weights dtype, and Tensor shape is ()
- Supported Platforms:
AscendGPUCPU
Examples
>>> scale = 0.5 >>> net = nn.L1Regularizer(scale) >>> weights = Tensor(np.array([[1.0, -2.0], [-3.0, 4.0]]).astype(np.float32)) >>> output = net(weights) >>> print(output.asnumpy()) 5.0
-
class
tinyms.layers.Embedding(vocab_size, embedding_size, use_one_hot=False, embedding_table='normal', dtype=mindspore.float32, padding_idx=None)[source]¶ A simple lookup table that stores embeddings of a fixed dictionary and size.
This module is often used to store word embeddings and retrieve them using indices. The input to the module is a list of indices, and the output is the corresponding word embeddings.
Note
When ‘use_one_hot’ is set to True, the type of the input must be mindspore.int32.
- Parameters
vocab_size (int) – Size of the dictionary of embeddings.
embedding_size (int) – The size of each embedding vector.
use_one_hot (bool) – Specifies whether to apply one_hot encoding form. Default: False.
embedding_table (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the embedding_table. Refer to class initializer for the values of string when a string is specified. Default: ‘normal’.
dtype (
mindspore.dtype) – Data type of input. Default: mindspore.float32.padding_idx (int, None) – When the padding_idx encounters index, the output embedding vector of this index will be initialized to zero. Default: None. The feature is inactivated.
- Inputs:
input (Tensor) - Tensor of shape \((\text{batch_size}, \text{input_length})\). The elements of the Tensor must be integer and not larger than vocab_size. Otherwise the corresponding embedding vector will be zero.
- Outputs:
Tensor of shape \((\text{batch_size}, \text{input_length}, \text{embedding_size})\).
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.Embedding(20000, 768, True) >>> input_data = Tensor(np.ones([8, 128]), mindspore.int32) >>> >>> # Maps the input word IDs to word embedding. >>> output = net(input_data) >>> result = output.shape >>> print(result) (8, 128, 768)
-
class
tinyms.layers.EmbeddingLookup(vocab_size, embedding_size, param_init='normal', target='CPU', slice_mode='batch_slice', manual_shapes=None, max_norm=None, sparse=True, vocab_cache_size=0)[source]¶ Returns a slice of the input tensor based on the specified indices.
Note
When ‘target’ is set to ‘CPU’, this module will use P.EmbeddingLookup().add_prim_attr(‘primitive_target’, ‘CPU’) which specified ‘offset = 0’ to lookup table. When ‘target’ is set to ‘DEVICE’, this module will use P.Gather() which specified ‘axis = 0’ to lookup table. In field slice mode, the manual_shapes must be given. It is a tuple ,where the element is vocab[i], vocab[i] is the row numbers for i-th part.
- Parameters
vocab_size (int) – Size of the dictionary of embeddings.
embedding_size (int) – The size of each embedding vector.
param_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the embedding_table. Refer to class initializer for the values of string when a string is specified. Default: ‘normal’.
target (str) – Specifies the target where the op is executed. The value must in [‘DEVICE’, ‘CPU’]. Default: ‘CPU’.
slice_mode (str) – The slicing way in semi_auto_parallel/auto_parallel. The value must get through nn.EmbeddingLookup. Default: nn.EmbeddingLookup.BATCH_SLICE.
manual_shapes (tuple) – The accompaniment array in field slice mode.
max_norm (Union[float, None]) – A maximum clipping value. The data type must be float16, float32 or None. Default: None
sparse (bool) – Using sparse mode. When ‘target’ is set to ‘CPU’, ‘sparse’ has to be true. Default: True.
vocab_cache_size (int) – Cache size of the dictionary of embeddings. Default: 0. It is valid only in parameter server trainning mode and ‘DEVICE’ target. And the moment parameter of corresponding optimizer will also be set to the cache size. In addition, it should be noted that it will cost the ‘DEVICE’ memory, so suggests setting a reasonable value to avoid insufficient memory.
- Inputs:
input_indices (Tensor) - The shape of tensor is \((y_1, y_2, ..., y_S)\). Specifies the indices of elements of the original Tensor. Values can be out of range of embedding_table, and the exceeding part will be filled with 0 in the output. Values does not support negative and the result is undefined if values are negative. Input_indices must only be a 2d tensor in this interface when run in semi auto parallel/auto parallel mode.
- Outputs:
Tensor, the shape of tensor is \((z_1, z_2, ..., z_N)\).
- Supported Platforms:
AscendCPU
Examples
>>> input_indices = Tensor(np.array([[1, 0], [3, 2]]), mindspore.int32) >>> result = nn.EmbeddingLookup(4,2)(input_indices) >>> print(result.shape) (2, 2, 2)
-
class
tinyms.layers.MultiFieldEmbeddingLookup(vocab_size, embedding_size, field_size, param_init='normal', target='CPU', slice_mode='batch_slice', feature_num_list=None, max_norm=None, sparse=True, operator='SUM')[source]¶ Returns a slice of input tensor based on the specified indices and the field ids. This operation supports looking up embeddings using multi hot and one hot fields simultaneously.
Note
When ‘target’ is set to ‘CPU’, this module will use P.EmbeddingLookup().add_prim_attr(‘primitive_target’, ‘CPU’) which specified ‘offset = 0’ to lookup table. When ‘target’ is set to ‘DEVICE’, this module will use P.Gather() which specified ‘axis = 0’ to lookup table. The vectors with the same field_ids will be combined by the ‘operator’, such as ‘SUM’, ‘MAX’ and ‘MEAN’. Ensure the input_values of the padded id is zero, so that they can be ignored. The final output will be zeros if the sum of absolute weight of the field is zero. This class only supports [‘table_row_slice’, ‘batch_slice’ and ‘table_column_slice’]. For the operation ‘MAX’ on device Ascend, there is a constrain where batch_size * (seq_length + field_size) < 3500.
- Parameters
vocab_size (int) – The size of the dictionary of embeddings.
embedding_size (int) – The size of each embedding vector.
field_size (int) – The field size of the final outputs.
param_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the embedding_table. Refer to class initializer for the values of string when a string is specified. Default: ‘normal’.
target (str) – Specifies the target where the op is executed. The value must in [‘DEVICE’, ‘CPU’]. Default: ‘CPU’.
slice_mode (str) – The slicing way in semi_auto_parallel/auto_parallel. The value must get through nn.EmbeddingLookup. Default: nn.EmbeddingLookup.BATCH_SLICE.
feature_num_list (tuple) – The accompaniment array in field slice mode. This is unused currently.
max_norm (Union[float, None]) – A maximum clipping value. The data type must be float16, float32 or None. Default: None
sparse (bool) – Using sparse mode. When ‘target’ is set to ‘CPU’, ‘sparse’ has to be true. Default: True.
operator (string) – The pooling method for the features in one field. Support ‘SUM, ‘MEAN’ and ‘MAX’
- Inputs:
input_indices (Tensor) - The shape of tensor is \((batch\_size, seq\_length)\). Specifies the indices of elements of the original Tensor. Input_indices must be a 2d tensor in this interface. Type is Int32, Int64.
input_values (Tensor) - The shape of tensor is \((batch\_size, seq\_length)\). Specifies the weights of elements of the input_indices. The lookout vector will multiply with the input_values. Type is Float32.
field_ids (Tensor) - The shape of tensor is \((batch\_size, seq\_length)\). Specifies the field id of elements of the input_indices. Type is Int32.
- Outputs:
Tensor, the shape of tensor is \((batch\_size, field\_size, embedding\_size)\). Type is Float32.
- Supported Platforms:
AscendGPU
Examples
>>> input_indices = Tensor([[2, 4, 6, 0, 0], [1, 3, 5, 0, 0]], mindspore.int32) >>> input_values = Tensor([[1, 1, 1, 0, 0], [1, 1, 1, 0, 0]], mindspore.float32) >>> field_ids = Tensor([[0, 1, 1, 0, 0], [0, 0, 1, 0, 0]], mindspore.int32) >>> net = nn.MultiFieldEmbeddingLookup(10, 2, field_size=2, operator='SUM') >>> out = net(input_indices, input_values, field_ids) >>> print(out.shape) (2, 2, 2)
-
class
tinyms.layers.AvgPool2d(kernel_size=1, stride=1, pad_mode='valid', data_format='NCHW')[source]¶ 2D average pooling for temporal data.
Applies a 2D average pooling over an input Tensor which can be regarded as a composition of 2D input planes.
Typically the input is of shape \((N_{in}, C_{in}, H_{in}, W_{in})\), AvgPool2d outputs regional average in the \((H_{in}, W_{in})\)-dimension. Given kernel size \(ks = (h_{ker}, w_{ker})\) and stride \(s = (s_0, s_1)\), the operation is as follows.
\[\text{output}(N_i, C_j, h, w) = \frac{1}{h_{ker} * w_{ker}} \sum_{m=0}^{h_{ker}-1} \sum_{n=0}^{w_{ker}-1} \text{input}(N_i, C_j, s_0 \times h + m, s_1 \times w + n)\]Note
pad_mode for training only supports “same” and “valid”.
- Parameters
kernel_size (Union[int, tuple[int]]) – The size of kernel used to take the average value. The data type of kernel_size must be int and the value represents the height and width, or a tuple of two int numbers that represent height and width respectively. Default: 1.
stride (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Default: 1.
pad_mode (str) –
The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
data_format (str) – The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: ‘NCHW’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, H_{out}, W_{out})\).
- Supported Platforms:
AscendGPU
Examples
>>> pool = nn.AvgPool2d(kernel_size=3, stride=1) >>> x = Tensor(np.random.randint(0, 10, [1, 2, 4, 4]), mindspore.float32) >>> output = pool(x) >>> print(output.shape) (1, 2, 2, 2)
-
class
tinyms.layers.MaxPool2d(kernel_size=1, stride=1, pad_mode='valid', data_format='NCHW')[source]¶ 2D max pooling operation for temporal data.
Applies a 2D max pooling over an input Tensor which can be regarded as a composition of 2D planes.
Typically the input is of shape \((N_{in}, C_{in}, H_{in}, W_{in})\), MaxPool2d outputs regional maximum in the \((H_{in}, W_{in})\)-dimension. Given kernel size \(ks = (h_{ker}, w_{ker})\) and stride \(s = (s_0, s_1)\), the operation is as follows.
\[\text{output}(N_i, C_j, h, w) = \max_{m=0, \ldots, h_{ker}-1} \max_{n=0, \ldots, w_{ker}-1} \text{input}(N_i, C_j, s_0 \times h + m, s_1 \times w + n)\]Note
pad_mode for training only supports “same” and “valid”.
- Parameters
kernel_size (Union[int, tuple[int]]) – The size of kernel used to take the max value, is an int number that represents height and width are both kernel_size, or a tuple of two int numbers that represent height and width respectively. Default: 1.
stride (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Default: 1.
pad_mode (str) –
The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
data_format (str) – The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: ‘NCHW’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, H_{out}, W_{out})\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> pool = nn.MaxPool2d(kernel_size=3, stride=1) >>> x = Tensor(np.random.randint(0, 10, [1, 2, 4, 4]), mindspore.float32) >>> output = pool(x) >>> print(output.shape) (1, 2, 2, 2)
-
class
tinyms.layers.AvgPool1d(kernel_size=1, stride=1, pad_mode='valid')[source]¶ 1D average pooling for temporal data.
Applies a 1D average pooling over an input Tensor which can be regarded as a composition of 1D input planes.
Typically the input is of shape \((N_{in}, C_{in}, L_{in})\), AvgPool1d outputs regional average in the \((L_{in})\)-dimension. Given kernel size \(ks = l_{ker}\) and stride \(s = s_0\), the operation is as follows.
\[\text{output}(N_i, C_j, l) = \frac{1}{l_{ker}} \sum_{n=0}^{l_{ker}-1} \text{input}(N_i, C_j, s_0 \times l + n)\]Note
pad_mode for training only supports “same” and “valid”.
- Parameters
kernel_size (int) – The size of kernel window used to take the average value, Default: 1.
stride (int) – The distance of kernel moving, an int number that represents the width of movement is strides, Default: 1.
pad_mode (str) –
The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, L_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, L_{out})\).
- Supported Platforms:
Ascend
Examples
>>> pool = nn.AvgPool1d(kernel_size=6, stride=1) >>> x = Tensor(np.random.randint(0, 10, [1, 3, 6]), mindspore.float32) >>> output = pool(x) >>> result = output.shape >>> print(result) (1, 3, 1)
-
class
tinyms.layers.MaxPool1d(kernel_size=1, stride=1, pad_mode='valid')[source]¶ 1D max pooling operation for temporal data.
Applies a 1D max pooling over an input Tensor which can be regarded as a composition of 1D planes.
Typically the input is of shape \((N_{in}, C_{in}, L_{in})\), MaxPool1d outputs regional maximum in the \((L_{in})\)-dimension. Given kernel size \(ks = (l_{ker})\) and stride \(s = (s_0)\), the operation is as follows.
\[\text{output}(N_i, C_j, l) = \max_{n=0, \ldots, l_{ker}-1} \text{input}(N_i, C_j, s_0 \times l + n)\]Note
pad_mode for training only supports “same” and “valid”.
- Parameters
kernel_size (int) – The size of kernel used to take the max value, Default: 1.
stride (int) – The distance of kernel moving, an int number that represents the width of movement is stride, Default: 1.
pad_mode (str) –
The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
same: Adopts the way of completion. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
- Inputs:
input (Tensor) - Tensor of shape \((N, C, L_{in})\).
- Outputs:
Tensor of shape \((N, C, L_{out}))\).
- Supported Platforms:
Ascend
Examples
>>> max_pool = nn.MaxPool1d(kernel_size=3, stride=1) >>> x = Tensor(np.random.randint(0, 10, [1, 2, 4]), mindspore.float32) >>> output = max_pool(x) >>> result = output.shape >>> print(result) (1, 2, 2)
-
class
tinyms.layers.ReduceLogSumExp(axis, keep_dims=False)[source]¶ Reduces a dimension of a tensor by calculating exponential for all elements in the dimension, then calculate logarithm of the sum.
The dtype of the tensor to be reduced is number.
\[ReduceLogSumExp(x) = \log(\sum(e^x))\]- Parameters
- Inputs:
x (Tensor) - The input tensor. With float16 or float32 data type.
- Outputs:
Tensor, has the same dtype as the x.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the sum of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.random.randn(3, 4, 5, 6).astype(np.float32)) >>> op = nn.ReduceLogSumExp(1, keep_dims=True) >>> output = op(input_x) >>> print(output.shape) (3, 1, 5, 6)
-
class
tinyms.layers.Range(start, limit=None, delta=1)[source]¶ Creates a sequence of numbers in range [start, limit) with step size delta.
The size of output is \(\left \lfloor \frac{limit-start}{delta} \right \rfloor + 1\) and delta is the gap between two values in the tensor.
\[out_{i+1} = out_{i} +delta\]- Parameters
start (Union[int, float]) – If limit is None, the value acts as limit in the range and first entry defaults to 0. Otherwise, it acts as first entry in the range.
limit (Union[int, float]) – Acts as upper limit of sequence. If None, defaults to the value of start while set the first entry of the range to 0. It can not be equal to start.
delta (Union[int, float]) – Increment of the range. It can not be equal to zero. Default: 1.
- Outputs:
Tensor, the dtype is int if the dtype of start, limit and delta all are int. Otherwise, dtype is float.
- Supported Platforms:
Ascend
Examples
>>> net = nn.Range(1, 8, 2) >>> output = net() >>> print(output) [1 3 5 7]
-
class
tinyms.layers.LGamma[source]¶ Calculates LGamma using Lanczos’ approximation referring to “A Precision Approximation of the Gamma Function”. The algorithm is:
\[\begin{split}\begin{array}{ll} \\ lgamma(z + 1) = \frac{(\log(2) + \log(pi))}{2} + (z + 1/2) * log(t(z)) - t(z) + A(z) \\ t(z) = z + kLanczosGamma + 1/2 \\ A(z) = kBaseLanczosCoeff + \sum_{k=1}^n \frac{kLanczosCoefficients[i]}{z + k} \end{array}\end{split}\]However, if the input is less than 0.5 use Euler’s reflection formula:
\[lgamma(x) = \log(pi) - lgamma(1-x) - \log(abs(sin(pi * x)))\]And please note that
\[lgamma(+/-inf) = +inf\]Thus, the behaviour of LGamma follows: when x > 0.5, return log(Gamma(x)) when x < 0.5 and is not an interger, return the real part of Log(Gamma(x)) where Log is the complex logarithm when x is an integer less or equal to 0, return +inf when x = +/- inf, return +inf
- Supported Platforms:
AscendGPU- Inputs:
x (Tensor) - The input tensor. Only float16, float32 are supported.
- Outputs:
Tensor, has the same shape and dtype as the x.
Examples
>>> input_x = Tensor(np.array([2, 3, 4]).astype(np.float32)) >>> op = nn.LGamma() >>> output = op(input_x) >>> print(output) [3.5762787e-07 6.9314754e-01 1.7917603e+00]
-
class
tinyms.layers.DiGamma[source]¶ Calculates Digamma using Lanczos’ approximation referring to “A Precision Approximation of the Gamma Function”. The algorithm is:
\[ \begin{align}\begin{aligned}digamma(z + 1) = log(t(z)) + A'(z) / A(z) - kLanczosGamma / t(z)\\t(z) = z + kLanczosGamma + 1/2\\A(z) = kBaseLanczosCoeff + \sum_{k=1}^n \frac{kLanczosCoefficients[i]}{z + k}\\A'(z) = \sum_{k=1}^n \frac{kLanczosCoefficients[i]}{{z + k}^2}\end{aligned}\end{align} \]However, if the input is less than 0.5 use Euler’s reflection formula:
\[digamma(x) = digamma(1 - x) - pi * cot(pi * x)\]- Inputs:
x (Tensor[Number]) - The input tensor. Only float16, float32 are supported.
- Outputs:
Tensor, has the same shape and dtype as the x.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([2, 3, 4]).astype(np.float32)) >>> op = nn.DiGamma() >>> output = op(input_x) >>> print(output) [0.42278463 0.92278427 1.2561178]
-
class
tinyms.layers.IGamma[source]¶ Calculates lower regularized incomplete Gamma function. The lower regularized incomplete Gamma function is defined as:
\[P(a, x) = gamma(a, x) / Gamma(a) = 1 - Q(a, x)\]where
\[gamma(a, x) = \int_0^x t^{a-1} \exp^{-t} dt\]is the lower incomplete Gamma function.
Above \(Q(a, x)\) is the upper regularized complete Gamma function.
- Supported Platforms:
AscendGPU- Inputs:
a (Tensor) - The input tensor. With float32 data type. a should have the same dtype with x.
x (Tensor) - The input tensor. With float32 data type. x should have the same dtype with a.
- Outputs:
Tensor, has the same dtype as a and x.
Examples
>>> input_a = Tensor(np.array([2.0, 4.0, 6.0, 8.0]).astype(np.float32)) >>> input_x = Tensor(np.array([2.0, 3.0, 4.0, 5.0]).astype(np.float32)) >>> igamma = nn.IGamma() >>> output = igamma(input_a, input_x) >>> print (output) [0.593994 0.35276785 0.21486944 0.13337152]
-
class
tinyms.layers.LBeta[source]¶ This is semantically equal to lgamma(x) + lgamma(y) - lgamma(x + y).
The method is more accurate for arguments above 8. The reason for accuracy loss in the naive computation is catastrophic cancellation between the lgammas. This method avoids the numeric cancellation by explicitly decomposing lgamma into the Stirling approximation and an explicit log_gamma_correction, and cancelling the large terms from the Striling analytically.
- Supported Platforms:
AscendGPU- Inputs:
x (Tensor) - The input tensor. With float16 or float32 data type. x should have the same dtype with y.
y (Tensor) - The input tensor. With float16 or float32 data type. y should have the same dtype with x.
- Outputs:
Tensor, has the same dtype as x and y.
Examples
>>> input_x = Tensor(np.array([2.0, 4.0, 6.0, 8.0]).astype(np.float32)) >>> input_y = Tensor(np.array([2.0, 3.0, 14.0, 15.0]).astype(np.float32)) >>> lbeta = nn.LBeta() >>> output = lbeta(input_y, input_x) >>> print(output) [-1.7917596 -4.094345 -12.000229 -14.754799]
-
class
tinyms.layers.MatMul(transpose_x1=False, transpose_x2=False)[source]¶ Multiplies matrix x1 by matrix x2.
If both x1 and x2 are 1-dimensional, the dot product is returned.
If the dimensions of x1 and x2 are all not greater than 2, the matrix-matrix product will be returned. Note if one of ‘x1’ and ‘x2’ is 1-dimensional, the argument will first be expanded to 2 dimension. After the matrix multiply, the expanded dimension will be removed.
If at least one of x1 and x2 is N-dimensional (N>2), the none-matrix dimensions(batch) of inputs will be broadcasted and must be broadcastable. Note if one of ‘x1’ and ‘x2’ is 1-dimensional, the argument will first be expanded to 2 dimension and then the none-matrix dimensions will be broadcasted. After the matrix multiply, the expanded dimension will be removed. For example, if x1 is a \((j \times 1 \times n \times m)\) tensor and x2 is a \((k \times m \times p)\) tensor, the output will be a \((j \times k \times n \times p)\) tensor.
- Parameters
- Inputs:
input_x1 (Tensor) - The first tensor to be multiplied.
input_x2 (Tensor) - The second tensor to be multiplied.
- Outputs:
Tensor, the shape of the output tensor depends on the dimension of input tensors.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.MatMul() >>> input_x1 = Tensor(np.ones(shape=[3, 2, 3]), mindspore.float32) >>> input_x2 = Tensor(np.ones(shape=[3, 4]), mindspore.float32) >>> output = net(input_x1, input_x2) >>> print(output.shape) (3, 2, 4)
-
class
tinyms.layers.Moments(axis=None, keep_dims=None)[source]¶ Calculates the mean and variance of x.
- Parameters
- Inputs:
input_x (Tensor) - The tensor to be calculated. Only float16 and float32 are supported.
- Outputs:
mean (Tensor) - The mean of input x, with the same date type as input x.
variance (Tensor) - The variance of input x, with the same date type as input x.
- Supported Platforms:
Ascend
Examples
>>> net = nn.Moments(axis=3, keep_dims=True) >>> input_x = Tensor(np.array([[[[1, 2, 3, 4], [3, 4, 5, 6]]]]), mindspore.float32) >>> output = net(input_x) >>> print(output) (Tensor(shape=[1, 1, 2, 1], dtype=Float32, value= [[[[ 2.50000000e+00], [ 4.50000000e+00]]]]), Tensor(shape=[1, 1, 2, 1], dtype=Float32, value= [[[[ 1.25000000e+00], [ 1.25000000e+00]]]]))
-
class
tinyms.layers.MatInverse[source]¶ Calculates the inverse of Positive-Definite Hermitian matrix using Cholesky decomposition.
- Supported Platforms:
GPU- Inputs:
a (Tensor[Number]) - The input tensor. It must be a positive-definite matrix. With float16 or float32 data type.
- Outputs:
Tensor, has the same dtype as the a.
Examples
>>> input_a = Tensor(np.array([[4, 12, -16], [12, 37, -43], [-16, -43, 98]]).astype(np.float32)) >>> op = nn.MatInverse() >>> output = op(input_a) >>> print(output) [[49.36112 -13.555558 2.1111116] [-13.555558 3.7777784 -0.5555557] [2.1111116 -0.5555557 0.11111111]]
-
class
tinyms.layers.MatDet[source]¶ Calculates the determinant of Positive-Definite Hermitian matrix using Cholesky decomposition.
- Supported Platforms:
GPU- Inputs:
a (Tensor[Number]) - The input tensor. It must be a positive-definite matrix. With float16 or float32 data type.
- Outputs:
Tensor, has the same dtype as the a.
Examples
>>> input_a = Tensor(np.array([[4, 12, -16], [12, 37, -43], [-16, -43, 98]]).astype(np.float32)) >>> op = nn.MatDet() >>> output = op(input_a) >>> print(output) 35.999996
-
class
tinyms.layers.Conv2dBnAct(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros', has_bn=False, momentum=0.997, eps=1e-05, activation=None, alpha=0.2, after_fake=True)[source]¶ A combination of convolution, Batchnorm, and activation layer.
This part is a more detailed overview of Conv2d operation.
- Parameters
in_channels (int) – The number of input channel \(C_{in}\).
out_channels (int) – The number of output channel \(C_{out}\).
kernel_size (Union[int, tuple]) – The data type is int or a tuple of 2 integers. Specifies the height and width of the 2D convolution window. Single int means the value is for both height and width of the kernel. A tuple of 2 ints means the first value is for the height and the other is for the width of the kernel.
stride (int) – Specifies stride for all spatial dimensions with the same value. The value of stride must be greater than or equal to 1 and lower than any one of the height and width of the input. Default: 1.
pad_mode (str) – Specifies padding mode. The optional values are “same”, “valid”, “pad”. Default: “same”.
padding (int) – Implicit paddings on both sides of the input. Default: 0.
dilation (int) – Specifies the dilation rate to use for dilated convolution. If set to be \(k > 1\), there will be \(k - 1\) pixels skipped for each sampling location. Its value must be greater than or equal to 1 and lower than any one of the height and width of the input. Default: 1.
group (int) – Splits filter into groups, in_ channels and out_channels must be divisible by the number of groups. Default: 1.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the convolution kernel. It can be a Tensor, a string, an Initializer or a number. When a string is specified, values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as well as constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’ and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values of Initializer for more details. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. Possible Initializer and string are the same as ‘weight_init’. Refer to the values of Initializer for more details. Default: ‘zeros’.
has_bn (bool) – Specifies to used batchnorm or not. Default: False.
momentum (float) – Momentum for moving average for batchnorm, must be [0, 1]. Default:0.9
eps (float) – Term added to the denominator to improve numerical stability for batchnorm, should be greater than 0. Default: 1e-5.
activation (Union[str, Cell, Primitive]) – Specifies activation type. The optional values are as following: ‘softmax’, ‘logsoftmax’, ‘relu’, ‘relu6’, ‘tanh’, ‘gelu’, ‘sigmoid’, ‘prelu’, ‘leakyrelu’, ‘hswish’, ‘hsigmoid’. Default: None.
alpha (float) – Slope of the activation function at x < 0 for LeakyReLU. Default: 0.2.
after_fake (bool) – Determine whether there must be a fake quantization operation after Cond2dBnAct.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, H_{out}, W_{out})\).
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.Conv2dBnAct(120, 240, 4, has_bn=True, activation='relu') >>> input = Tensor(np.ones([1, 120, 1024, 640]), mindspore.float32) >>> result = net(input) >>> output = result.shape >>> print(output) (1, 240, 1024, 640)
-
class
tinyms.layers.DenseBnAct(in_channels, out_channels, weight_init='normal', bias_init='zeros', has_bias=True, has_bn=False, momentum=0.9, eps=1e-05, activation=None, alpha=0.2, after_fake=True)[source]¶ A combination of Dense, Batchnorm, and the activation layer.
This part is a more detailed overview of Dense op.
- Parameters
in_channels (int) – The number of channels in the input space.
out_channels (int) – The number of channels in the output space.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – The trainable weight_init parameter. The dtype is same as input. The values of str refer to the function initializer. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – The trainable bias_init parameter. The dtype is same as input. The values of str refer to the function initializer. Default: ‘zeros’.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: True.
has_bn (bool) – Specifies to use batchnorm or not. Default: False.
momentum (float) – Momentum for moving average for batchnorm, must be [0, 1]. Default:0.9
eps (float) – Term added to the denominator to improve numerical stability for batchnorm, should be greater than 0. Default: 1e-5.
activation (Union[str, Cell, Primitive]) – Specifies activation type. The optional values are as following: ‘softmax’, ‘logsoftmax’, ‘relu’, ‘relu6’, ‘tanh’, ‘gelu’, ‘sigmoid’, ‘prelu’, ‘leakyrelu’, ‘hswish’, ‘hsigmoid’. Default: None.
alpha (float) – Slope of the activation function at x < 0 for LeakyReLU. Default: 0.2.
after_fake (bool) – Determine whether there must be a fake quantization operation after DenseBnAct.
- Inputs:
input (Tensor) - Tensor of shape \((N, in\_channels)\).
- Outputs:
Tensor of shape \((N, out\_channels)\).
- Supported Platforms:
Ascend
Examples
>>> net = nn.DenseBnAct(3, 4) >>> input = Tensor(np.random.randint(0, 255, [2, 3]), mindspore.float32) >>> result = net(input) >>> output = result.shape >>> print(output) (2, 4)
tinyms.model¶
-
class
tinyms.model.Model(network)[source]¶ High-Level API for Training or Evaluation.
Model groups layers into an object with training and inference features.
- Parameters
network (layers.Layer) – A training or testing network.
Examples
>>> from tinyms.model import Model, lenet5 >>> form tinyms.losses import SoftmaxCrossEntropyWithLogits >>> from tinyms.optimizers import Momentum >>> >>> net = lenet5(class_num=10) >>> model = Model(net) >>> net_loss = SoftmaxCrossEntropyWithLogits() >>> net_opt = Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> model.compile(loss_fn=net_loss, optimizer=net_opt, metrics=None) >>> # For details about how to build the dataset, please refer to the API document on the official website. >>> ds_train = create_custom_dataset() >>> model.train(2, ds_train)
-
compile(loss_fn=None, optimizer=None, metrics=None, eval_network=None, amp_level='O0', **kwargs)[source]¶ High-Level API for configure the train or eval network.
- Parameters
loss_fn (layers.Layer) – Objective function, if loss_fn is None, the network should contain the logic of loss and grads calculation, and the logic of parallel if needed. Default: None.
optimizer (layers.Layer) – Optimizer for updating the weights. Default: None.
metrics (Union[dict, set]) – A Dictionary or a set of metrics to be evaluated by the model during training and testing. eg: {‘accuracy’, ‘recall’}. Default: None.
eval_network (layers.Layer) – Network for evaluation. If not defined, network and loss_fn would be wrapped as eval_network. Default: None.
amp_level (str) –
Option for argument level in mindspore.amp.build_train_network, level for mixed precision training. Supports [“O0”, “O2”, “O3”, “auto”]. Default: “O0”.
O0: Do not change.
O2: Cast network to float16, keep batchnorm run in float32, using dynamic loss scale.
O3: Cast network to float16, with additional property ‘keep_batchnorm_fp32=False’.
auto: Set to level to recommended level in different devices. Set level to O2 on GPU, Set
level to O3 Ascend. The recommended level is choose by the export experience, cannot always generalize. User should specify the level for special network.
O2 is recommended on GPU, O3 is recommended on Ascend.
-
eval(valid_dataset, callbacks=None, dataset_sink_mode=True)[source]¶ Evaluation API where the iteration is controlled by python front-end.
Configure to pynative mode or CPU, the evaluating process will be performed with dataset non-sink mode.
Note
If dataset_sink_mode is True, data will be sent to device. If device is Ascend, features of data will be transferred one by one. The limitation of data transmission per time is 256M.
- Parameters
- Returns
Dict, which returns the loss value and metrics values for the model in the test mode.
Examples
>>> dataset = create_custom_dataset() >>> net = Net() >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=None, metrics={'acc'}) >>> acc = model.eval(dataset, dataset_sink_mode=False)
-
export(*inputs, file_name, file_format='MINDIR', **kwargs)[source]¶ Export the TinyMS prediction model to a file in the specified format.
- Parameters
inputs (Tensor) – Inputs of the net.
file_name (str) – File name of the model to be exported.
file_name – File name of the model to be exported.
file_format (str) –
MindSpore currently supports AIR, ONNX and MINDIR format for exported model. Default: MINDIR.
AIR: Ascend Intermediate Representation. An intermediate representation format of Ascend model.
Recommended suffix for output file is ‘.air’.
ONNX: Open Neural Network eXchange. An open format built to represent machine learning models.
Recommended suffix for output file is ‘.onnx’.
MINDIR: MindSpore Native Intermediate Representation for Anf. An intermediate representation format
for MindSpore models. Recommended suffix for output file is ‘.mindir’.
kwargs (dict) –
Configuration options dictionary.
quant_mode: The mode of quant.
mean: Input data mean. Default: 127.5.
std_dev: Input data variance. Default: 127.5.
-
infer_predict_layout(*predict_data)[source]¶ Generate parameter layout for the predict network in auto or semi auto parallel mode.
Data could be a single tensor or multiple tensors.
Note
Batch data should be put together in one tensor.
- Parameters
predict_data (Tensor) – One tensor or multiple tensors of predict data.
- Returns
Parameter layout dictionary used for load distributed checkpoint
- Return type
parameter_layout_dict (dict)
Examples
>>> context.set_context(mode=context.GRAPH_MODE) >>> context.set_auto_parallel_context(full_batch=True, parallel_mode=ParallelMode.SEMI_AUTO_PARALLEL) >>> input_data = Tensor(np.random.randint(0, 255, [1, 3, 224, 224]), mindspore.float32) >>> model = Model(Net()) >>> model.infer_predict_layout(input_data)
-
load_checkpoint(ckpt_file_name, strict_load=False)[source]¶ Loads checkpoint info from a specified file.
- Parameters
- Returns
Dict, key is parameter name, value is a Parameter.
- Raises
ValueError – Checkpoint file is incorrect.
Examples
>>> ckpt_file_name = "./checkpoint/LeNet5-1_32.ckpt" >>> param_dict = model.load_checkpoint(ckpt_file_name)
-
predict(*predict_data)[source]¶ Generate output predictions for the input samples.
Data could be a single tensor, a list of tensor, or a tuple of tensor.
Note
Batch data should be put together in one tensor.
- Parameters
predict_data – The predict data, can be bool, int, float, str, None, tensor, or tuple, list and dict that store these types.
- Returns
Tensor, array(s) of predictions.
Examples
>>> input_data = Tensor(np.random.randint(0, 255, [1, 1, 32, 32]), mindspore.float32) >>> model = Model(Net()) >>> result = model.predict(input_data)
-
train(epoch, train_dataset, callbacks=None, dataset_sink_mode=True, sink_size=-1)[source]¶ Training API where the iteration is controlled by python front-end.
When setting pynative mode or CPU, the training process will be performed with dataset not sink.
Note
If dataset_sink_mode is True, data will be sent to device. If device is Ascend, features of data will be transferred one by one. The limitation of data transmission per time is 256M. If sink_size > 0, each epoch the dataset can be traversed unlimited times until you get sink_size elements of the dataset. Next epoch continues to traverse from the end position of the previous traversal.
- Parameters
epoch (int) – Generally, total number of iterations on the data per epoch. When dataset_sink_mode is set to true and sink_size>0, each epoch sink sink_size steps on the data instead of total number of iterations.
train_dataset (Dataset) – A training dataset iterator. If there is no loss_fn, a tuple with multiple data (data1, data2, data3, …) should be returned and passed to the network. Otherwise, a tuple (data, label) should be returned. The data and label would be passed to the network and loss function respectively.
callbacks (list) – List of callback objects which should be executed while training. Default: None.
dataset_sink_mode (bool) – Determines whether to pass the data through dataset channel. Default: True. Configure pynative mode or CPU, the training process will be performed with dataset not sink.
sink_size (int) – Control the amount of data in each sink. If sink_size = -1, sink the complete dataset for each epoch. If sink_size > 0, sink sink_size data for each epoch. If dataset_sink_mode is False, set sink_size as invalid. Default: -1.
Examples
>>> from mindspore.train.loss_scale_manager import FixedLossScaleManager >>> dataset = create_custom_dataset() >>> net = Net() >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> loss_scale_manager = FixedLossScaleManager() >>> optim = Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> model = Model(net, loss_fn=loss, optimizer=optim, metrics=None, loss_scale_manager=loss_scale_manager) >>> model.train(2, dataset)
-
tinyms.model.lenet5(class_num=10)[source]¶ Get LeNet5 neural network.
- Parameters
class_num (int) – Class number.
- Returns
layers.Layer, layer instance of LeNet5 neural network.
Examples
>>> net = lenet5(class_num=10)
-
class
tinyms.model.LeNet(class_num=10, channel_num=1)[source]¶ LeNet architecture.
- Parameters
- Returns
Tensor, output tensor.
Examples
>>> LeNet(class_num=10)
-
tinyms.model.resnet50(class_num=10)[source]¶ Get ResNet50 neural network.
- Parameters
class_num (int) – Class number. Default: 10.
- Returns
layers.Layer, layer instance of ResNet50 neural network.
Examples
>>> net = resnet50(10)
-
class
tinyms.model.ResNet(block, layer_nums, in_channels, out_channels, strides, num_classes)[source]¶ ResNet architecture.
- Parameters
block (layers.Layer) – Block for network.
layer_nums (list) – Numbers of block in different layers.
in_channels (list) – Input channel in each layer.
out_channels (list) – Output channel in each layer.
strides (list) – Stride size in each layer.
num_classes (int) – The number of classes that the training images are belonging to.
- Returns
Tensor, output tensor.
Examples
>>> ResNet(ResidualBlock, >>> [3, 4, 6, 3], >>> [64, 256, 512, 1024], >>> [256, 512, 1024, 2048], >>> [1, 2, 2, 2], >>> 10)
-
tinyms.model.mobilenetv2(class_num=1000, is_training=True)[source]¶ Get MobileNetV2 instance for training.
-
tinyms.model.mobilenetv2_infer(class_num=1000)[source]¶ Get MobileNetV2 instance for predict.
- Parameters
class_num (int) – The number of classes.
- Returns
MobileNetV2 instance.
-
class
tinyms.model.MobileNetV2(class_num=1000, width_mult=1.0, round_nearest=8, input_channel=32, last_channel=1280, is_training=True)[source]¶ MobileNetV2 architecture.
- Parameters
- Returns
Tensor, output tensor.
-
class
tinyms.model.SSD300(backbone, class_num=21, is_training=True)[source]¶ SSD300 Network. Default backbone is MobileNetV2.
- Parameters
backbone (layers.Layer) – backbone of ssd300 model.
class_num (int) – number of classes. Default: 21.
is_training (bool) – Specify if in training step. Default: True.
- Returns
Tensor, localization predictions. Tensor, class conf scores.
-
tinyms.model.cycle_gan(G_A, G_B)[source]¶ Get Cycle GAN network.
- Parameters
G_A (layers.Layer) – The generator net, currently it should be in [resnet, unet].
G_B (layers.Layer) – The generator net, currently it should be in [resnet, unet].
- Returns
Cycle GAN instance.
Examples
>>> gan_net = cycle_gan(G_A, G_B)
-
tinyms.model.cycle_gan_infer(g_model='resnet')[source]¶ Get Cycle GAN network for predict.
- Parameters
G_A (layers.Layer) – The generator net, currently it should be in [resnet, unet].
G_B (layers.Layer) – The generator net, currently it should be in [resnet, unet].
- Returns
Cycle GAN instance.
Examples
>>> gan_net = cycle_gan(G_A, G_B)
tinyms.initializers¶
-
class
tinyms.initializers.Initializer(**kwargs)[source]¶ The base class of the initializer. Initialization of tensor basic attributes and model weight values.
- Parameters
kwargs (dict) – Keyword arguments for Initializer.
- Returns
Array, an array after being initialized.
-
tinyms.initializers.initializer(init, shape=None, dtype=mindspore.float32)[source]¶ Create and initialize a tensor.
- Parameters
init (Union[Tensor, str, Initializer, numbers.Number]) –
Initialize value.
str: The init should be the alias of the class inheriting from Initializer and the corresponding class will be called.
Initializer: The init should be the class inheriting from Initializer to initialize tensor.
numbers.Number: The Constant will be called to initialize tensor.
shape (Union[tuple, list, int]) – A list of integers, a tuple of integers or an integer as the shape of output. Default: None.
dtype (
mindspore.dtype) – The type of data in initialized tensor. Default: mindspore.float32.
- Returns
Union[Tensor, MetaTensor], When init is Tensor, the return is Tensor object, otherwise the return is Initialize object.
Examples
>>> tensor = initializer('ones', [1, 2, 3], mindspore.float32) >>> tensor = initializer(One(), [1, 2, 3], mindspore.float32) >>> tensor = initializer(0, [1, 2, 3], mindspore.float32)
-
class
tinyms.initializers.TruncatedNormal(sigma=0.01)[source]¶ Initialize a truncated normal distribution which is a bounded normal distribution within N(low, high).
- Parameters
sigma (float) – The sigma of the array. Default: 0.01.
- Returns
Array, truncated normal array.
-
class
tinyms.initializers.Normal(sigma=0.01)[source]¶ Initialize a normal array, and obtain values N(0, sigma) from the uniform distribution to fill the input tensor.
- Parameters
sigma (float) – The sigma of the array. Default: 0.01.
- Returns
Array, normal array.
-
class
tinyms.initializers.Uniform(scale=0.07)[source]¶ Initialize a uniform array, and obtain values U(-scale, scale) from the uniform distribution to fill the input tensor.
- Parameters
scale (float) – The scale of the array. Default: 0.07.
- Returns
Array, uniform array.
-
class
tinyms.initializers.HeUniform(negative_slope=0, mode='fan_in', nonlinearity='leaky_relu')[source]¶ Initialize the array with He kaiming uniform algorithm, and from a uniform distribution collect samples within U[-boundary, boundary] The boundary is defined as :
where \(boundary = \sqrt{\frac{6}{(1 + a^2) \times \text{fan_in}}}\).
-
class
tinyms.initializers.HeNormal(negative_slope=0, mode='fan_in', nonlinearity='leaky_relu')[source]¶ Initialize the array with He kaiming Normal algorithm, and from a normal distribution collect samples within N(0, sigma).
-
class
tinyms.initializers.XavierUniform(gain=1)[source]¶ Initialize the array with xavier uniform algorithm, and from a uniform distribution collect samples within U[-boundary, boundary] The boundary is defined as :
where \(boundary = gain * \sqrt{\frac{6}{n_{in} + n_{out}}}\).
where \(n_{in}\) is the number of input units in the weight tensor. where \(n_{out}\) is the number of output units in the weight tensor.
- Parameters
gain (Array) – The array to be assigned. Default: 1.
- Returns
Array, assigned array.
-
class
tinyms.initializers.One(**kwargs)[source]¶ Initialize the array to one.
- Parameters
arr (Array) – The array to be assigned.
- Returns
Array, assigned array.
-
class
tinyms.initializers.Zero(**kwargs)[source]¶ Initialize the array to zero.
- Parameters
arr (Array) – The array to be assigned.
- Returns
Array, an array after being assigned.
-
class
tinyms.initializers.Constant(value)[source]¶ Initialize a constant.
- Parameters
value (Union[int, numpy.ndarray]) – The value to initialize.
- Returns
Array, an array after being assigned.
tinyms.losses¶
Losses module. Loss function in machine learning is the target of the model. It shows how well the model works on a dataset and the optimization target which the optimizer is searching.
-
tinyms.losses.net_with_loss(net)[source]¶ This function is provided for AI beginners who are not familiar with which loss should be chosen for the network to be trained. Instead of choosing different loss function, users could directly get the best suitable loss function by specifying network.
- Parameters
net (layers.Layer) – The instance of network to be trained.
- Raises
TypeError – When network type is not supported.
Note
Currently this function only supports few networks, if the network type is not supported, the system would raise TypeError exception.
Examples
>>> from tinyms.model import ssd300 >>> from tinyms.losses import net_with_loss >>> >>> net = ssd300() >>> net_loss = net_with_loss(net)
-
class
tinyms.losses.SSD300WithLoss(network)[source]¶ Provide SSD300 training loss through network.
- Parameters
network (layers.Layer) – The training network.
- Returns
Tensor, the loss of the network.
Examples
>>> from tinyms.model import ssd300 >>> from tinyms.losses import SSD300WithLoss >>> >>> net = SSD300WithLoss(ssd300())
-
class
tinyms.losses.CrossEntropyWithLabelSmooth(smooth_factor=0.0, num_classes=1000)[source]¶ CrossEntropyWith LabelSmooth.
- Parameters
- Returns
None.
Examples
>>> CrossEntropyWithLabelSmooth(smooth_factor=0., num_classes=1000)
-
class
tinyms.losses.CycleGANGeneratorLoss(generator, D_A, D_B)[source]¶ Cycle GAN generator loss.
- Parameters
generator (layers.Layer) – Generator of CycleGAN.
D_A (layers.Layer) – The discriminator network of domain A to domain B.
D_B (layers.Layer) – The discriminator network of domain B to domain A.
- Outputs:
Tuple Tensor, the losses of generator.
-
class
tinyms.losses.CycleGANDiscriminatorLoss(D_A, D_B, reduction='none')[source]¶ Cycle GAN discriminator loss.
- Parameters
D_A (layers.Layer) – The discriminator network of domain A to domain B.
D_B (layers.Layer) – The discriminator network of domain B to domain A.
reduction (str) – The discriminator network of reduction. Default: none.
- Outputs:
the loss of discriminator.
-
class
tinyms.losses.L1Loss(reduction='mean')[source]¶ L1Loss creates a criterion to measure the mean absolute error (MAE) between \(x\) and \(y\) element-wise, where \(x\) is the input Tensor and \(y\) is the target Tensor.
For simplicity, let \(x\) and \(y\) be 1-dimensional Tensor with length \(N\), the unreduced loss (i.e. with argument reduction set to ‘none’) of \(x\) and \(y\) is given as:
\[L(x, y) = \{l_1,\dots,l_N\}, \quad \text{with } l_n = \left| x_n - y_n \right|\]When argument reduction is ‘mean’, the mean value of \(L(x, y)\) will be returned. When argument reduction is ‘sum’, the sum of \(L(x, y)\) will be returned. \(N\) is the batch size.
- Parameters
reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. Default: “mean”.
- Inputs:
input_data (Tensor) - Tensor of shape \((x_1, x_2, ..., x_R)\). The data type must be float16 or float32.
target_data (Tensor) - Tensor of shape \((y_1, y_2, ..., y_S)\). The data type must be float16 or float32.
- Outputs:
Tensor, loss float tensor.
- Supported Platforms:
AscendGPU
Examples
>>> loss = nn.L1Loss() >>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32) >>> output = loss(input_data, target_data) >>> print(output) 0.33333334
-
class
tinyms.losses.MSELoss(reduction='mean')[source]¶ MSELoss creates a criterion to measure the mean squared error (squared L2-norm) between \(x\) and \(y\) element-wise, where \(x\) is the input and \(y\) is the target.
For simplicity, let \(x\) and \(y\) be 1-dimensional Tensor with length \(N\), the unreduced loss (i.e. with argument reduction set to ‘none’) of \(x\) and \(y\) is given as:
\[L(x, y) = \{l_1,\dots,l_N\}, \quad \text{with} \quad l_n = (x_n - y_n)^2.\]When argument reduction is ‘mean’, the mean value of \(L(x, y)\) will be returned. When argument reduction is ‘sum’, the sum of \(L(x, y)\) will be returned. \(N\) is the batch size.
- Parameters
reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. Default: “mean”.
- Inputs:
input_data (Tensor) - Tensor of shape \((x_1, x_2, ..., x_R)\).
target_data (Tensor) - Tensor of shape \((y_1, y_2, ..., y_S)\).
- Outputs:
Tensor, weighted loss float tensor.
- Supported Platforms:
AscendGPU
Examples
>>> loss = nn.MSELoss() >>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32) >>> output = loss(input_data, target_data) >>> print(output) 0.33333334
-
class
tinyms.losses.SmoothL1Loss(beta=1.0)[source]¶ A loss class for learning region proposals.
SmoothL1Loss can be regarded as modified version of L1Loss or a combination of L1Loss and L2Loss. L1Loss computes the element-wise absolute difference between two input Tensor while L2Loss computes the squared difference between two input Tensor. L2Loss often leads to faster convergence but it is less robust to outliers.
Given two input \(x,\ y\) of length \(N\), the unreduced SmoothL1Loss can be described as follows:
\[\begin{split}L_{i} = \begin{cases} \frac{0.5 (x_i - y_i)^{2}}{\text{beta}}, & \text{if } |x_i - y_i| < \text{beta} \\ |x_i - y_i| - 0.5 \text{beta}, & \text{otherwise. } \end{cases}\end{split}\]Here \(\text{beta}\) controls the point where the loss function changes from quadratic to linear. Its default value is 1.0. \(N\) is the batch size. This function returns an unreduced loss Tensor.
- Parameters
beta (float) – A parameter used to control the point where the function will change from quadratic to linear. Default: 1.0.
- Inputs:
input_data (Tensor) - Tensor of shape \((x_1, x_2, ..., x_R)\).
target_data (Tensor) - Tensor of shape \((y_1, y_2, ..., y_S)\).
- Outputs:
Tensor, loss float tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> loss = nn.SmoothL1Loss() >>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32) >>> output = loss(input_data, target_data) >>> print(output) [0. 0. 0.5]
-
class
tinyms.losses.SoftmaxCrossEntropyWithLogits(sparse=False, reduction='none')[source]¶ Computes softmax cross entropy between logits and labels.
Measures the distribution error between the probabilities of the input (computed with softmax function) and the target where the classes are mutually exclusive (only one class is positive) using cross entropy loss.
Typical input into this function is unnormalized scores and target of each class. Scores Tensor \(x\) is of shape \((N, C)\) and target Tensor \(t\) is a Tensor of shape \((N, C)\) which contains one-hot labels of length \(C\).
For each instance \(N_i\), the loss is given as:
\[\ell(x_i, t_i) = - \log\left(\frac{\exp(x_{t_i})}{\sum_j \exp(x_j)}\right) = -x_{t_i} + \log\left(\sum_j \exp(x_j)\right)\]where \(x_i\) is a 1D score Tensor, \(t_i\) is a scalar.
Note
While the target classes are mutually exclusive, i.e., only one class is positive in the target, the predicted probabilities need not to be exclusive. It is only required that the predicted probability distribution of entry is a valid one.
- Parameters
- Inputs:
logits (Tensor) - Tensor of shape (N, C).
labels (Tensor) - Tensor of shape (N, ). If sparse is True, The type of labels is mindspore.int32. If sparse is False, the type of labels is the same as the type of logits.
- Outputs:
Tensor, a tensor of the same shape as logits with the component-wise logistic losses.
- Supported Platforms:
AscendGPUCPU
Examples
>>> loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True) >>> np.random.seed(0) >>> logits = Tensor(np.random.randint(0, 9, [1, 10]), mindspore.float32) >>> labels_np = np.ones([1,]).astype(np.int32) >>> labels = Tensor(labels_np) >>> output = loss(logits, labels) >>> print(output) [7.868383]
-
class
tinyms.losses.BCELoss(weight=None, reduction='none')[source]¶ BCELoss creates a criterion to measure the binary cross entropy between the true labels and predicted labels.
Set the predicted labels as \(x\), true labels as \(y\), the output loss as \(\ell(x, y)\). Let,
\[L = \{l_1,\dots,l_N\}^\top, \quad l_n = - w_n \left[ y_n \cdot \log x_n + (1 - y_n) \cdot \log (1 - x_n) \right]\]Then,
\[\begin{split}\ell(x, y) = \begin{cases} L, & \text{if reduction} = \text{`none';}\\ \operatorname{mean}(L), & \text{if reduction} = \text{`mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{`sum'.} \end{cases}\end{split}\]Note
Note that the predicted labels should always be the output of sigmoid and the true labels should be numbers between 0 and 1.
- Parameters
- Inputs:
inputs (Tensor) - The input Tensor. The data type must be float16 or float32.
labels (Tensor) - The label Tensor which has same shape and data type as inputs.
- Outputs:
Tensor or Scalar, if reduction is ‘none’, then output is a tensor and has the same shape as inputs. Otherwise, the output is a scalar.
- Supported Platforms:
AscendGPU
Examples
>>> weight = Tensor(np.array([[1.0, 2.0, 3.0], [4.0, 3.3, 2.2]]), mindspore.float32) >>> loss = nn.BCELoss(weight=weight, reduction='mean') >>> inputs = Tensor(np.array([[0.1, 0.2, 0.3], [0.5, 0.7, 0.9]]), mindspore.float32) >>> labels = Tensor(np.array([[0, 1, 0], [0, 0, 1]]), mindspore.float32) >>> output = loss(inputs, labels) >>> print(output) 1.8952923
-
class
tinyms.losses.CosineEmbeddingLoss(margin=0.0, reduction='mean')[source]¶ Computes the similarity between two tensors using cosine distance.
Given two tensors x1, x2, and a Tensor label y with values 1 or -1:
\[\begin{split}loss(x_1, x_2, y) = \begin{cases} 1-cos(x_1, x_2), & \text{if } y = 1\\ max(0, cos(x_1, x_2)-margin), & \text{if } y = -1\\ \end{cases}\end{split}\]- Parameters
- Inputs:
input_x1 (Tensor) - Input tensor.
input_x2 (Tensor) - Its shape and data type must be the same as input_x1’s shape and data type.
y (Tensor) - Contains value 1 or -1. Suppose the shape of input_x1 is \((x_1, x_2, x_3,..., x_R)\), then the shape of target must be \((x_1, x_3, x_4, ..., x_R)\).
- Outputs:
loss (Tensor) - If reduction is “none”, its shape is the same as y’s shape, otherwise a scalar value will be returned.
- Supported Platforms:
AscendGPU
Examples
>>> x1 = Tensor(np.array([[0.3, 0.8], [0.4, 0.3]]), mindspore.float32) >>> x2 = Tensor(np.array([[0.4, 1.2], [-0.4, -0.9]]), mindspore.float32) >>> y = Tensor(np.array([1,-1]), mindspore.int32) >>> cosine_embedding_loss = nn.CosineEmbeddingLoss() >>> output = cosine_embedding_loss(x1, x2, y) >>> print(output) 0.0003426075
-
class
tinyms.losses.SampledSoftmaxLoss(num_sampled, num_classes, num_true=1, sampled_values=None, remove_accidental_hits=True, seed=0, reduction='none')[source]¶ Computes the sampled softmax training loss.
- Parameters
num_sampled (int) – The number of classes to randomly sample per batch.
num_classes (int) – The number of possible classes.
num_true (int) – The number of target classes per training example.
sampled_values (Tuple) – Tuple of (sampled_candidates, true_expected_count, sampled_expected_count) returned by a *CandidateSampler function. Default to None, UniformCandidateSampler is applied.
remove_accidental_hits (bool) – Whether to remove “accidental hits” where a sampled class equals one of the target classes. Default is True.
seed (int) – Random seed for candidate sampling. Default: 0
reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. If “none”, do not perform reduction. Default: “none”.
- Inputs:
weights (Tensor) - Tensor of shape (C, dim).
bias (Tensor) - Tensor of shape (C). The class biases.
labels (Tensor) - Tensor of shape (N, num_true), type int64, int32. The target classes.
inputs (Tensor) - Tensor of shape (N, dim). The forward activations of the input network.
- Outputs:
Tensor, a tensor of shape (N) with the per-example sampled softmax losses.
- Supported Platforms:
GPU
Examples
>>> mindspore.set_seed(1) >>> loss = nn.SampledSoftmaxLoss(num_sampled=4, num_classes=7, num_true=1) >>> weights = Tensor(np.random.randint(0, 9, [7, 10]), mindspore.float32) >>> biases = Tensor(np.random.randint(0, 9, [7]), mindspore.float32) >>> labels = Tensor([0, 1, 2]) >>> inputs = Tensor(np.random.randint(0, 9, [3, 10]), mindspore.float32) >>> output = loss(weights, biases, labels, inputs) >>> print(output) [4.6051701e+01 1.4000047e+01 6.1989022e-06]
tinyms.optimizers¶
Optimizers module provides common optimizers for training, such as SGD, ADAM, Momentum. The optimizer is used to calculate and update the gradients.
-
class
tinyms.optimizers.Optimizer(learning_rate, parameters, weight_decay=0.0, loss_scale=1.0)[source]¶ Base class for all optimizers.
Note
This class defines the API to add Ops to train a model. Never use this class directly, but instead instantiate one of its subclasses.
Different parameter groups can set different learning_rate and weight_decay.
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight_decay is positive. For most optimizer, when not separating parameters, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float.
parameters (Union[list[Parameter], list[dict]]) –
When the parameters is a list of Parameter which will be updated, the element in parameters must be class Parameter. When the parameters is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
weight_decay (float) – A floating point value for the weight decay. It must be equal to or greater than 0. If the type of weight_decay input is int, it will be converted to float. Default: 0.0.
loss_scale (float) – A floating point value for the loss scale. It must be greater than 0. If the type of loss_scale input is int, it will be converted to float. Default: 1.0.
- Raises
ValueError – If the learning_rate is a Tensor, but the dimension of tensor is greater than 1.
TypeError – If the learning_rate is not any of the three types: float, Tensor, nor Iterable.
- Supported Platforms:
AscendGPU
-
broadcast_params(optim_result)[source]¶ Apply Broadcast operations in the sequential order of parameter groups.
- Returns
bool, the status flag.
-
decay_weight(gradients)[source]¶ Weight decay.
An approach to reduce the overfitting of a deep learning neural network model.
-
get_lr()[source]¶ Get the learning rate of current step.
- Returns
float, the learning rate of current step.
-
scale_grad(gradients)[source]¶ Loss scale for mixed precision.
An approach of mixed precision training to improve the speed and energy efficiency of training deep neural network.
-
property
target¶ The method is used to determine whether the parameter is updated on host or device. The input type is str and can only be ‘CPU’, ‘Ascend’ or ‘GPU’.
-
property
unique¶ The method is to see whether to make unique. The input type is bool. The method is read-only.
-
class
tinyms.optimizers.Momentum(params, learning_rate, momentum, weight_decay=0.0, loss_scale=1.0, use_nesterov=False)[source]¶ Implements the Momentum algorithm.
Refer to the paper on the importance of initialization and momentum in deep learning for more details.
\[v_{t} = v_{t-1} \ast u + gradients\]If use_nesterov is True:
\[p_{t} = p_{t-1} - (grad \ast lr + v_{t} \ast u \ast lr)\]If use_nesterov is Flase:
\[p_{t} = p_{t-1} - lr \ast v_{t}\]Here: where grad, lr, p, v and u denote the gradients, learning_rate, params, moments, and momentum respectively.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float.
momentum (float) – Hyperparameter of type float, means momentum for the moving average. It must be at least 0.0.
weight_decay (int, float) – Weight decay (L2 penalty). It must be equal to or greater than 0.0. Default: 0.0.
loss_scale (int, float) – A floating point value for the loss scale. It must be greater than 0.0. Default: 1.0.
use_nesterov (bool) – Enable Nesterov momentum. Default: False.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
tuple[bool], all elements are True.
- Raises
ValueError – If the momentum is less than 0.0.
TypeError – If the momentum is not a float or use_nesterov is not a bool.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.Momentum(group_params, learning_rate=0.1, momentum=0.9, weight_decay=0.0) >>> # The conv_params's parameters will use a learning rate of default value 0.1 and a weight decay of 0.01. >>> # The no_conv_params's parameters will use a learning rate of 0.01 and a weight decay of default value 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim, metrics=None)
-
class
tinyms.optimizers.LARS(optimizer, epsilon=1e-05, coefficient=0.001, use_clip=False, lars_filter=<function LARS.<lambda>>)[source]¶ Implements the LARS algorithm with LARSUpdate Operator.
LARS is an optimization algorithm employing a large batch optimization technique. Refer to paper LARGE BATCH TRAINING OF CONVOLUTIONAL NETWORKS.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ \lambda = \frac{\theta \text{ * } || \omega || }{|| g_{t} || \text{ + } \delta \text{ * } || \omega || } \\ \lambda = \begin{cases} \min(\frac{\lambda}{\alpha }, 1) & \text{ if } clip = True \\ \lambda & \text{ otherwise } \end{cases}\\ g_{t+1} = \lambda * (g_{t} + \delta * \omega) \end{array}\end{split}\]\(\theta\) represents coefficient, \(\omega\) represents parameters, \(g\) represents gradients, \(t\) represents updateing step, \(\delta\) represents weight_decay, \(\alpha\) represents learning_rate, \(clip\) represents use_clip.
- Parameters
optimizer (Optimizer) – MindSpore optimizer for which to wrap and modify gradients.
epsilon (float) – Term added to the denominator to improve numerical stability. Default: 1e-05.
coefficient (float) – Trust coefficient for calculating the local learning rate. Default: 0.001.
use_clip (bool) – Whether to use clip operation for calculating the local learning rate. Default: False.
lars_filter (Function) – A function to determine whether apply the LARS algorithm. Default: lambda x: ‘LayerNorm’ not in x.name and ‘bias’ not in x.name.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params in the optimizer, the shape is the as same as the params in the optimizer.
- Outputs:
Union[Tensor[bool], tuple[Parameter]], it depends on the output of optimizer.
- Supported Platforms:
Ascend
Examples
>>> net = Net() >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> opt = nn.Momentum(net.trainable_params(), 0.1, 0.9) >>> opt_lars = nn.LARS(opt, epsilon=1e-08, coefficient=0.02) >>> model = Model(net, loss_fn=loss, optimizer=opt_lars, metrics=None)
-
class
tinyms.optimizers.Adam(params, learning_rate=0.001, beta1=0.9, beta2=0.999, eps=1e-08, use_locking=False, use_nesterov=False, weight_decay=0.0, loss_scale=1.0)[source]¶ Updates gradients by the Adaptive Moment Estimation (Adam) algorithm.
The Adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ w = w - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector moment1, \(v\) represents the 2nd moment vector moment2, \(g\) represents gradients, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents params, \(\epsilon\) represents eps.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters is supported.
The sparse strategy is applied while the SparseGatherV2 operator is used for forward network. The sparse feature is under continuous development. If the sparse strategy wants to be executed on the host, set the target to the CPU.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” is in the keys, the value of the corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” is in the keys, the value of the corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” is in the keys, the value must be the order of parameters and the order will be followed in the optimizer. There are no other keys in the dict and the parameters which in the ‘order_params’ must be in one of group parameters.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use the dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 1e-3.
beta1 (float) – The exponential decay rate for the 1st moment estimations. Should be in range (0.0, 1.0). Default: 0.9.
beta2 (float) – The exponential decay rate for the 2nd moment estimations. Should be in range (0.0, 1.0). Default: 0.999.
eps (float) – Term added to the denominator to improve numerical stability. Should be greater than 0. Default: 1e-8.
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
weight_decay (float) – Weight decay (L2 penalty). It must be equal to or greater than 0. Default: 0.0.
loss_scale (float) – A floating point value for the loss scale. Should be greater than 0. Default: 1.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
Tensor[bool], the value is True.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.Adam(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.Adam(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01. >>> # The no_conv_params's parameters will use learning rate of 0.01 and defaule weight decay of 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
property
target¶ The method is used to determine whether the parameter is updated on host or device. The input type is str and can only be ‘CPU’, ‘Ascend’ or ‘GPU’.
-
class
tinyms.optimizers.AdamWeightDecay(params, learning_rate=0.001, beta1=0.9, beta2=0.999, eps=1e-06, weight_decay=0.0)[source]¶ Implements the Adam algorithm to fix the weight decay.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” is in the keys, the value of the corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” is in the keys, the value of the corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” is in the keys, the value must be the order of parameters and the order will be followed in the optimizer. There are no other keys in the dict and the parameters which in the ‘order_params’ must be in one of group parameters.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use the dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 1e-3.
beta1 (float) – The exponential decay rate for the 1st moment estimations. Default: 0.9. Should be in range (0.0, 1.0).
beta2 (float) – The exponential decay rate for the 2nd moment estimations. Default: 0.999. Should be in range (0.0, 1.0).
eps (float) – Term added to the denominator to improve numerical stability. Default: 1e-6. Should be greater than 0.
weight_decay (float) – Weight decay (L2 penalty). It must be equal to or greater than 0. Default: 0.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
tuple[bool], all elements are True.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.AdamWeightDecay(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.AdamWeightDecay(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
class
tinyms.optimizers.LazyAdam(params, learning_rate=0.001, beta1=0.9, beta2=0.999, eps=1e-08, use_locking=False, use_nesterov=False, weight_decay=0.0, loss_scale=1.0)[source]¶ This optimizer will apply a lazy adam algorithm when gradient is sparse.
The original adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ w = w - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector moment1, \(v\) represents the 2nd moment vector moment2, \(g\) represents gradients, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents params, \(\epsilon\) represents eps.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters can be supported.
The sparse strategy is applied while the SparseGatherV2 operator being used for forward network. The sparse behavior, to be notice, is not equivalent to the original Adam algorithm, as only the current indices parames will be updated. The sparse feature is under continuous development. If the sparse strategy wants to be executed on the host, set the target to the CPU.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr” and “weight_decay” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 1e-3.
beta1 (float) – The exponential decay rate for the 1st moment estimations. Should be in range (0.0, 1.0). Default: 0.9.
beta2 (float) – The exponential decay rate for the 2nd moment estimations. Should be in range (0.0, 1.0). Default: 0.999.
eps (float) – Term added to the denominator to improve numerical stability. Should be greater than 0. Default: 1e-8.
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
weight_decay (float) – Weight decay (L2 penalty). Default: 0.0.
loss_scale (float) – A floating point value for the loss scale. Should be equal to or greater than 1. Default: 1.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
Tensor[bool], the value is True.
- Supported Platforms:
Ascend
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.LazyAdam(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.LazyAdam(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
property
target¶ The method is used to determine whether the parameter is updated on host or device. The input type is str and can only be ‘CPU’, ‘Ascend’ or ‘GPU’.
-
class
tinyms.optimizers.AdamOffload(params, learning_rate=0.001, beta1=0.9, beta2=0.999, eps=1e-08, use_locking=False, use_nesterov=False, weight_decay=0.0, loss_scale=1.0)[source]¶ This optimizer will offload Adam optimizer to host CPU and keep parameters being updated on the device, to minimize the memory cost. Although that would bring about an increase of performance overhead, the optimizer could be used to run a larger model.
The Adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ w = w - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector moment1, \(v\) represents the 2nd moment vector moment2, \(g\) represents gradients, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents params, \(\epsilon\) represents eps.
Note
This optimizer only supports GRAPH_MODE currently.
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters is supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” is in the keys, the value of the corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” is in the keys, the value of the corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” is in the keys, the value must be the order of parameters and the order will be followed in the optimizer. There are no other keys in the dict and the parameters which in the ‘order_params’ must be in one of group parameters.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use the dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 1e-3.
beta1 (float) – The exponential decay rate for the 1st moment estimations. Should be in range (0.0, 1.0). Default: 0.9.
beta2 (float) – The exponential decay rate for the 2nd moment estimations. Should be in range (0.0, 1.0). Default: 0.999.
eps (float) – Term added to the denominator to improve numerical stability. Should be greater than 0. Default: 1e-8.
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
weight_decay (float) – Weight decay (L2 penalty). It must be equal to or greater than 0. Default: 0.0.
loss_scale (float) – A floating point value for the loss scale. Should be greater than 0. Default: 1.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
Tensor[bool], the value is True.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.AdamOffload(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.AdamOffload(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01. >>> # The no_conv_params's parameters will use learning rate of 0.01 and defaule weight decay of 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
class
tinyms.optimizers.Lamb(params, learning_rate, beta1=0.9, beta2=0.999, eps=1e-06, weight_decay=0.0)[source]¶ Lamb Dynamic Learning Rate.
LAMB is an optimization algorithm employing a layerwise adaptive large batch optimization technique. Refer to the paper LARGE BATCH OPTIMIZATION FOR DEEP LEARNING: TRAINING BERT IN 76 MINUTES.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float.
beta1 (float) – The exponential decay rate for the 1st moment estimations. Default: 0.9. Should be in range (0.0, 1.0).
beta2 (float) – The exponential decay rate for the 2nd moment estimations. Default: 0.999. Should be in range (0.0, 1.0).
eps (float) – Term added to the denominator to improve numerical stability. Default: 1e-6. Should be greater than 0.
weight_decay (float) – Weight decay (L2 penalty). Default: 0.0. Should be equal to or greater than 0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
tuple[bool], all elements are True.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.Lamb(params=net.trainable_params(), learning_rate=0.1) >>> >>> #2) Use parameter groups and set different values >>> poly_decay_lr = learning_rate_schedule.PolynomialDecayLR(learning_rate=0.1, end_learning_rate=0.01, ... decay_steps=4, power = 0.5) >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': poly_decay_lr}, ... {'order_params': net.trainable_params(0.01)}] >>> optim = nn.Lamb(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01. >>> # The no_conv_params's parameters will use dynamic learning rate of poly decay learning rate and default >>> # weight decay of 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
class
tinyms.optimizers.SGD(params, learning_rate=0.1, momentum=0.0, dampening=0.0, weight_decay=0.0, nesterov=False, loss_scale=1.0)[source]¶ Implements stochastic gradient descent. Momentum is optional.
Introduction to SGD can be found at https://en.wikipedia.org/wiki/Stochastic_gradient_descent. Nesterov momentum is based on the formula from paper On the importance of initialization and momentum in deep learning.
\[v_{t+1} = u \ast v_{t} + gradient \ast (1-dampening)\]If nesterov is True:
\[p_{t+1} = p_{t} - lr \ast (gradient + u \ast v_{t+1})\]If nesterov is Flase:
\[p_{t+1} = p_{t} - lr \ast v_{t+1}\]To be noticed, for the first step, v_{t+1} = gradient
Here : where p, v and u denote the parameters, accum, and momentum respectively.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 0.1.
momentum (float) – A floating point value the momentum. must be at least 0.0. Default: 0.0.
dampening (float) – A floating point value of dampening for momentum. must be at least 0.0. Default: 0.0.
weight_decay (float) – Weight decay (L2 penalty). It must be equal to or greater than 0. Default: 0.0.
nesterov (bool) – Enables the Nesterov momentum. If use nesterov, momentum must be positive, and dampening must equal to 0.0. Default: False.
loss_scale (float) – A floating point value for the loss scale, which must be larger than 0.0. Default: 1.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
Tensor[bool], the value is True.
- Raises
ValueError – If the momentum, dampening or weight_decay value is less than 0.0.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.SGD(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.SGD(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use a learning rate of default value 0.1 and a weight decay of 0.01. >>> # The no_conv_params's parameters will use a learning rate of 0.01 and a weight decay of default value 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
class
tinyms.optimizers.FTRL(params, initial_accum=0.1, learning_rate=0.001, lr_power=-0.5, l1=0.0, l2=0.0, use_locking=False, loss_scale=1.0, weight_decay=0.0)[source]¶ Implements the FTRL algorithm with ApplyFtrl Operator.
FTRL is an online convex optimization algorithm that adaptively chooses its regularization function based on the loss functions. Refer to paper Adaptive Bound Optimization for Online Convex Optimization. Refer to paper Ad Click Prediction: a View from the Trenches for engineering document.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m_{t+1} = m_{t} + g^2 \\ u_{t+1} = u_{t} + g - \frac{m_{t+1}^\text{-p} - m_{t}^\text{-p}}{\alpha } * \omega_{t} \\ \omega_{t+1} = \begin{cases} \frac{(sign(u_{t+1}) * l1 - u_{t+1})}{\frac{m_{t+1}^\text{-p}}{\alpha } + 2 * l2 } & \text{ if } |u_{t+1}| > l1 \\ 0.0 & \text{ otherwise } \end{cases}\\ \end{array}\end{split}\]\(m\) represents accum, \(g\) represents grads, \(t\) represents updateing step, \(u\) represents linear, \(p\) represents lr_power, \(\alpha\) represents learning_rate, \(\omega\) represents params.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on all of the parameters.
To improve parameter groups performance, the customized order of parameters can be supported.
The sparse strategy is applied while the SparseGatherV2 operator being used for forward network. The sparse feature is under continuous development. If the sparse strategy wants to be executed on the host, set the target to the CPU.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Using different learning rate by separating parameters is currently not supported.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
initial_accum (float) – The starting value for accumulators, must be zero or positive values. Default: 0.1.
learning_rate (float) – The learning rate value, must be zero or positive, dynamic learning rate is currently not supported. Default: 0.001.
lr_power (float) – Learning rate power controls how the learning rate decreases during training, must be less than or equal to zero. Use fixed learning rate if lr_power is zero. Default: -0.5.
l1 (float) – l1 regularization strength, must be greater than or equal to zero. Default: 0.0.
l2 (float) – l2 regularization strength, must be greater than or equal to zero. Default: 0.0.
use_locking (bool) – If true, use locks for updating operation. Default: False.
loss_scale (float) – Value for the loss scale. It must be equal to or greater than 1.0. Default: 1.0.
weight_decay (float) – Weight decay value to multiply weight, must be zero or positive value. Default: 0.0.
- Inputs:
grads (tuple[Tensor]) - The gradients of params in the optimizer, the shape is the same as the params in optimizer.
- Outputs:
tuple[Parameter], the updated parameters, the shape is the same as params.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.FTRL(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params}, ... {'order_params': net.trainable_params()}] >>> optim = nn.FTRL(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use weight decay of 0.01. >>> # The no_conv_params's parameters will use default weight decay of 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
property
target¶ The method is used to determine whether the parameter is updated on host or device. The input type is str and can only be ‘CPU’, ‘Ascend’ or ‘GPU’.
-
class
tinyms.optimizers.RMSProp(params, learning_rate=0.1, decay=0.9, momentum=0.0, epsilon=1e-10, use_locking=False, centered=False, loss_scale=1.0, weight_decay=0.0)[source]¶ Implements Root Mean Squared Propagation (RMSProp) algorithm.
Update params according to the RMSProp algorithm.
The equation is as follows:
\[s_{t} = \rho s_{t-1} + (1 - \rho)(\nabla Q_{i}(w))^2\]\[m_{t} = \beta m_{t-1} + \frac{\eta} {\sqrt{s_{t} + \epsilon}} \nabla Q_{i}(w)\]\[w = w - m_{t}\]The first equation calculates moving average of the squared gradient for each weight. Then dividing the gradient by \(\sqrt{ms_{t} + \epsilon}\).
if centered is True:
\[g_{t} = \rho g_{t-1} + (1 - \rho)\nabla Q_{i}(w)\]\[s_{t} = \rho s_{t-1} + (1 - \rho)(\nabla Q_{i}(w))^2\]\[m_{t} = \beta m_{t-1} + \frac{\eta} {\sqrt{s_{t} - g_{t}^2 + \epsilon}} \nabla Q_{i}(w)\]\[w = w - m_{t}\]where \(w\) represents params, which will be updated. \(g_{t}\) is mean gradients, \(g_{t-1}\) is the last moment of \(g_{t}\). \(s_{t}\) is the mean square gradients, \(s_{t-1}\) is the last moment of \(s_{t}\), \(m_{t}\) is moment, the delta of w, \(m_{t-1}\) is the last moment of \(m_{t}\). \(\rho\) represents decay. \(\beta\) is the momentum term, represents momentum. \(\epsilon\) is a smoothing term to avoid division by zero, represents epsilon. \(\eta\) is learning rate, represents learning_rate. \(\nabla Q_{i}(w)\) is gradients, represents gradients.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 0.1.
decay (float) – Decay rate. Should be equal to or greater than 0. Default: 0.9.
momentum (float) – Hyperparameter of type float, means momentum for the moving average. Should be equal to or greater than 0. Default: 0.0.
epsilon (float) – Term added to the denominator to improve numerical stability. Should be greater than 0. Default: 1e-10.
use_locking (bool) – Whether to enable a lock to protect the variable and accumlation tensors from being updated. Default: False.
centered (bool) – If true, gradients are normalized by the estimated variance of the gradient. Default: False.
loss_scale (float) – A floating point value for the loss scale. Should be greater than 0. Default: 1.0.
weight_decay (float) – Weight decay (L2 penalty). Should be equal to or greater than 0. Default: 0.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
Tensor[bool], the value is True.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.RMSProp(params=net.trainable_params(), learning_rate=0.1) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.RMSProp(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use a learning rate of default value 0.1 and a weight decay of 0.01. >>> # The no_conv_params's parameters will use a learning rate of 0.01 and a weight decay of default value 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
class
tinyms.optimizers.ProximalAdagrad(params, accum=0.1, learning_rate=0.001, l1=0.0, l2=0.0, use_locking=False, loss_scale=1.0, weight_decay=0.0)[source]¶ Implements the ProximalAdagrad algorithm with ApplyProximalAdagrad Operator.
ProximalAdagrad is an online Learning and Stochastic Optimization. Refer to paper Efficient Learning using Forward-Backward Splitting.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters can be supported.
The sparse strategy is applied while the SparseGatherV2 operator being used for forward network. The sparse feature is under continuous development. If the sparse strategy wants to be executed on the host, set the target to the CPU.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
accum (float) – The starting value for accumulators, must be zero or positive values. Default: 0.1.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 0.001.
l1 (float) – l1 regularization strength, must be greater than or equal to zero. Default: 0.0.
l2 (float) – l2 regularization strength, must be greater than or equal to zero. Default: 0.0.
use_locking (bool) – If true, use locks for updating operation. Default: False.
loss_scale (float) – Value for the loss scale. It must be greater than 0.0. Default: 1.0.
weight_decay (float) – Weight decay value to multiply weight, must be zero or positive value. Default: 0.0.
- Inputs:
grads (tuple[Tensor]) - The gradients of params in the optimizer, the shape is the same as the params in optimizer.
- Outputs:
Tensor[bool], the value is True.
- Supported Platforms:
Ascend
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.ProximalAdagrad(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.ProximalAdagrad(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
property
target¶ The method is used to determine whether the parameter is updated on host or device. The input type is str and can only be ‘CPU’, ‘Ascend’ or ‘GPU’.
-
class
tinyms.optimizers.Adagrad(params, accum=0.1, learning_rate=0.001, update_slots=True, loss_scale=1.0, weight_decay=0.0)[source]¶ Implements the Adagrad algorithm with ApplyAdagrad Operator.
Adagrad is an online Learning and Stochastic Optimization. Refer to paper Efficient Learning using Forward-Backward Splitting.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
accum (float) – The starting value for accumulators, must be zero or positive values. Default: 0.1.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 0.001.
update_slots (bool) – If true, update accumulation. Default: True.
loss_scale (float) – Value for the loss scale. It must be greater than 0.0. Default: 1.0.
weight_decay (float) – Weight decay value to multiply weight, must be zero or positive value. Default: 0.0.
- Inputs:
grads (tuple[Tensor]) - The gradients of params in the optimizer, the shape is the same as the params in optimizer.
- Outputs:
Tensor[bool], the value is True.
- Supported Platforms:
AscendCPUGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.Adagrad(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.Adagrad(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
tinyms.callbacks¶
Callback related classes and functions in model training phase.
-
class
tinyms.callbacks.LossTimeMonitor(lr_init=None)[source]¶ Monitor loss and time.
- Parameters
lr_init (numpy.ndarray) – Train learning rate. Default: None.
- Returns
None
Examples
>>> from tinyms import Tensor >>> from tinyms.callbacks import LossTimeMonitor >>> >>> LossTimeMonitor(lr_init=Tensor([0.05] * 100).asnumpy())
-
class
tinyms.callbacks.Callback[source]¶ Abstract base class used to build a callback class. Callbacks are context managers which will be entered and exited when passing into the Model. You can use this mechanism to initialize and release resources automatically.
Callback function will execute some operations in the current step or epoch.
Examples
>>> class Print_info(Callback): >>> def step_end(self, run_context): >>> cb_params = run_context.original_args() >>> print(cb_params.cur_epoch_num) >>> print(cb_params.cur_step_num) >>> >>> print_cb = Print_info() >>> model.train(epoch, dataset, callbacks=print_cb)
-
begin(run_context)[source]¶ Called once before the network executing.
- Parameters
run_context (RunContext) – Include some information of the model.
-
end(run_context)[source]¶ Called once after network training.
- Parameters
run_context (RunContext) – Include some information of the model.
-
epoch_begin(run_context)[source]¶ Called before each epoch beginning.
- Parameters
run_context (RunContext) – Include some information of the model.
-
epoch_end(run_context)[source]¶ Called after each epoch finished.
- Parameters
run_context (RunContext) – Include some information of the model.
-
step_begin(run_context)[source]¶ Called before each epoch beginning.
- Parameters
run_context (RunContext) – Include some information of the model.
-
step_end(run_context)[source]¶ Called after each step finished.
- Parameters
run_context (RunContext) – Include some information of the model.
-
-
class
tinyms.callbacks.LossMonitor(per_print_times=1)[source]¶ Monitor the loss in training.
If the loss is NAN or INF, it will terminate training.
Note
If per_print_times is 0, do not print loss.
- Parameters
per_print_times (int) – Print the loss each every time. Default: 1.
- Raises
ValueError – If print_step is not an integer or less than zero.
-
class
tinyms.callbacks.TimeMonitor(data_size=None)[source]¶ Monitor the time in training.
- Parameters
data_size (int) – Dataset size. Default: None.
-
class
tinyms.callbacks.ModelCheckpoint(prefix='CKP', directory=None, config=None)[source]¶ The checkpoint callback class.
It is called to combine with train process and save the model and network parameters after traning.
- Parameters
prefix (str) – The prefix name of checkpoint files. Default: “CKP”.
directory (str) – The path of the folder which will be saved in the checkpoint file. Default: None.
config (CheckpointConfig) – Checkpoint strategy configuration. Default: None.
- Raises
ValueError – If the prefix is invalid.
TypeError – If the config is not CheckpointConfig type.
-
end(run_context)[source]¶ Save the last checkpoint after training finished.
- Parameters
run_context (RunContext) – Context of the train running.
-
property
latest_ckpt_file_name¶ Return the latest checkpoint path and file name.
-
step_end(run_context)[source]¶ Save the checkpoint at the end of step.
- Parameters
run_context (RunContext) – Context of the train running.
-
class
tinyms.callbacks.SummaryCollector(summary_dir, collect_freq=10, collect_specified_data=None, keep_default_action=True, custom_lineage_data=None, collect_tensor_freq=None, max_file_size=None)[source]¶ SummaryCollector can help you to collect some common information.
It can help you to collect loss, learning late, computational graph and so on. SummaryCollector also enables the summary operator to collect data from a summary file.
Note
Multiple SummaryCollector instances in callback list are not allowed.
Not all information is collected at the training phase or at the eval phase.
SummaryCollector always record the data collected by the summary operator.
SummaryCollector only supports Linux systems.
- Parameters
summary_dir (str) – The collected data will be persisted to this directory. If the directory does not exist, it will be created automatically.
collect_freq (int) – Set the frequency of data collection, it should be greater then zero, and the unit is step. Default: 10. If a frequency is set, we will collect data when (current steps % freq) equals to 0, and the first step will be collected at any time. It is important to note that if the data sink mode is used, the unit will become the epoch. It is not recommended to collect data too frequently, which can affect performance.
collect_specified_data (Union[None, dict]) –
Perform custom operations on the collected data. Default: None. By default, if set to None, all data is collected as the default behavior. You can customize the collected data with a dictionary. For example, you can set {‘collect_metric’: False} to control not collecting metrics. The data that supports control is shown below.
collect_metric: Whether to collect training metrics, currently only the loss is collected. The first output will be treated as the loss and it will be averaged. Optional: True/False. Default: True.
collect_graph: Whether to collect the computational graph. Currently, only training computational graph is collected. Optional: True/False. Default: True.
collect_train_lineage: Whether to collect lineage data for the training phase, this field will be displayed on the lineage page of Mindinsight. Optional: True/False. Default: True.
collect_eval_lineage: Whether to collect lineage data for the evaluation phase, this field will be displayed on the lineage page of Mindinsight. Optional: True/False. Default: True.
collect_input_data: Whether to collect dataset for each training. Currently only image data is supported. Optional: True/False. Default: True.
collect_dataset_graph: Whether to collect dataset graph for the training phase. Optional: True/False. Default: True.
histogram_regular: Collect weight and bias for parameter distribution page and displayed in MindInsight. This field allows regular strings to control which parameters to collect. Default: None, it means only the first five parameters are collected. It is not recommended to collect too many parameters at once, as it can affect performance. Note that if you collect too many parameters and run out of memory, the training will fail.
keep_default_action (bool) – This field affects the collection behavior of the ‘collect_specified_data’ field. Optional: True/False, Default: True. True: it means that after specified data is set, non-specified data is collected as the default behavior. False: it means that after specified data is set, only the specified data is collected, and the others are not collected.
custom_lineage_data (Union[dict, None]) – Allows you to customize the data and present it on the MingInsight lineage page. In the custom data, the type of the key supports str, and the type of value supports str, int and float. Default: None, it means there is no custom data.
collect_tensor_freq (Optional[int]) – The same semantics as the collect_freq, but controls TensorSummary only. Because TensorSummary data is too large to be compared with other summary data, this parameter is used to reduce its collection. By default, The maximum number of steps for collecting TensorSummary data is 20, but it will not exceed the number of steps for collecting other summary data. Default: None, which means to follow the behavior as described above. For example, given collect_freq=10, when the total steps is 600, TensorSummary will be collected 20 steps, while other summary data 61 steps, but when the total steps is 20, both TensorSummary and other summary will be collected 3 steps. Also note that when in parallel mode, the total steps will be splitted evenly, which will affect the number of steps TensorSummary will be collected.
max_file_size (Optional[int]) – The maximum size in bytes of each file that can be written to the disk. Default: None, which means no limit. For example, to write not larger than 4GB, specify max_file_size=4 * 1024**3.
- Raises
ValueError – If the parameter value is not expected.
TypeError – If the parameter type is not expected.
RuntimeError – If an error occurs during data collection.
Examples
>>> # Simple usage: >>> from mindspore.train import Model >>> summary_collector = SummaryCollector(summary_dir='./summary_dir') >>> dataset = get_dataset('/path/to/MNIST') >>> network = LeNet5() >>> model = Model(network) >>> model.train(epoch=1, dataset=dataset, callbacks=summary_collector) >>> >>> # Do not collect metric and collect the first layer parameter, others are collected by default >>> specified={'collect_metric': False, 'histogram_regular': '^conv1.*'} >>> summary_collector = SummaryCollector(summary_dir='./summary_dir', collect_specified_data=specified) >>> model.train(epoch=1, dataset=dataset, callbacks=summary_collector) >>> >>> # Only collect metric, custom lineage data and record data that collected by the summary operator, >>> # others are not collected >>> specified = {'collect_metric': True} >>> summary_collector = SummaryCollector('./summary_dir', >>> collect_specified_data=specified, >>> keep_default_action=False, >>> custom_lineage_data={'version': 'resnet50_v1'} >>> ) >>> model.train(epoch=1, dataset=dataset, callbacks=summary_collector)
-
class
tinyms.callbacks.CheckpointConfig(save_checkpoint_steps=1, save_checkpoint_seconds=0, keep_checkpoint_max=5, keep_checkpoint_per_n_minutes=0, integrated_save=True, async_save=False, saved_network=None)[source]¶ The configuration of model checkpoint.
Note
During the training process, if dataset is transmitted through the data channel, It is suggested to set ‘save_checkpoint_steps’ to an integer multiple of loop_size. Otherwise, the time to save the checkpoint may be biased.
- Parameters
save_checkpoint_steps (int) – Steps to save checkpoint. Default: 1.
save_checkpoint_seconds (int) – Seconds to save checkpoint. Default: 0. Can’t be used with save_checkpoint_steps at the same time.
keep_checkpoint_max (int) – Maximum number of checkpoint files can be saved. Default: 5.
keep_checkpoint_per_n_minutes (int) – Keep one checkpoint every n minutes. Default: 0. Can’t be used with keep_checkpoint_max at the same time.
integrated_save (bool) – Whether to perform integrated save function in automatic model parallel scene. Default: True. Integrated save function is only supported in automatic parallel scene, not supported in manual parallel.
async_save (bool) – Whether asynchronous execution saves the checkpoint to a file. Default: False.
saved_network (Cell) – Network to be saved in checkpoint file. If the saved_network has no relation with the network in training, the initial value of saved_network will be saved. Default: None.
- Raises
ValueError – If the input_param is None or 0.
Examples
>>> class LeNet5(nn.Cell): >>> def __init__(self, num_class=10, num_channel=1): >>> super(LeNet5, self).__init__() >>> self.conv1 = nn.Conv2d(num_channel, 6, 5, pad_mode='valid') >>> self.conv2 = nn.Conv2d(6, 16, 5, pad_mode='valid') >>> self.fc1 = nn.Dense(16 * 5 * 5, 120, weight_init=Normal(0.02)) >>> self.fc2 = nn.Dense(120, 84, weight_init=Normal(0.02)) >>> self.fc3 = nn.Dense(84, num_class, weight_init=Normal(0.02)) >>> self.relu = nn.ReLU() >>> self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) >>> self.flatten = nn.Flatten() >>> >>> def construct(self, x): >>> x = self.max_pool2d(self.relu(self.conv1(x))) >>> x = self.max_pool2d(self.relu(self.conv2(x))) >>> x = self.flatten(x) >>> x = self.relu(self.fc1(x)) >>> x = self.relu(self.fc2(x)) >>> x = self.fc3(x) >>> return x >>> >>> net = LeNet5() >>> loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean') >>> optim = nn.Momentum(net.trainable_params(), 0.01, 0.9) >>> model = Model(net, loss_fn=loss, optimizer=optim) >>> data_path = './MNIST_Data' >>> dataset = create_dataset(data_path) >>> config = CheckpointConfig(saved_network=net) >>> ckpoint_cb = ModelCheckpoint(prefix='LeNet5', directory='./checkpoint', config=config) >>> model.train(10, dataset, callbacks=ckpoint_cb)
-
property
async_save¶ Get the value of _async_save.
-
property
integrated_save¶ Get the value of _integrated_save.
-
property
keep_checkpoint_max¶ Get the value of _keep_checkpoint_max.
-
property
keep_checkpoint_per_n_minutes¶ Get the value of _keep_checkpoint_per_n_minutes.
-
property
save_checkpoint_seconds¶ Get the value of _save_checkpoint_seconds.
-
property
save_checkpoint_steps¶ Get the value of _save_checkpoint_steps.
-
property
saved_network¶ Get the value of _saved_network
-
class
tinyms.callbacks.RunContext(original_args)[source]¶ Provides information about the model.
Provides information about original request to model function. Callback objects can stop the loop by calling request_stop() of run_context.
- Parameters
original_args (dict) – Holding the related information of model.
-
get_stop_requested()[source]¶ Returns whether a stop is requested or not.
- Returns
bool, if true, model.train() stops iterations.
-
class
tinyms.callbacks.LearningRateScheduler(learning_rate_function)[source]¶ Change the learning_rate during training.
Note
This class are not supported on CPU.
- Parameters
learning_rate_function (Function) – The function about how to change the learning rate during training.
Examples
>>> from _lr_scheduler_callback import LearningRateScheduler >>> import mindspore.nn as nn >>> from mindspore.train import Model ... >>> def learning_rate_function(lr, cur_step_num): ... if cur_step_num%1000 == 0: ... lr = lr*0.1 ... return lr ... >>> lr = 0.1 >>> momentum = 0.9 >>> net = Net() >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> optim = nn.Momentum(net.trainable_params(), learning_rate=lr, momentum=momentum) >>> model = Model(net, loss_fn=loss, optimizer=optim) ... >>> dataset = create_custom_dataset("custom_dataset_path") >>> model.train(1, dataset, callbacks=[LearningRateScheduler(learning_rate_function)], ... dataset_sink_mode=False)
tinyms.metrics¶
Metrics module provides functions to measure the performance of the machine learning models on the evaluation dataset. It’s used to choose the best model.
-
tinyms.metrics.names()[source]¶ Gets the names of the metric methods.
- Returns
List, the name list of metric methods.
-
tinyms.metrics.get_metric_fn(name, *args, **kwargs)[source]¶ Gets the metric method based on the input name.
- Parameters
name (str) – The name of metric method. Refer to the ‘__factory__’ object for the currently supported metrics.
args – Arguments for the metric function.
kwargs – Keyword arguments for the metric function.
- Returns
Metric object, class instance of the metric method.
Examples
>>> metric = nn.get_metric_fn('precision', eval_type='classification')
-
class
tinyms.metrics.Accuracy(eval_type='classification')[source]¶ Calculates the accuracy for classification and multilabel data.
The accuracy class creates two local variables, the correct number and the total number that are used to compute the frequency with which predictions matches labels. This frequency is ultimately returned as the accuracy: an idempotent operation that simply divides the correct number by the total number.
\[\text{accuracy} =\frac{\text{true_positive} + \text{true_negative}} {\text{true_positive} + \text{true_negative} + \text{false_positive} + \text{false_negative}}\]- Parameters
eval_type (str) – Metric to calculate the accuracy over a dataset, for classification (single-label), and multilabel (multilabel classification). Default: ‘classification’.
Examples
>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]]), mindspore.float32) >>> y = Tensor(np.array([1, 0, 1]), mindspore.float32) >>> metric = nn.Accuracy('classification') >>> metric.clear() >>> metric.update(x, y) >>> accuracy = metric.eval() >>> print(accuracy) 0.6666666666666666
-
eval()[source]¶ Computes the accuracy.
- Returns
Float, the computed result.
- Raises
RuntimeError – If the sample size is 0.
-
update(*inputs)[source]¶ Updates the internal evaluation result \(y_{pred}\) and \(y\).
- Parameters
inputs – Input y_pred and y. y_pred and y are a Tensor, a list or an array. For the ‘classification’ evaluation type, y_pred is in most cases (not strictly) a list of floating numbers in range \([0, 1]\) and the shape is \((N, C)\), where \(N\) is the number of cases and \(C\) is the number of categories. Shape of y can be \((N, C)\) with values 0 and 1 if one-hot encoding is used or the shape is \((N,)\) with integer values if index of category is used. For ‘multilabel’ evaluation type, y_pred and y can only be one-hot encoding with values 0 or 1. Indices with 1 indicate the positive category. The shape of y_pred and y are both \((N, C)\).
- Raises
ValueError – If the number of the inputs is not 2.
-
class
tinyms.metrics.MAE[source]¶ Calculates the mean absolute error.
Creates a criterion that measures the mean absolute error (MAE) between each element in the input: \(x\) and the target: \(y\).
\[\text{MAE} = \frac{\sum_{i=1}^n \|y_i - x_i\|}{n}\]Here \(y_i\) is the prediction and \(x_i\) is the true value.
Note
The method update must be called with the form update(y_pred, y).
Examples
>>> x = Tensor(np.array([0.1, 0.2, 0.6, 0.9]), mindspore.float32) >>> y = Tensor(np.array([0.1, 0.25, 0.7, 0.9]), mindspore.float32) >>> error = nn.MAE() >>> error.clear() >>> error.update(x, y) >>> result = error.eval() >>> print(result) 0.037499990314245224
-
eval()[source]¶ Computes the mean absolute error.
- Returns
Float, the computed result.
- Raises
RuntimeError – If the number of the total samples is 0.
-
update(*inputs)[source]¶ Updates the internal evaluation result \(y_{pred}\) and \(y\).
- Parameters
inputs – Input y_pred and y for calculating mean absolute error where the shape of y_pred and y are both N-D and the shape are the same.
- Raises
ValueError – If the number of the input is not 2.
-
-
class
tinyms.metrics.MSE[source]¶ Measures the mean squared error.
Creates a criterion that measures the mean squared error (squared L2 norm) between each element in the input: \(x\) and the target: \(y\).
\[\text{MSE}(x,\ y) = \frac{\sum_{i=1}^n(y_i - x_i)^2}{n}\]where \(n\) is batch size.
Examples
>>> x = Tensor(np.array([0.1, 0.2, 0.6, 0.9]), mindspore.float32) >>> y = Tensor(np.array([0.1, 0.25, 0.5, 0.9]), mindspore.float32) >>> error = nn.MSE() >>> error.clear() >>> error.update(x, y) >>> result = error.eval()
-
eval()[source]¶ Compute the mean squared error.
- Returns
Float, the computed result.
- Raises
RuntimeError – If the number of samples is 0.
-
update(*inputs)[source]¶ Updates the internal evaluation result \(y_{pred}\) and \(y\).
- Parameters
inputs – Input y_pred and y for calculating mean square error where the shape of y_pred and y are both N-D and the shape are the same.
- Raises
ValueError – If the number of input is not 2.
-
-
class
tinyms.metrics.Metric[source]¶ Base class of metric.
Note
For examples of subclasses, please refer to the definition of class MAE, ‘Recall’ etc.
-
abstract
clear()[source]¶ An interface describes the behavior of clearing the internal evaluation result.
Note
All subclasses must override this interface.
-
abstract
-
class
tinyms.metrics.Precision(eval_type='classification')[source]¶ Calculates precision for classification and multilabel data.
The precision function creates two local variables, \(\text{true_positive}\) and \(\text{false_positive}\), that are used to compute the precision. This value is ultimately returned as the precision, an idempotent operation that simply divides \(\text{true_positive}\) by the sum of \(\text{true_positive}\) and \(\text{false_positive}\).
\[\text{precision} = \frac{\text{true_positive}}{\text{true_positive} + \text{false_positive}}\]Note
In the multi-label cases, the elements of \(y\) and \(y_{pred}\) must be 0 or 1.
- Parameters
eval_type (str) – Metric to calculate accuracy over a dataset, for classification or multilabel. Default: ‘classification’.
Examples
>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]])) >>> y = Tensor(np.array([1, 0, 1])) >>> metric = nn.Precision('classification') >>> metric.clear() >>> metric.update(x, y) >>> precision = metric.eval() >>> print(precision) [0.5 1. ]
-
eval(average=False)[source]¶ Computes the precision.
- Parameters
average (bool) – Specify whether calculate the average precision. Default value is False.
- Returns
Float, the computed result.
-
update(*inputs)[source]¶ Updates the internal evaluation result with y_pred and y.
- Parameters
inputs – Input y_pred and y. y_pred and y are Tensor, list or numpy.ndarray. For ‘classification’ evaluation type, y_pred is in most cases (not strictly) a list of floating numbers in range \([0, 1]\) and the shape is \((N, C)\), where \(N\) is the number of cases and \(C\) is the number of categories. Shape of y can be \((N, C)\) with values 0 and 1 if one-hot encoding is used or the shape is \((N,)\) with integer values if index of category is used. For ‘multilabel’ evaluation type, y_pred and y can only be one-hot encoding with values 0 or 1. Indices with 1 indicate positive category. The shape of y_pred and y are both \((N, C)\).
- Raises
ValueError – If the number of input is not 2.
-
class
tinyms.metrics.Recall(eval_type='classification')[source]¶ Calculates recall for classification and multilabel data.
The recall class creates two local variables, \(\text{true_positive}\) and \(\text{false_negative}\), that are used to compute the recall. This value is ultimately returned as the recall, an idempotent operation that simply divides \(\text{true_positive}\) by the sum of \(\text{true_positive}\) and \(\text{false_negative}\).
\[\text{recall} = \frac{\text{true_positive}}{\text{true_positive} + \text{false_negative}}\]Note
In the multi-label cases, the elements of \(y\) and \(y_{pred}\) must be 0 or 1.
- Parameters
eval_type (str) – Metric to calculate the recall over a dataset, for classification or multilabel. Default: ‘classification’.
Examples
>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]])) >>> y = Tensor(np.array([1, 0, 1])) >>> metric = nn.Recall('classification') >>> metric.clear() >>> metric.update(x, y) >>> recall = metric.eval() >>> print(recall) [1. 0.5]
-
eval(average=False)[source]¶ Computes the recall.
- Parameters
average (bool) – Specify whether calculate the average recall. Default value is False.
- Returns
Float, the computed result.
-
update(*inputs)[source]¶ Updates the internal evaluation result with y_pred and y.
- Parameters
inputs – Input y_pred and y. y_pred and y are a Tensor, a list or an array. For ‘classification’ evaluation type, y_pred is in most cases (not strictly) a list of floating numbers in range \([0, 1]\) and the shape is \((N, C)\), where \(N\) is the number of cases and \(C\) is the number of categories. Shape of y can be \((N, C)\) with values 0 and 1 if one-hot encoding is used or the shape is \((N,)\) with integer values if index of category is used. For ‘multilabel’ evaluation type, y_pred and y can only be one-hot encoding with values 0 or 1. Indices with 1 indicate positive category. The shape of y_pred and y are both \((N, C)\).
- Raises
ValueError – If the number of input is not 2.
-
class
tinyms.metrics.Fbeta(beta)[source]¶ Calculates the fbeta score.
Fbeta score is a weighted mean of precison and recall.
\[F_\beta=\frac{(1+\beta^2) \cdot true\_positive} {(1+\beta^2) \cdot true\_positive +\beta^2 \cdot false\_negative + false\_positive}\]Examples
>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]])) >>> y = Tensor(np.array([1, 0, 1])) >>> metric = nn.Fbeta(1) >>> metric.clear() >>> metric.update(x, y) >>> fbeta = metric.eval() >>> print(fbeta) [0.66666667 0.66666667]
-
eval(average=False)[source]¶ Computes the fbeta.
- Parameters
average (bool) – Whether to calculate the average fbeta. Default value is False.
- Returns
Float, computed result.
-
update(*inputs)[source]¶ Updates the internal evaluation result y_pred and y.
- Parameters
inputs – Input y_pred and y. y_pred and y are Tensor, list or numpy.ndarray. y_pred is in most cases (not strictly) a list of floating numbers in range \([0, 1]\) and the shape is \((N, C)\), where \(N\) is the number of cases and \(C\) is the number of categories. y contains values of integers. The shape is \((N, C)\) if one-hot encoding is used. Shape can also be \((N,)\) if category index is used.
-
-
class
tinyms.metrics.F1[source]¶ Calculates the F1 score. F1 is a special case of Fbeta when beta is 1. Refer to class
mindspore.nn.Fbetafor more details.\[F_1=\frac{2\cdot true\_positive}{2\cdot true\_positive + false\_negative + false\_positive}\]Examples
>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]])) >>> y = Tensor(np.array([1, 0, 1])) >>> metric = nn.F1() >>> metric.update(x, y) >>> result = metric.eval() >>> print(result) [0.66666667 0.66666667]
-
class
tinyms.metrics.TopKCategoricalAccuracy(k)[source]¶ Calculates the top-k categorical accuracy.
Note
The method update must receive input of the form \((y_{pred}, y)\). If some samples have the same accuracy, the first sample will be chosen.
- Parameters
k (int) – Specifies the top-k categorical accuracy to compute.
- Raises
TypeError – If k is not int.
ValueError – If k is less than 1.
Examples
>>> x = Tensor(np.array([[0.2, 0.5, 0.3, 0.6, 0.2], [0.1, 0.35, 0.5, 0.2, 0.], ... [0.9, 0.6, 0.2, 0.01, 0.3]]), mindspore.float32) >>> y = Tensor(np.array([2, 0, 1]), mindspore.float32) >>> topk = nn.TopKCategoricalAccuracy(3) >>> topk.clear() >>> topk.update(x, y) >>> output = topk.eval() >>> print(output) 0.6666666666666666
-
update(*inputs)[source]¶ Updates the internal evaluation result y_pred and y.
- Parameters
inputs – Input y_pred and y. y_pred and y are Tensor, list or numpy.ndarray. y_pred is in most cases (not strictly) a list of floating numbers in range \([0, 1]\) and the shape is \((N, C)\), where \(N\) is the number of cases and \(C\) is the number of categories. y contains values of integers. The shape is \((N, C)\) if one-hot encoding is used. Shape can also be \((N,)\) if category index is used.
-
class
tinyms.metrics.Top1CategoricalAccuracy[source]¶ Calculates the top-1 categorical accuracy. This class is a specialized class for TopKCategoricalAccuracy. Refer to class ‘TopKCategoricalAccuracy’ for more details.
Examples
>>> x = Tensor(np.array([[0.2, 0.5, 0.3, 0.6, 0.2], [0.1, 0.35, 0.5, 0.2, 0.], ... [0.9, 0.6, 0.2, 0.01, 0.3]]), mindspore.float32) >>> y = Tensor(np.array([2, 0, 1]), mindspore.float32) >>> topk = nn.Top1CategoricalAccuracy() >>> topk.clear() >>> topk.update(x, y) >>> output = topk.eval() >>> print(output) 0.0
-
class
tinyms.metrics.Top5CategoricalAccuracy[source]¶ Calculates the top-5 categorical accuracy. This class is a specialized class for TopKCategoricalAccuracy. Refer to class ‘TopKCategoricalAccuracy’ for more details.
Examples
>>> x = Tensor(np.array([[0.2, 0.5, 0.3, 0.6, 0.2], [0.1, 0.35, 0.5, 0.2, 0.], ... [0.9, 0.6, 0.2, 0.01, 0.3]]), mindspore.float32) >>> y = Tensor(np.array([2, 0, 1]), mindspore.float32) >>> topk = nn.Top5CategoricalAccuracy() >>> topk.clear() >>> topk.update(x, y) >>> output = topk.eval() >>> print(output) 1.0
-
class
tinyms.metrics.Loss[source]¶ Calculates the average of the loss. If method ‘update’ is called every \(n\) iterations, the result of evaluation will be:
\[loss = \frac{\sum_{k=1}^{n}loss_k}{n}\]Examples
>>> x = Tensor(np.array(0.2), mindspore.float32) >>> loss = nn.Loss() >>> loss.clear() >>> loss.update(x) >>> result = loss.eval()
-
eval()[source]¶ Calculates the average of the loss.
- Returns
Float, the average of the loss.
- Raises
RuntimeError – If the total number is 0.
-
update(*inputs)[source]¶ Updates the internal evaluation result.
- Parameters
inputs – Inputs contain only one element, the element is loss. The dimension of loss must be 0 or 1.
- Raises
ValueError – If the length of inputs is not 1.
ValueError – If the dimensions of loss is not 1.
-
tinyms.serving¶
-
tinyms.serving.list_servables()[source]¶ List the model that is currently served by the backend server.
A GET request will be sent to the server(127.0.0.1:5000) which will then be routed to 127.0.0.1:5000/servables, and the backend servalbe information will be returned to the client.
- Returns
res_body[‘servables’] (str) will be returned, the backend servable information. Error message will be returned and printed if requests.status_code is not ok. ‘Server not started’ will be returned if server is not started
Examples
>>> # Running the quickstart tutorial, after server started and servable json defined >>> list_servables() [{'description': 'This servable hosts a lenet5 model predicting numbers', 'model': {'class_num': 10, 'format': 'ckpt', 'name': 'lenet5'}, 'name': 'lenet5'}]
-
tinyms.serving.predict(img_path, servable_name, dataset_name='mnist', strategy='TOP1_CLASS')[source]¶ Send the predict request to the backend server, get the return value and do the post process
Predict the input image, and get the result. User must specify the image_path, servable_name, dataset_name and output_strategy to get the predict result.
- Parameters
img_path (str) – path to the image
servable_name (str) – the name in servable_json, now supports 6 servables: lenet5, resnet50_imagenet2012, resnet50_cifar10, mobilenetv2, ssd300 and cyclegan_cityscape.
dataset_name (str) – the name of the dataset that is used to train the model, now supports 5 datasets: mnist, imagenet2012, cifar10, voc, cityscape
strategy (str) – the output strategy, for lenet5, resnet50 and mobilenetv2, select between ‘TOP1_CLASS’ and ‘TOP5_CLASS’, for ssd300, only TOP1_CLASS, for cyclegan_cityscape, select between gray2color and color2gray
- Returns
For lenet5, resnet50, mobilenetv2, the output is a string of predict result. For ssd300, the output is a string of bounding boxes coordinates and labels, which can be further processed using ImageViewer function For cyclegan, the output is a numpy of image, which can be transformed to image using Image.fromarray
Examples
>>> # Running the quickstart tutorial, after server started and servable json defined >>> print(predict('/root/7.png', 'lenet5', 'mnist', 'TOP1_CLASS')) TOP1: 7, score: 0.99943381547927856445
-
tinyms.serving.server_started(host='127.0.0.1', port=5000)[source]¶ Detect whether the serving server is started or not.
A bool value of True will be returned if the server is started, else False.
- Parameters
- Returns
A bool value of True(if server started) or False(if server not started).
Examples
>>> # Running the quickstart tutorial, after starting the server >>> if server_started() is True: >>> print(predict(image_path, 'lenet5', 'mnist', strategy))
-
tinyms.serving.start_server(host='127.0.0.1', port=5000)[source]¶ Start the flask server in a subprocess.
Catch the signal of CTRL + D to shutdown, otherwise call shutdown() function to shutdown the server, if the ip and port already in use, server won’t start for a second time.
- Parameters
- Returns
Start the server in a sub process.
Examples
>>> # In the client part >>> start_server() Server starts at host 127.0.0.1, port 5000
-
tinyms.serving.shutdown()[source]¶ Shutdown the flask server.
Search fot the pid of the process running on port 5000, and kill it. This function will be automatically called when SIGINT, SIGHUP and SIGTERM signals catched.
- Returns
A string message of server shutting down or not.
Examples
>>> # In the client part, after predict() >>> shutdown() 'Server shutting down...'
-
tinyms.serving.run_flask(host='127.0.0.1', port=5000)[source]¶ Start the flask server, only be used to trigger starting the flask server in subprocess.
Directly calling this function is not recommended, please use start_server(). Only Error message will be displayed.
- Parameters
- Returns
Server Started
Examples
>>> # In the start_server function >>> cmd = ['python -c "from tinyms.serving import run_flask; run_flask()"'] >>> server_process = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)
Release Roadmap¶
TODO
Contributing Guidelines¶
Firstly a great welcome for anyone who wants to participate in TinyMS community👏. For those who are not familiar with how this community works, these are some guidelines to get you quickly getting started.
Contributor License Agreement¶
It’s required to sign CLA before your first code submission to TinyMS community.
For individual contributor, please refer to ICLA sign page for the detailed information.
Getting Started¶
Fork the repository on GitHub.
Read the README.md and install page for project information and build instructions.
Try our first quick start tutorial in one minutes!😍
Contribution Workflow¶
Code style¶
Please follow this style to make TinyMS easy to review, maintain and develop.
Coding guidelines
The Python coding style suggested by Python PEP 8 Coding Style is adopted in TinyMS community.
Unittest guidelines
The Python unittest style suggested by pytest is adopted in TinyMS community.
Autodoc guidelines
The Autodoc generated style suggested by Sphinx is adopted in TinyMS community.
Fork-Pull development model¶
Fork TinyMS repository
Before submitting code to TinyMS project, please make sure that this project have been forked to your own repository. It means that there will be parallel development between TinyMS repository and your own repository, so be careful to avoid the inconsistency between them.
NOTICE: The default branch name of TinyMS project is
maininstead ofmaster.Clone the remote repository
If you want to download the code to the local machine,
gitis the best choice:git clone https://github.com/{insert_your_forked_repo}/tinyms.git git remote add upstream https://github.com/tinyms-ai/tinyms.git
Develop code locally
To avoid inconsistency between multiple branches, checking out to a new branch for every pull request is
SUGGESTED:git checkout -b {new_branch_name}
NOTICE: Please try to pull the latest code from upstream repository (
git pull upstream main) every time before checking out a new branch.Then you can change the code arbitrarily.
Push the code to the remote repository
After updating the code, you should push the update in the formal way:
git add . git status # Check the update status git commit -m "Your commit title" git commit -s --amend # Add the concrete description of your commit git push origin {new_branch_name}
Pull a request to TinyMS repository
In the last step, your need to pull a compare request between your new branch and TinyMS
mainbranch. After finishing the pull request, the Travis CI will be automatically set up for building test.
Report issues¶
A great way to contribute to the project is to send a detailed report when you encounter an issue. We always appreciate a well-written, thorough bug report, and will thank you for it!🤝
When reporting issues, refer to this format:
What version of env (tinyms, mindspore, os, python etc) are you using?
Is this a BUG REPORT or FEATURE REQUEST?
What happened?
What you expected to happen?
How to reproduce it? (as minimally and precisely as possible)
Special notes for your reviewers?
Issues advisory:
If you find an unclosed issue, which is exactly what you are going to solve, please put some comments on that issue to tell others you would be in charge of it.
If an issue is opened for a while, it’s recommended for contributors to precheck before working on solving that issue.
If you resolve an issue which is reported by yourself, it’s also required to let others know before closing that issue.
Propose PRs¶
Working on your first Pull Request? 📚You can learn how from this free series How to Contribute to an Open Source Project on GitHub📚
When proposing pull requests, please adhere to these rules:
Raise your idea as an issue on GitHub.
If it is a new feature that needs lots of design details, a design proposal should also be submitted.
After reaching consensus in the issue discussions and design proposal reviews, complete the development on the forked repo and submit a PR.
None of PRs is not permitted until it receives 2+ LGTM from approvers. Please NOTICE that approver is NOT allowed to add LGTM on his own PR.
After PR is sufficiently discussed, it will get merged, abandoned or rejected depending on the outcome of the discussion.
PRs advisory:
Any irrelevant changes should be avoided.
Make sure your commit history being ordered.
Always keep your branch up with the master branch.
For bug-fix PRs, make sure all related issues being linked.
Community Support¶
Whenever you feel confused in this community, please be free to reach out the help from TinyMS community with these approaches:
Wechat communication. Add
mindspore0328Wechat ID to ask for help.QQ group. TBD.
Slack channel. Join
tinymschannel in MindSpore Slack to communicate with each other.
Communication¶
TODO
FAQ¶
TODO